Corn abnormal ear detection system based on image recognition

Abstract

The invention relates to the technical field of agricultural intelligent detection, and discloses an abnormal corn ear detection system based on image recognition, and the system comprises an image collection module which is used for collecting field images containing ears in a corn growth period, covering different illumination, angles and shielding scenes, and outputting original RGB image data; the preprocessing module is used for carrying out illumination correction and noise reduction processing on the original image and outputting a feature-enhanced preprocessed image; and the feature extraction module is used for performing multi-scale feature extraction on the pre-processed image through a convolutional neural network to generate a feature map containing cluster contour, color and texture information. The corn abnormal ear detection system based on image recognition aims at solving the problems that corn abnormal ear detection depends on artificial experience, a traditional algorithm is poor in robustness, and edge deployment is missing in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural intelligent detection, and specifically to a maize abnormal ear detection system based on image recognition. Background Art

[0002] During the process of maize planting and harvesting, abnormal ears (such as those affected by pests and diseases, underdeveloped, deformed, etc.) will affect the yield and quality. This detection technology obtains images of maize ears through devices such as high-definition cameras, and uses image recognition algorithms to analyze and process the images.

[0003] There are some deficiencies in the existing technology for detecting maize abnormal ears. Firstly, manual detection relies on experience and is inefficient. Detection personnel need to visually observe the ear morphology such as the length of the bare tip and the integrity of the grains, which is significantly affected by subjective judgment. The missed detection rate is over 20%, and it is impossible to conduct real-time monitoring of large-scale planting areas, making it difficult to adapt to large-scale agricultural production. Secondly, traditional image processing algorithms lack robustness. Existing methods rely on manually designed features and have poor adaptability to complex field environments: the reflection on the ear surface under strong light causes color distortion, the contrast between mildew patches and normal tissues is insufficient under weak light, and when the leaf occlusion rate exceeds 30%, the failure rate of ear contour extraction reaches over 40%, and there is also a lack of effective recognition ability for irregular abnormalities such as abnormal development. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems in the existing technology of maize abnormal ear detection, such as relying on manual experience, poor robustness of traditional algorithms, and lack of edge deployment, and to propose a maize abnormal ear detection system based on image recognition.

[0005] The technical solution for the present invention to solve the above technical problems is as follows:

[0006] A maize abnormal ear detection system based on image recognition, comprising:

[0007] An image acquisition module, used to acquire field images containing ears during the maize growth cycle, covering different lighting, angles, and occlusion scenarios, and output original RGB image data;

[0008] A preprocessing module, used to perform light correction and noise reduction processing on the original image, and output a preprocessed image with enhanced features;

[0009] A feature extraction module, used to perform multi-scale feature extraction on the preprocessed image through a convolutional neural network, and generate a feature map containing ear contour, color, and texture information;

[0010] Anomaly detection model, constructed based on the improved YOLO architecture, integrating an attention mechanism and a multi-scale feature fusion module, used to detect and classify abnormal ears in the feature map, and output the coordinates of the detection box, the abnormal category, and the confidence level;

[0011] Edge inference module, based on the Ascend Atlas 200 IDK A2 developer kit and 310B inference chip, used to perform quantization optimization and hardware acceleration on the anomaly detection model, and realize real-time inference and anomaly determination in the field environment.

[0012] Based on the above technical solutions, the present invention can also be improved as follows.

[0013] Furthermore, the image acquisition module includes a binocular camera or a handheld terminal, the collected image resolution is not less than 1920×1080, the frame rate is ≥30fps, and the types of abnormal ears covered include too long bald tips, missing grains, mildew / insect damage, and abnormal development, where the too long bald tip is defined as the bald tip length exceeding 1 / 3 of the total ear length, and the missing grains are defined as more than 5 consecutive grains missing.

[0014] Furthermore, the preprocessing module includes an adaptive light correction unit and a noise reduction and enhancement unit;

[0015] The adaptive light correction unit uses the CLAHE algorithm to divide the image into 8×8 sub-blocks, and performs contrast-limited histogram equalization on each sub-block. The formula is:

[0016] I ,

[0015] ,

[0014] , norm ,

[0019] ,

[0018] ,

[0017] , ,

[0016] ,

[0020] , , , norm , , , , , ,

[0021] (x,y) = CLAHE(I(x,y), clipLimit = 2.0, gridSize = 8×8)

[0017] The original image pixel point I(x,y), where (x,y) is the image coordinate. The processing process is to divide the image into 8×8 sub-blocks gridSize, and each sub-block independently performs histogram equalization. clipLimit = 2.0 represents the clipping threshold for limiting the histogram, and the output is the normalized image I norm (x,y);

[0018] The noise reduction and enhancement unit uses the bilateral filtering algorithm to retain edge details through the product kernel function of the spatial proximity weight and the pixel similarity weight. The kernel function is:

[0019]

[0020] Ω represents the neighborhood centered on the pixel point (x,y);

[0021] The spatial proximity weight c(i,j) is used to measure the distance between the neighborhood pixel (i,j) and the central pixel (x,y). The formula is The closer the distance, the higher the weight σ c = 3 is the preset standard deviation;

[0022] The pixel similarity weight s(i,j) measures the color difference between neighboring pixels and the central pixel, and the formula is The closer the colors, the higher the weight σ s = 20 is the preset color difference threshold;

[0023] The denoised image I denoise (x,y), by considering both spatial position and color similarity, while removing Gaussian noise, the ear edge is retained.

[0024] Furthermore, the feature extraction module is based on the YOLOv5s network architecture, and the CBAM convolutional block attention module is embedded in the Backbone. The feature map is weighted through channel attention and spatial attention to enhance the feature sensitivity to abnormal ear color variation and contour deformity.

[0025] Furthermore, the Neck part of the anomaly detection model uses the BiFPN bidirectional feature pyramid network to perform weighted fusion on the feature maps at the 32×, 16×, and 8× downsampling scales. Specifically:

[0026]

[0027] The multi-scale feature map set I i , including the 32× large target, 16× medium target, and 8× small target downsampling scales, w i is the initial weight for each scale;

[0028] Through the denominator ∈ + ∑ j w j The weights of each scale are normalized to a probability distribution to prevent gradient disappearance and to prevent the denominator from being zero; a bottom-up connection is added during the fusion process, allowing bidirectional flow between the small-scale feature 8× and the large-scale feature 32×. The weight of the 8× scale is automatically increased to more than 0.6 for small targets with missing grains, and the weight of the 32× scale is automatically increased to more than 0.7 for large targets with overgrown bald tips.

[0029] Furthermore, the loss function of the anomaly detection model includes focal loss and CIoU loss, and the formula is:

[0030] L total = α·L focal + (1 - α)·L CIoU

[0031] Among them, α = 0.9 is the normal / abnormal class weight, and L focal is used to handle class imbalance, and L totalInclude the distance loss, overlap rate loss, and aspect ratio loss of the detection box;

[0032] The training data augmentation strategy of the anomaly detection model includes:

[0033] Rotate the labeled image by ±15°, perform random scaling by 0.8 - 1.2 times, add Gaussian noise with σ = 0.05, and perform dynamic blur processing with a 3×3 kernel;

[0034] Construct a dataset containing 20,000 images, with the ratio of normal ears to abnormal ears being 1:1, covering complex field scenarios with a leaf occlusion rate ≤ 70%, to improve the robustness of the model to the occlusion environment.

[0035] Furthermore, the model quantization process of the edge inference module includes:

[0036] Convert the PyTorch training model to the.om format through the ATC tool, with a fixed input size of 640×640, use FP16 mixed - precision quantization, and compress the model size to less than 15MB;

[0037] Utilize the parallel computing advantage of the NPU unit of the Ascend 310B chip to accelerate the convolution and activation function operations through the ACL computing library, ensuring that the single - frame inference time < 20ms and the power consumption < 10W;

[0038] The edge inference module supports the joint detection of multiple anomaly types, with the detection accuracy for four types of anomalies, namely overly long bare tips, missing grains, mildew / insect damage, and abnormal development, all ≥ 95%, the missed detection rate < 3%, and the detection frame rate ≥ 50fps, meeting the requirements of field real - time monitoring.

[0039] Furthermore, the anomaly determination mechanism of the edge inference module includes:

[0040] Perform Majority Vote on the detection results of three consecutive frames of the same ear to filter out accidental false detections;

[0041] Define the anomaly score function:

[0042]

[0043] where β = 0.7 is the confidence weight, C is the detection confidence. When S ≥ 0.5, it is determined as an abnormal ear, and combined with geometric feature quantization analysis to reduce the false alarm rate of a single threshold.

[0044] Compared with the prior art, the technical solution of this application has the following beneficial technical effects:

[0045] The image acquisition module of the present invention acquires field images containing corn ears during the corn growth cycle, covering different lighting, angle, and occlusion scenarios, and outputs the original RGB image data, realizing the acquisition of image data for a large-scale planting area, providing a basis for subsequent detection, so that abnormal conditions of corn ears can be detected in time, reducing the missed detection rate, not relying on manual experience judgment, improving the detection efficiency. The preprocessing module performs light correction and noise reduction processing on the original image and outputs a preprocessed image with enhanced features, which can improve the quality of the image under complex lighting conditions such as strong light or weak light, reduce the influence of color distortion and insufficient contrast, and improve the extractability of image features. The feature extraction module performs multi-scale feature extraction on the preprocessed image through a convolutional neural network, generating a feature map containing the contour, color, and texture information of the corn ear. Compared with traditional algorithms that rely on manually designed features, the convolutional neural network can automatically learn and extract image features, has better adaptability to complex scenarios such as leaf occlusion, and improves the success rate of corn ear contour extraction. The abnormal detection model is constructed based on an improved YOLO architecture, integrating an attention mechanism and a multi-scale feature fusion module, which can detect and classify abnormal corn ears in the feature map, and output the detection box coordinates, abnormal category, and confidence level. The edge inference module is based on the Ascend Atlas 200 IDK A2 developer kit and 310B inference chip, performs quantization optimization and hardware acceleration on the abnormal detection model, and realizes real-time inference and abnormal determination in the field environment. Description of the Drawings

[0046] Figure 1 It is a structural block diagram of a corn abnormal ear detection system based on image recognition according to the present invention. Detailed Embodiments

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0048] A corn abnormal ear detection system based on image recognition according to the present invention includes:

[0049] An image acquisition module, configured to acquire field images containing corn ears during the corn growth cycle, covering different lighting, angle, and occlusion scenarios, and output the original RGB image data;

[0050] A preprocessing module, configured to perform light correction and noise reduction processing on the original image and output a preprocessed image with enhanced features;

[0051] A feature extraction module, which is used to perform multi-scale feature extraction on the preprocessed image through a convolutional neural network to generate a feature map containing ear contour, color, and texture information;

[0052] An anomaly detection model, which is constructed based on an improved YOLO architecture and integrates an attention mechanism and a multi-scale feature fusion module, and is used to detect and classify abnormal ears in the feature map, and output the coordinates of the detection box, the abnormal category, and the confidence level;

[0053] An edge inference module, based on the Ascend Atlas200IDKA2 developer kit and the 310B inference chip, is used to perform quantization optimization and hardware acceleration on the anomaly detection model to achieve real-time inference and anomaly determination in the field environment.

[0054] In a preferred embodiment of the present invention, it can be further configured that: the image acquisition module includes a binocular camera or a handheld terminal, the resolution of the acquired image is not less than 1920×1080, the frame rate is ≥30fps, and the types of abnormal ears covered include too long bald tips, missing grains, mildew / insect damage, and abnormal development. Among them, too long bald tips are defined as the length of the bald tip exceeding 1 / 3 of the total length of the ear, and missing grains are defined as more than 5 consecutive grains missing. The image acquisition module uses a binocular camera or a handheld terminal, with a resolution not less than 1920×1080 and a frame rate ≥30fps, which can obtain high-definition and smooth images, accurately capture the details of the ear, solve the problems of low efficiency of manual detection and difficulty in covering large areas, cover various abnormal types such as too long bald tips, missing grains, mildew / insect damage, and abnormal development, and clarify the quantization standards for too long bald tips and missing grains, which helps to more accurately identify various abnormalities, overcomes the defects of manual detection relying on experience and being prone to missed detection, and also makes up for the problem of insufficient ability of traditional algorithms to identify irregular abnormalities.

[0055] The binocular camera can be installed on agricultural machinery equipment. As the agricultural machinery moves in the field, it continuously collects images of corn ears. The handheld terminal is convenient for the detection personnel to perform flexible detection in a small area. For example, in an experimental field or a small planting area, the detection personnel can hold the device close to the ear to obtain clear images. In addition, for corn fields with different planting densities and terrains, appropriate devices can be selected according to the actual situation. In areas with a large planting density, the wide-angle advantage of the binocular camera can collect more ear images; in areas with complex terrain where agricultural machinery is difficult to enter, the portability of the handheld terminal can play a greater role.

[0056] In a preferred embodiment of the present invention, it can be further configured that: the preprocessing module includes an adaptive light correction unit and a noise reduction and enhancement unit;

[0057] The adaptive light correction unit uses the CLAHE algorithm to divide the image into 8×8 sub-blocks and perform contrast-limited histogram equalization on each sub-block. The formula is:

[0058] I norm (x, y) = CLAHE(I(x, y), clipLimit = 2.0, gridSize = 8×8)

[0059] The original image pixel I(x, y), where (x, y) is the image coordinate. The processing process is to divide the image into 8×8 sub - blocks gridSize, and each sub - block performs histogram equalization independently. clipLimit = 2.0 represents the clipping threshold for limiting the histogram, and the output is the normalized image I after illumination correction norm (x, y);

[0060] The noise reduction and enhancement unit adopts the bilateral filtering algorithm, and retains edge details through the product kernel function of the spatial proximity weight and the pixel similarity weight. The kernel function is:

[0061]

[0062] Ω represents the neighborhood centered on the pixel point (x, y);

[0063] The spatial proximity weight c(i, j) measures the distance between the neighborhood pixel (i, j) and the central pixel (x, y). The formula is The closer the distance, the higher the weight σ c = 3 is the preset standard deviation;

[0064] The pixel similarity weight s(i, j) measures the color difference between the neighborhood pixel and the central pixel. The formula is The closer the color, the higher the weight σ s = 20 is the preset color difference threshold;

[0065] The denoised image I denoise (x, y), by considering both spatial position and color similarity, while removing Gaussian noise, retains the ear edge. The adaptive illumination correction unit of the pre - processing module adopts the CLAHE algorithm, divides the image into 8×8 sub - blocks for contrast - limited histogram equalization, which can effectively improve the quality of the image under different illumination conditions. Whether it is the problem of surface reflection of the ear under strong light or the insufficient contrast between mildew patches and normal tissues under weak light, the clarity and contrast of the image can be improved through this algorithm, enhancing the distinguishability of abnormal features. The noise reduction and enhancement unit uses the bilateral filtering algorithm, through the product kernel function of the spatial proximity weight and the pixel similarity weight, while removing Gaussian noise, retains the ear edge, avoiding the problem of losing edge details in traditional noise reduction methods, and improving the accuracy of subsequent feature extraction and anomaly detection

[0066] In actual operation, the adaptive light correction unit can automatically adjust parameters according to different light intensities of images. For example, when the light is weak in the morning or evening, the intensity of contrast enhancement is appropriately increased; when the strong light shines directly at noon, the cropping threshold is finely adjusted to prevent image distortion caused by over-enhancement. The noise reduction and enhancement unit can dynamically adjust the preset standard deviation and color difference threshold according to the actual situation of noise in the image. If there is a lot of noise and a large color difference in the image, these thresholds are appropriately increased to better remove noise and retain edges. When processing some images with a lot of noise caused by equipment aging or environmental interference, this unit can play an important role in ensuring that the image quality meets the detection requirements.

[0067] In a preferred embodiment of the present invention, it can be further configured that: the feature extraction module is based on the YOLOv5s network architecture, and the CBAM convolutional block attention module is embedded in the Backbone. The feature map is weighted through channel attention and spatial attention to enhance the feature sensitivity to abnormal ear color variation and contour deformity. The feature extraction module is based on the YOLOv5s network architecture and embeds the CBAM convolutional block attention module. The feature map is weighted through channel attention and spatial attention, which can focus on the key features of abnormal ears. For color variation, such as the color change of mildewed ears, channel attention can enhance the feature expression of the corresponding color channel; for contour deformity, spatial attention can accurately locate the deformed area and strengthen the features of these areas, thus greatly improving the ability to extract abnormal ear features and making up for the deficiencies of traditional algorithms in this regard.

[0068] During the training process, the CBAM convolutional block attention module can dynamically adjust the weights of channel and spatial attention according to different abnormal ear samples. For some ears with insignificant color variation but prominent contour deformity, the weight of spatial attention will be relatively increased; for mildewed ears with significant color change and small contour change, the weight of channel attention will be strengthened. In actual application, in the face of the differences in the colors and shapes of corn ears of different varieties, this module can adaptively adjust the attention distribution to ensure accurate extraction of abnormal features. For example, the colors of corn ears of different varieties may vary in depth, and this module can automatically adapt to these differences and accurately identify abnormalities.

[0069] In a preferred embodiment of the present invention, it can be further configured that: the Neck part of the anomaly detection model uses a BiFPN bidirectional feature pyramid network to perform weighted fusion on the feature maps at the 32×, 16×, and 8× downsampling scales. Specifically:

[0070]

[0071] Multi-scale feature map set I i, including downsampling scales of 32× large targets, 16× medium targets, and 8× small targets, w i is the initial weight for each scale;

[0072] Through the denominator ∈ + ∑ j w j Normalize the weights of each scale into a probability distribution to prevent gradient vanishing and prevent the denominator from being zero; Add a bottom-up connection during the fusion process, allowing two-way flow between small-scale features (8×) and large-scale features (32×). Automatically increase the 8× scale weight to more than 0.6 for small targets with missing grains, and automatically increase the 32× scale weight to more than 0.7 for large targets with overgrown bald tips.

[0073] Two-way fusion: Add a bottom-up connection on the basis of the traditional top-down path, allowing two-way flow between small-scale features (such as missing grains) and large-scale features (such as bald tips). For example:

[0074] When detecting small-area wormholes, increase the 8× scale weight to 0.6 to enhance detailed features;

[0075] When detecting large-area deformities, increase the 32× scale weight to 0.7 to strengthen contour features;

[0076] The Neck part of the anomaly detection model uses a BiFPN two-way feature pyramid network to perform weighted fusion on the feature maps of the 32×, 16×, and 8× downsampling scales. It also adds a bottom-up connection to allow two-way flow between small-scale features and large-scale features. When detecting small targets such as missing grains, it can automatically increase the 8× scale weight to more than 0.6 to enhance the ability to capture detailed features; when detecting large targets with overgrown bald tips, it increases the 32× scale weight to more than 0.7 to strengthen contour features. This effectively solves the problem of poor detection effect of traditional algorithms on abnormal targets of different sizes and improves the detection accuracy of various abnormal ear of corn.

[0077] In actual detection, for some corn ears at different growth stages, the size and shape of their abnormal manifestations will be different. In the early stage of corn growth, the ear is smaller, and the abnormality may be manifested as small targets such as missing grains. The BiFPN network can capture these small abnormalities in time; in the later stage of growth, the ear becomes larger, and large target abnormalities such as overgrown bald tips are more likely to appear. At this time, the network can strengthen the detection of large targets. In addition, in a complex field environment, the occlusion situation of different ears will also affect the feature extraction and detection. The BiFPN network can dynamically adjust the weights of the feature maps of each scale according to the actual situation, reduce the influence of occlusion on the detection result, and improve the accuracy and stability of detection.

[0078] In a preferred embodiment of the present invention, it can be further configured that: the loss function of the anomaly detection model includes focal loss and CIoU loss, and the formula is:

[0079] L total = α·L focal +(1 - α)·L CIoU

[0080] where α = 0.9 is the normal / anomaly class weight, and L focal is used to handle class imbalance, and L totel includes the distance loss, overlap rate loss, and aspect ratio loss of the detection box;

[0081] The training data augmentation strategy of the anomaly detection model includes:

[0082] Performing ±15° rotation, 0.8 - 1.2 times random scaling, adding Gaussian noise with σ = 0.05, and 3×3 kernel dynamic blur processing on the annotated images;

[0083] Constructing a dataset containing 20,000 images with a ratio of normal ears to abnormal ears of 1:1, covering complex field scenarios with a leaf occlusion rate ≤ 70%, to enhance the robustness of the model to the occlusion environment. The loss function of the anomaly detection model includes focal loss and CIoU loss. Focal loss can effectively handle the class imbalance problem, making the model pay more attention to the abnormal ears of the minority class and avoiding ignoring the features of abnormal ears due to too many normal ear samples. CIoU loss comprehensively considers the distance loss, overlap rate loss, and aspect ratio loss of the detection box, making the positioning of the detection box more accurate. The training data augmentation strategy improves the robustness of the model to various complex environments and solves the problem of poor adaptability of traditional algorithms by rotating, scaling, adding noise, and performing dynamic blur processing on the annotated images, and constructing a dataset containing 20,000 images with a ratio of normal to abnormal ears of 1:1 and covering complex field scenarios with a leaf occlusion rate ≤ 70%.

[0084] During the training process, the weight parameter of the focal loss can be dynamically adjusted according to the degree of class imbalance in the actual data. If the number of abnormal ear samples is too small, appropriately increase the weight of the abnormal class in the focal loss to make the model pay more attention to the feature learning of abnormal ears. For CIoU loss, in different detection scenarios, according to the size and shape of the ear, adjust the weight ratio of the distance loss, overlap rate loss, and aspect ratio loss. When detecting small kernel - missing anomalies, appropriately increase the weight of the distance loss to more accurately locate small targets; when detecting large abnormal tip - over anomalies, strengthen the weight of the aspect ratio loss to ensure that the detection box can better fit the shape of the large target. In terms of data augmentation, in addition to the existing operations, operations such as image translation and cropping can be added to further enrich the diversity of the dataset and improve the generalization ability of the model.

[0085] In a preferred embodiment of the present invention, it can be further configured that: the model quantization process of the edge inference module includes:

[0086] The PyTorch trained model is converted into the.om format through the ATC tool. The input size is fixed at 640×640, and FP16 mixed-precision quantization is adopted to compress the model size to less than 15MB.

[0087] Taking advantage of the parallel computing of the NPU unit of the Ascend 310B chip, the convolution and activation function operations are accelerated through the ACL computing library to ensure that the single-frame inference time < 20ms and the power consumption < 10W.

[0088] The edge inference module supports the joint detection of multiple anomaly types. The detection accuracy rates for the four types of anomalies, namely too long bald tips, missing grains, mildew / insect damage, and abnormal development, are all ≥ 95%, the missed detection rate < 3%, and the detection frame rate ≥ 50fps, meeting the requirements of field real-time monitoring. The edge inference module converts the PyTorch trained model into the.om format through the ATC tool, adopts FP16 mixed-precision quantization, compresses the model size to less than 15MB, reducing the pressure of model storage and transmission. Taking advantage of the parallel computing of the NPU unit of the Ascend 310B chip and the ACL computing library to accelerate the convolution and activation function operations, it ensures that the single-frame inference time < 20ms and the power consumption < 10W, achieving fast and low-power real-time inference. The detection accuracy rates for the four types of anomalies, namely too long bald tips, missing grains, mildew / insect damage, and abnormal development, are all ≥ 95%, the missed detection rate < 3%, and the detection frame rate ≥ 50fps, meeting the high requirements of field real-time monitoring and overcoming the defects of traditional systems.

[0089] When deploying the edge inference module in the field, considering the network signal differences in different regions, the local storage and offline inference methods can be adopted. For remote areas with poor network signals, the collected image data is first stored in local devices, and the detection results are uploaded when the network resumes or data transmission is carried out regularly. At the same time, to further reduce power consumption, the operating frequency of the chip can be dynamically adjusted according to the light intensity and the working status of agricultural machinery. When the light is sufficient and the agricultural machinery is working continuously, the chip frequency is increased to ensure the detection speed; when it is night or the agricultural machinery pauses work, the chip frequency is reduced to reduce power consumption. In addition, the performance of the edge inference module can be monitored and optimized regularly, and the quantization parameters and computing resource allocation can be adjusted according to the actual usage situation to ensure that the system always maintains an efficient and stable operating state.

[0090] In a preferred embodiment of the present invention, it can be further configured that the anomaly determination mechanism of the edge inference module includes:

[0091] Perform Majority Vote on the detection results of 3 consecutive frames of the same ear to filter out accidental misdetections.

[0092] Define the anomaly score function:

[0093]

[0094] Among them, β = 0.7 is the confidence weight, C is the detection confidence. When S≥0.5, it is determined as an abnormal ear. By combining geometric feature quantification analysis, the false positive rate of a single threshold is reduced. The abnormal determination mechanism of the edge inference module performs Majority Vote voting on the detection results of 3 consecutive frames of the same ear, which can effectively filter accidental misdetections and improve the reliability of the detection results. Define an abnormal score function, combine the confidence weight and the detection confidence, and comprehensively consider the ratio of the number of abnormal feature pixels to the total number of ear pixels. When the score is greater than 0.5, it is determined as an abnormal ear. By combining geometric feature quantification analysis, the false positive rate of a single threshold is further reduced, making the determination of abnormal ears more accurate.

[0095] In practical applications, the confidence weight of the abnormal score function can be adjusted according to different abnormal types. For some abnormalities that are prone to misjudgment, such as mild grain loss, appropriately reduce the confidence weight to avoid false alarms; for abnormalities with obvious characteristics, such as severe mildew / insect damage, increase the confidence weight to ensure accurate determination. In terms of combining geometric feature quantification analysis, geometric parameters such as the area, perimeter, and shape complexity of the abnormal area can be measured. For example, if the detected abnormal area is small but has a regular shape, it may be a normal physiological feature, and misjudgment can be avoided through geometric feature analysis; if the abnormal area is large and has an irregular shape, combined with the abnormal score function, it can be more accurately determined as an abnormal ear. In addition, the abnormal determination results can be compared and analyzed with historical data, and for some suddenly occurring abnormal situations, an alarm can be issued in a timely manner to provide more valuable information for growers.

[0096] During the maize growth cycle, the image acquisition module acquires field images containing ears through a binocular camera or a handheld terminal. The binocular camera is installed on agricultural machinery and can continuously acquire images while moving in the field. The handheld terminal is convenient for flexible detection in a small area. The acquisition resolution is not less than 1920×1080, and the frame rate ≥30fps, covering various types of abnormal ears, and outputting the original RGB image data;

[0097] The acquired original images enter the preprocessing module. Among them, the adaptive illumination correction unit uses the CLAHE algorithm to divide the image into 8×8 sub-blocks for contrast-limited histogram equalization to improve the image quality under different illumination conditions, enhance the distinguishability of abnormal features, and can automatically adjust parameters according to the illumination intensity; the noise reduction and enhancement unit uses the bilateral filtering algorithm, through the product kernel function of spatial proximity and pixel similarity weights, to remove Gaussian noise while retaining the ear edge, and can also dynamically adjust relevant thresholds according to the image noise situation;

[0098] The preprocessed image enters the feature extraction module, which is based on the YOLOv5s network architecture and embeds the CBAM convolutional block attention module. It weights the feature map through channel attention and spatial attention, focuses on the key features of abnormal ears, improves the extraction ability of color variation and contour deformity features, and can dynamically adjust the attention weights according to different abnormal ear samples and maize variety differences;

[0099] After that, the image with ear features enters the anomaly detection model. The Neck part of the model uses the BiFPN bidirectional feature pyramid network to weight and fuse the feature maps at the 32×, 16×, and 8× downsampling scales, increases the bottom-up connection, realizes the bidirectional flow of small-scale and large-scale features, automatically adjusts the corresponding scale weights for abnormal targets of different sizes, and improves the detection accuracy. Its loss function includes focal loss and CIoU loss. The focal loss deals with the class imbalance problem, and the CIoU loss comprehensively considers various factors of the detection box to achieve accurate positioning. At the same time, the model's robustness to complex environments is improved through specific training data augmentation strategies;

[0100] Finally, the results of the anomaly detection model enter the edge inference module. This module first uses the ATC tool to convert the PyTorch training model into the.om format, compresses the model size using FP16 mixed-precision quantization, and utilizes the parallel computing advantage of the NPU unit of the Ascend 310B chip and the ACL computing library to accelerate the operation, achieving real-time inference with a single-frame inference time < 20ms and power consumption < 10W. It has high accuracy, low miss rate, and high detection frame rate for various anomalies. When determining anomalies, it filters out false detections through Majority Vote on the detection results of three consecutive frames of the same ear, uses the anomaly score function to combine geometric feature quantization analysis, adjusts the confidence weight according to different anomaly types, accurately determines abnormal ears, and can also combine historical data for comparative analysis to provide valuable information for growers.

[0101] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0102] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A corn abnormal ear detection system based on image recognition, characterized in that, Including: An image acquisition module, which is used to acquire field images containing corn ears during the corn growth cycle, covering different lighting, angle, and occlusion scenarios, and output the original RGB image data; A preprocessing module, which is used to perform light correction and noise reduction processing on the original image and output a preprocessed image with enhanced features; A feature extraction module, which is used to perform multi-scale feature extraction on the preprocessed image through a convolutional neural network to generate a feature map containing the contour, color, and texture information of the corn ear; An anomaly detection model, which is constructed based on an improved YOLO architecture and integrates an attention mechanism and a multi-scale feature fusion module, and is used to detect and classify abnormal corn ears in the feature map, and output the detection box coordinates, anomaly category, and confidence level; An edge inference module, which is based on the Ascend Atlas200IDKA2 developer kit and the 310B inference chip, and is used to perform quantization optimization and hardware acceleration on the anomaly detection model to achieve real-time inference and anomaly determination in the field environment.

2. The maize abnormal ear detection system based on image recognition according to claim 1, wherein, The image acquisition module includes a binocular camera or a handheld terminal. The resolution of the acquired image is not less than 1920×1080, the frame rate ≥ 30fps, and the types of abnormal corn ears covered include overly long bald tips, missing grains, mildew / insect damage, and abnormal development. Among them, an overly long bald tip is defined as a bald tip length exceeding 1 / 3 of the total length of the corn ear, and missing grains are defined as more than 5 consecutive grains missing.

3. The maize abnormal ear detection system based on image recognition according to claim 1, wherein The preprocessing module includes an adaptive light correction unit and a noise reduction and enhancement unit; The adaptive light correction unit uses the CLAHE algorithm to divide the image into 8×8 sub-blocks and perform contrast-limited histogram equalization on each sub-block. The formula is: I norm (x,y) = CLAHE(I(x,y), clipLimit = 2.0, gridSize = 8×8) Original image pixel I(x, y), where (x, y) are the image coordinates. The processing procedure is to divide the image into 8×8 sub-blocks gridSize, and perform histogram equalization on each sub-block independently. clipLimit = 2.0 represents the clipping threshold for limiting the histogram, and the output is the normalized image I after illumination correction norm (x, y); The noise reduction and enhancement unit uses the bilateral filtering algorithm to retain edge details through the product kernel function of the spatial proximity weight and the pixel similarity weight. The kernel function is: Ω represents the neighborhood centered on the pixel point (x,y); The spatial proximity weight c(i,j) measures the distance between the neighborhood pixel (i,j) and the central pixel (x,y), and the formula is The closer the distance, the higher the weight σ c = 3 is the preset standard deviation; The pixel similarity weight s(i, j) is used to measure the color difference between neighboring pixels and the central pixel, and the formula is The closer the colors are, the higher the weight σ s = 20 is the preset color difference threshold; The image I after noise reduction denoise (x, y), while removing Gaussian noise and retaining the ear edge by simultaneously considering spatial position and color similarity.

4. The maize abnormal ear detection system based on image recognition according to claim 1, wherein, The feature extraction module is based on the YOLOv5s network architecture, and embeds a CBAM convolutional block attention module in the Backbone to weight the feature map through channel attention and spatial attention, enhancing the feature sensitivity to color variation and contour deformity of abnormal corn ears.

5. The maize abnormal ear detection system based on image recognition according to claim 1, characterized in that, The Neck part of the anomaly detection model uses a BiFPN bidirectional feature pyramid network to perform weighted fusion on the feature maps at 32×, 16×, and 8× downsampling scales. Specifically: Multi-scale feature map set I i , including downsampling scales of 32× large targets, 16× medium targets, and 8× small targets, where w i is the initial weight for each scale; By denominator ∈ + ∑ j w j Normalize the weights of each scale into a probability distribution to prevent gradient vanishing ∈ prevent the denominator from being zero; add bottom-up connections during the fusion process to allow bidirectional flow between small-scale features 8× and large-scale features 32×, automatically boost the 8× scale weight to above 0.6 for small missing targets of grains, and automatically boost the 32× scale weight to above 0.7 for overgrown targets with bare tips.

6. The maize abnormal ear detection system based on image recognition according to claim 1, wherein The loss function of the anomaly detection model includes focal loss and CIoU loss. The formula is: L total = α·L focal +(1 - α)·L CIoU where α = 0.9 is the normal / abnormal class weight, L focal is used to handle class imbalance, L total includes the distance loss, overlap ratio loss, and aspect ratio loss of the detection box; The training data augmentation strategy of the anomaly detection model includes: Performing ±15° rotation, 0.8-1.2 times random scaling, adding Gaussian noise with σ = 0.05, and 3×3 kernel dynamic blur processing on the labeled images; Constructing a dataset containing 20,000 images, with the ratio of normal corn ears to abnormal corn ears being 1:1, covering complex field scenarios with a leaf occlusion rate ≤ 70%, and enhancing the robustness of the model to the occlusion environment.

7. The maize abnormal ear detection system based on image recognition according to claim 1, characterized in that, The model quantization process of the edge inference module includes: Converting the PyTorch training model to the.om format through the ATC tool, with the input size fixed at 640×640, using FP16 mixed-precision quantization, and compressing the model size to less than 15MB; Taking advantage of the parallel computing of the NPU unit of the Ascend 310B chip, the convolution and activation function operations are accelerated through the ACL computing library to ensure that the single-frame inference time < 20 ms and the power consumption < 10 W. The edge inference module supports the joint detection of multiple abnormal types, and the detection accuracy of the four types of abnormalities, namely too long bald tips, missing grains, mildew / insect damage, and abnormal development, is ≥ 95%, the missed detection rate < 3%, and the detection frame rate ≥ 50 fps, meeting the requirements of field real-time monitoring.

8. The maize abnormal ear detection system based on image recognition according to claim 1, characterized in that The abnormal determination mechanism of the edge inference module includes: Perform MajorityVote voting on the detection results of 3 consecutive frames of the same ear to filter accidental false detections. Define an abnormal score function: Among them, β = 0.7 is the confidence weight, C is the detection confidence, and when S ≥ 0.5, it is determined as an abnormal ear, and the false alarm rate of a single threshold is reduced by combining geometric feature quantitative analysis.

Images