A small target detection method for corn recycling cleaning operation
Patent Information
- Application Number
- CN202310802550.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-30
- Publication Date
- 2026-09-04
- Estimated Expiration
- 2043-06-30
AI Technical Summary
然而，受限于较少像素导致的视觉特征信息的缺乏，小目标的检测精度相对较低
[0031]本发明提出的方法可视为适应性区域生成网络和剩下的检测器网络两个部分并通过交替优化的方式实现参数的更新。利用ImageNet预训练模型初始化特征学习网络，并对适应性区域生成网络进行端到端的训练。然后，再次利用ImageNet预训练模型对特征学习网络进行初始化并基于适应性区域生成网络生成的候选框来训练检测器网络。最后，两个部分共享相同的卷积层并依次对适应性区域生成网络和检测器网络中特有的层进行微调，从而形成了统一的网络。
Smart Images
Figure CN117456399B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning-based target detection, and in particular to a method for detecting small targets in corn cyclic cleaning operations. Background Technology
[0002] As a globally cultivated and domesticated crop, maize boasts advantages such as strong environmental adaptability, stable yield, and wide range of uses. It is an important source of food and feed, rich in nutrients, and has high comprehensive utilization value, finding widespread application in animal husbandry, aquaculture, and medical and health fields. Based on the significant differences between normal grains and damaged, defective, and impurity grains in terms of shape, specific gravity, volume, and density, a multi-physics field coupling method is used to throw, disperse, transport, and screen the mixture to obtain a purer and more standardized grain crop. The performance of the cleaning system directly affects the grain loss rate and impurity content, and is crucial for subsequent efficient drying, quality-preserving transportation, and safe storage.
[0003] While mechanical equipment can improve cleaning efficiency to some extent, parameter settings still require operators to monitor the entire cleaning process and rely on experience, which is time-consuming and expensive. Therefore, real-time grain sentiment awareness can provide important guidance and reference for the adaptability and dynamic adjustment of cleaning strategies.
[0004] By combining target localization and recognition detection techniques, target detection technology can accurately regress bounding box coordinates and detect the specific category of a target in a given image. It is widely used in fields such as face recognition, medical image processing, and agricultural product processing. Different data augmentation strategies can expand and enrich the scale and diversity of datasets, thereby enhancing the robustness and generalization ability of detection models. Traditional target detection algorithms design corresponding feature extraction modules for different detection objects. However, due to excessive reliance on professional hand-crafted features and the need for manual adjustment of numerous parameters, their robustness and scalability are poor. Target detection technology, with its powerful feature extraction capabilities, is based on deep learning and can adaptively capture deep semantic information of images using multi-structured network models, significantly improving the detection efficiency and accuracy of common targets. However, limited by the lack of visual feature information due to fewer pixels, the detection accuracy of small targets is relatively low. Furthermore, information loss during network forward propagation, uneven sample distribution, and the setting of anchor box parameters all affect the final target classification and coordinate regression results.
[0005] In the cyclic cleaning process of harvested corn, an end-to-end small target detection network was designed with RGB corn images containing impurities as input to provide real-time feedback on the grain condition during the impurity removal process. Based on the type and distribution of impurities, the parameters and strategies were dynamically adjusted in a targeted manner, which has important application value for achieving efficient and low-loss intelligent corn cleaning. Summary of the Invention
[0006] To address the shortcomings of existing technologies, the present invention aims to provide a small-target detection method for cyclic corn cleaning operations, which can perceive the types and distribution of impurities in harvested corn during cyclic cleaning operations from end to end, thereby providing real-time grain condition feedback for the dynamic setting and adjustment of cleaning parameters and strategies.
[0007] To solve the above-mentioned technical problems, the technical solution provided by the present invention is as follows:
[0008] A method for detecting small targets in corn recycling operations includes the following steps:
[0009] (1) An image acquisition module is set at the inlet and outlet of the corn harvesting and cleaning equipment coupled with multi-physics field to capture images of corn containing impurities on the conveyor belt in real time and continuously. After uniformization processing, a dataset is obtained. The dataset is expanded based on the image enhancement method and used as the data input for training and testing the small target detection model. The small target detection model includes: feature learning network, adaptive region generation network and output layer.
[0010] (2) Introduce EfficientNetB7 as the backbone framework of the feature learning network in step (1), embed a cross-level fusion mechanism, perform convolution and upsampling operations on the deep-level feature matrix, and fuse it with the shallow-level feature matrix to generate cross-level fusion features and obtain a cross-level fusion feature map containing multi-scale mapping.
[0011] In step (2), a cross-level feature map containing multi-scale mapping is obtained. The specific steps are as follows:
[0012] (2.1) The deep feature map is transformed in dimension by a convolution operation with a kernel size of 1, and reduced to the same depth as the previous layer feature map to obtain the modified deep feature map;
[0013] (2.2) The shallow feature map is further captured by a convolution operation with a kernel size of 3 to obtain the modified shallow feature map, which is added to the modified deep feature map obtained in step (2.1) and upsampled based on bilinear interpolation;
[0014] (3) Based on the cross-level fusion features generated in step (2), combined with the distribution features and geometric properties of harvested corn kernels and impurities, the adaptive region generation network is used to classify and regress the initial anchor boxes of all targets in the generated full coverage image through continuous convolution operations, providing candidate boxes for the output layer, namely “regions of interest”.
[0015] The aspect ratios of the initial anchor frames are set to 1:1, 1:2, and 2:1, and the area scale is 64. 2 128 2 and 2562 ;
[0016] (4) Map the “region of interest” obtained in step (3) to the cross-level fusion feature obtained in step (2), and arrange two fully connected layers in series after ROI pooling and flattening. Then, classify and regress the final attribute information and coordinate parameters of the candidate box by connecting two fully connected layers with different functions and number of neurons in parallel, so as to complete the classification and localization of the target to be detected.
[0017] Furthermore, in step (1), the device for capturing fine-grained RGB cluttered corn images on the conveyor belt during the cyclic operation is a global shutter camera with high frame rate sampling performance and a fixed-focus camera with variable aperture, for data acquisition. The specific steps are as follows:
[0018] (1.1) Remove images with information size smaller than the preset value and information quality lower than the preset value, and perform uniformization processing on images of corn with impurities that have quality higher than the preset value, standardizing them to a uniform resolution;
[0019] (1.2) The dataset is augmented by adding noise, adjusting brightness and contrast, and flipping and rotating.
[0020] Furthermore, the unified resolution in step (1.1) is 600*600.
[0021] Furthermore, in step (3), considering the shooting scene and conditions of the input image, and taking into account the physical characteristics and distribution features of various targets in the harvested corn, an initial anchor box that can completely cover the image information is generated by setting the region scale and aspect ratio that are suitable for the small target detection task in the corn image containing impurities.
[0022] The specific steps are as follows:
[0023] (3.1) Use a convolution operation with a kernel size of 3 to traverse each position in the cross-level fusion feature map in step (2) to obtain a transition layer matrix with the same dimension;
[0024] (3.2) Two convolution operations with a kernel size of 1 are connected in parallel on the transition layer matrix in step (3.1) to adjust the classification and coordinates of the foreground and background of the initial anchor frame;
[0025] Based on the transition layer matrix, two convolution operations are performed in parallel to classify the initial anchor box attribute information and regress the coordinate parameters. After position adjustment and edge refinement, candidate boxes are provided for the output layer.
[0026] Furthermore, in step (4), the number of neurons in the regression layer for coordinate parameters is four times the number of neurons in the attribute information classification layer. The specific steps are as follows:
[0027] (4.1) Map the “region of interest” obtained in step (3) onto the cross-level fusion features obtained in step (2) in turn, perform ROI pooling and flattening operations to obtain a one-dimensional feature vector;
[0028] (4.2) After the one-dimensional feature vector obtained in step (4.1), two fully connected layers with 1024 neurons each are concatenated;
[0029] (4.3) In step (4.2), two fully connected layers are connected in parallel after the second fully connected layer to classify the target attribute information and regress the target coordinate parameters, respectively.
[0030] The beneficial effects of this invention are:
[0031] The method proposed in this invention can be viewed as two parts: an adaptive region generation network and a detector network, with parameters updated through alternating optimization. The feature learning network is initialized using an ImageNet pre-trained model, and the adaptive region generation network is trained end-to-end. Then, the feature learning network is initialized again using the ImageNet pre-trained model, and the detector network is trained based on the candidate boxes generated by the adaptive region generation network. Finally, the two parts share the same convolutional layers, and the layers specific to the adaptive region generation network and the detector network are fine-tuned sequentially, thus forming a unified network.
[0032] This invention utilizes a proposed end-to-end small target detection network to capture the types and distribution of impurities during the cyclic cleaning process based on input images of corn containing impurities. This provides crucial guidance and reference for the adaptability and dynamic adjustment of cleaning strategies. Its advantage lies in providing data support for operator decision-making and execution. Furthermore, real-time tracking and monitoring of the entire corn harvesting and cleaning process, along with grain condition feedback, significantly enhances the intelligence and production efficiency of the cleaning equipment. Attached Figure Description
[0033] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0034] Figure 1 This is a flowchart of the small target detection method for corn recycling operations according to the present invention;
[0035] Figure 2 This is a schematic diagram of a cross-level feature fusion mechanism;
[0036] Figure 3 This is a detailed structural diagram of the MBConv operation;
[0037] Figure 4This is the network flowchart of SEblock;
[0038] Figure 5 This is a schematic diagram showing the area scale and aspect ratio of the initial anchor frame;
[0039] Figure 6 This is a schematic diagram of a data augmentation method for images containing hybrid corn. Detailed Implementation
[0040] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0041] Figure 1 This paper presents the overall framework of the small target detection method for images containing cluttered corn, designed according to this invention. It mainly consists of three parts, arranged in the order of network propagation: a feature learning network, an adaptive region generation network, and an output layer. EfficientNet effectively balances the network's width, depth, and resolution, making the model easier to train while fully learning image features. Therefore, based on fine-grained target detection tasks, EfficientNetB7 is introduced as the backbone framework of the feature learning network for images containing cluttered corn. In the model's feedforward process, compared to deep-level feature matrices, shallow-level feature matrices contain richer spatial information, while their semantic descriptions are relatively vague. Therefore, a feature learning network is embedded on the EfficientNetB7 framework. Figure 2 The cross-level feature fusion mechanism shown obtains a feature map containing multi-scale mapping by performing convolution and upsampling operations on the deep-level feature matrix and fusing it with the shallow-level feature matrix.
[0042] Specifically, the input image containing corn noise undergoes feature learning through eight convolutional stages as shown in Table 1 (specific components of the feature learning network). The width and depth of each stage are closely related to the size of the original image, obtained by multiplying the resolution-corresponding scaling factor with the parameters of EfficientNetB0. Among these, H... i *W i *C i It is O i The dimension of the feature matrix before operation, and Figure 2 Correspondingly. L i It is O i The number of repetitions of the operation, i.e., the depth of stage i. The stride in Table 1 is the step size of the first kernel slide in the repeated convolution operation of each stage. Compared to subsequent stages, the first stage uses a traditional 3x3 kernel convolution operation. Simultaneously, a BN layer and a Swish activation function are introduced to address the gradient vanishing and gradient exploding problems during backpropagation and to improve the model's generalization ability.
[0043]
[0044] Table 1
[0045] The detailed structure of MBConv in stages 2-8 is as follows: Figure 3 As shown, it has a similar layout to the basic modules of MobileNetV3. The first convolutional operation with a 1x1 kernel size is used to increase the dimensionality of the input feature matrix. MBConv6 in Table 1 means that the number of convolutional kernels is 6 times the number of input features, while MBConv1 indicates that there is no 1x1 convolutional operation for dimensionality increase in the current stage. Similarly, k3x3 and k5x5 are the kernel sizes for depthwise convolutions in the corresponding stages. The edge padding for 3x3 and 5x5 kernels is 1 and 2 respectively, meaning that the size and number of channels of the feature matrix do not change after a planar convolution with a stride of 1. Furthermore, Figure 3 Having the same dimension for the input and output feature matrices is a necessary and sufficient condition for the existence of shortcut branches and dropout layers.
[0046] Utilize Figure 4 The lightweight, plug-and-play channel attention mechanism, SE block, compresses features in the spatial dimension and generates new weights for each channel based on the correlation between feature channels, which are then weighted sequentially into the input matrix. Its advantage lies in its ability to selectively enhance more important features by learning global information through its cross-channel interaction capabilities. Specifically, after global average pooling of each channel of the input matrix, the activation functions of the two one-dimensional fully connected layers are Swish and Sigmoid, respectively. Unlike the SE block in image classification tasks, the number of neurons in the channel-reduced fully connected layer 1 is one-quarter of the number of feature channels in the current MBConv input. The size of the fully connected layer 2 is the same as the number of feature channels after depthwise convolution. For the cross-layer fusion mechanism, a 3x3 kernel convolution operation is used to enhance the model's local perception capability, and the number of 1x1 convolution kernels allows for channel adjustment. Furthermore, the double upsampling process after feature fusion is achieved using bilinear interpolation.
[0047] Based on the distribution characteristics and geometric properties of corn kernels and impurities, the adaptive region generation network can classify and adjust the coordinates of the generated initial anchor boxes through continuous convolution. Specifically, a 3x3 convolutional kernel and a sliding window are used to sequentially traverse each position across the cross-level features, thereby generating the initial anchor boxes while obtaining a transition layer with the same size and dimensions. To more completely and accurately cover various targets in the corn-containing image, the aspect ratio of the initial anchor boxes is set to... Figure 5The ratios of 1:1, 1:2, and 2:1 shown represent regions with a scale of 64. 2 128 2 and 256 2 Finally, two parallel 1x1 convolution operations are used to classify each initial anchor box and obtain its corresponding coordinate regression parameters. The category information includes the probabilities of foreground and background, and the regression parameters are oriented towards the center coordinates, width, and height of the initial anchor box, respectively. Therefore, the number of convolution kernels is 2n and 4n, respectively.
[0048] In the end-to-end training process based on backpropagation and stochastic gradient descent, positive samples in anchor boxes are defined as: (a) anchor boxes with an IoU overlap of more than 0.7 with any ground truth box, or (b) anchor boxes with the highest IoU ratio with one of the ground truth boxes. Conversely, anchor boxes with an IoU of less than 0.3 with any ground truth box are considered negative samples. Anchor boxes that are neither positive nor negative samples will not participate in the network update process. To avoid poor model generalization and degradation caused by too many negative samples, the mini-batch loss is calculated by randomly sampling the same number of positive and negative samples. The loss function of the adaptive region generation network is shown in Equation (1), where N m =256 represents the number of mini-batches. If the number of positive samples is less than 128, then negative samples are used to supplement the mini-batches. i represents an anchor box in the current mini-batch, c i This represents the probability that the i-th anchor box is predicted to be the true label. If the current anchor box is a positive sample, then... The true value is 1 for a positive sample and 0 for a negative sample. The r represents the coordinate regression parameters corresponding to the ground truth bounding box for the i-th anchor box. i ={r x r y r w r h} represents the predicted value.
[0049]
[0050] L cls =-ln(c i (2)
[0051]
[0052] Classification loss L cls and regression loss L reg They are defined using formulas (2) and (3) respectively, as shown in formula (4). As a robust loss function, it is introduced. Furthermore, Equation (5) describes Attr bef ={xbef y bef w bef h bef}, r i and The relationship between Attr bef and Attr aft ={x aft y aft w aft h aft} represents the attribute information of the initial anchor box and the candidate box after coordinate adjustment, respectively. gt ={x * y * w * h *} represents the attribute information of the ground truth bounding box corresponding to the current anchor box. The network parameters are randomly initialized by drawing weights from a zero-mean Gaussian distribution with a standard deviation of 0.01. Simultaneously, anchor boxes crossing boundaries introduce a large number of difficult-to-correct errors, so anchor boxes with boundary crossing outliers are ignored during training. Finally, based on the category information of the generated regions, non-maximum suppression (NMS) is used to handle highly overlapping candidate boxes, with the IoU threshold for NMS set to 0.7.
[0053]
[0054]
[0055] The candidate boxes generated by the adaptive region generation network are considered as regions of interest (ROIs) for the specific classification and coordinate regression of the final target. These regions are mapped onto cross-layer fusion features obtained through the feature learning network. After ROI pooling, the feature matrix is regularized to a uniform size and flattened. The two subsequent fully connected layers each have 1024 neurons, and their output feature vectors are used as inputs to the classifier and regressor. The classification layer with the softmax function contains k+1 outputs, representing the probability that the target belongs to different categories. Here, k is the number of target categories, and the possibility that the candidate box is background must also be considered. Similar to the regression layer in the adaptive region generation network, the candidate box regressor contains 4*(k+1) neurons, which can adjust the position of each candidate box by outputting four parameters related to the center coordinates, length, and width. As shown in formula (6), {P x P y P w P h} represents the center coordinates, width, and height of the candidate box, {U x U y U w U h} represents the attribute information of the final bounding box output by the small object detection network for images containing corn. x f y f w f h} represents the coordinate regression parameters for the k+1 target categories output by the regressor.
[0056]
[0057] In the small object detection network, the loss of each candidate box is the classification loss L in formula (7). cat and regression loss L loc It consists of two parts. Where q = {q0, q1, ..., q} k Let} be the softmax probability distribution predicted by the classifier, v be the true class label corresponding to the target in the candidate box, and L be the class loss. cat Measured by formula (8).
[0058] The coordinate regression parameters for the corresponding class g predicted by the regressor, s = {s x s y s w s h} represents the coordinate regression parameters of the ground truth object corresponding to the candidate bounding box. The regression loss L... loc The value is calculated using formula (9), where α is a hyperparameter used to balance the two losses. Furthermore, the Iverson bracket index function [v>0] has a value of 1 when v>0, and 0 otherwise. Compared to the basic Faster R-CNN network, the proposed method replaces the initial backbone ZFNet with EfficientNetB7 and embeds a cross-layer fusion mechanism, which has the advantage of capturing multi-layer features in fine-grained, cluttered corn images. Simultaneously, the region scale of the anchor boxes in the adaptive region generation network is also adjusted accordingly for small target detection tasks, which has the advantage of more completely and accurately covering various targets in cluttered corn images.
[0059] Loss(q, v, b) g ,s)=L cat (q, v) + α[v > 0]L loc (b g ,s) (7)
[0060] L cat (q, v) = -ln q v (8)
[0061]
[0062] During image data preprocessing, the resolution of RGB images containing clutter was standardized to 600*600, which will better suit the feature learning network. To avoid uncertainty in network convergence direction and overfitting due to insufficient sample size, data augmentation methods were used to expand the dataset. Specifically, the original images were augmented as follows: Figure 6 The rotation, horizontal mirror symmetry, vertical mirror symmetry, contrast and brightness adjustment, Gaussian noise and salt-and-pepper noise were shown, and the mixed corn dataset was divided into training set and test set.
[0063] During model training, the proposed method can be viewed as two parts: an adaptive region generation network and a detector network. Parameters are updated through alternating optimization. Specifically, the feature learning network is initialized using an ImageNet pre-trained model, and the adaptive region generation network is trained end-to-end. Then, the feature learning network is initialized again using the ImageNet pre-trained model, and the detector network is trained based on the candidate boxes generated by the adaptive region generation network. Finally, the two parts share the same convolutional layers, and the layers specific to the adaptive region generation network and the detector network are fine-tuned sequentially, thus forming a unified network.
[0064] The contents not described in detail in this specification are existing technologies known to those skilled in the art.
[0065] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for detecting small targets in corn cyclic cleaning operations, characterized in that, Includes the following steps: (1) An image acquisition module is set at the inlet and outlet of the corn harvesting and cleaning equipment coupled with multi-physics field to capture images of corn containing impurities on the conveyor belt in real time and continuously. After uniformization processing, a dataset is obtained. The dataset is expanded based on the image enhancement method and used as the data input for training and testing the small target detection model. The small target detection model includes: feature learning network, adaptive region generation network and output layer. (2) Introduce EfficientNetB7 as the backbone framework of the feature learning network in step (1), embed a cross-level fusion mechanism, perform convolution and upsampling operations on the deep-level feature matrix, and fuse it with the shallow-level feature matrix to generate cross-level fusion features and obtain a cross-level fusion feature map containing multi-scale mapping. In step (2), a cross-level feature map containing multi-scale mapping is obtained. The specific steps are as follows: (2.1) The deep feature map is transformed in dimension by a convolution operation with a kernel size of 1, and reduced to the same depth as the previous layer feature map to obtain the modified deep feature map; (2.2) The shallow feature map is further captured by a convolution operation with a kernel size of 3 to obtain the modified shallow feature map, which is added to the modified deep feature map obtained in step (2.1) and upsampled based on bilinear interpolation; (3) Based on the cross-level fusion features generated in step (2), combined with the distribution features and geometric properties of harvested corn kernels and impurities, the adaptive region generation network is used to classify and regress the initial anchor boxes of all targets in the generated full-coverage image through continuous convolution operations, providing candidate boxes, i.e. regions of interest, for the output layer. The aspect ratio of the initial anchor frame is set to 1:1, 1:2, and 2:1, and the area scale is 64. 2 128 2 and 256 2 ; (4) Map the region of interest obtained in step (3) to the cross-level fusion feature obtained in step (2), arrange two fully connected layers in series after ROI pooling and flattening, and classify and regress the final attribute information and coordinate parameters of the candidate box by connecting two fully connected layers with different functions and number of neurons in parallel, so as to complete the classification and localization of the target to be detected. In step (3), considering the shooting scene and conditions of the input image, and taking into account the physical characteristics and distribution features of various targets in the harvested corn, an initial anchor box that can completely cover the image information is generated by setting the regional scale and aspect ratio that are suitable for the small target detection task of corn images containing impurities. The specific steps are as follows: (3.1) Use a convolution operation with a kernel size of 3 to traverse each position in the cross-level fusion feature map in step (2) to obtain a transition layer matrix with the same dimension; (3.2) Two convolution operations with a kernel size of 1 are performed in parallel on the transition layer matrix in step (3.1) to adjust the classification and coordinates of the foreground and background of the initial anchor frame; Based on the transition layer matrix, two convolution operations are performed in parallel to classify the initial anchor box attribute information and regress the coordinate parameters. After position adjustment and edge refinement, candidate boxes are provided for the output layer.
2. The method for detecting small targets in corn cyclic cleaning operations according to claim 1, characterized in that, In step (1), the device for capturing fine-grained RGB cluttered corn images on the conveyor belt during the cyclic operation is a global shutter camera with high frame rate sampling performance and a fixed-focus camera with variable aperture. The specific steps are as follows: (1.1) Remove images with information size smaller than the preset value and information quality lower than the preset value, and perform uniformization processing on the corn images with impurities that have quality higher than the preset value, and standardize them to a uniform resolution; (1.2) The dataset is augmented by adding noise, adjusting brightness and contrast, and flipping and rotating.
3. The method for detecting small targets in corn cyclic cleaning operations according to claim 2, characterized in that, The unified resolution in step (1.1) is 600.
600.
4. The method for detecting small targets in corn cyclic cleaning operations according to claim 1, characterized in that, In step (4), the number of neurons in the regression layer for coordinate parameters is four times the number of neurons in the attribute information classification layer. The specific steps are as follows: (4.1) Map the regions of interest obtained in step (3) sequentially onto the cross-level fusion features obtained in step (2), perform ROI pooling and flattening operations, and obtain a one-dimensional feature vector; (4.2) After the one-dimensional feature vector obtained in step (4.1), two fully connected layers with 1024 neurons each are concatenated; (4.3) In step (4.2), two fully connected layers are connected in parallel after the second fully connected layer to classify the target attribute information and regress the target coordinate parameters, respectively.