A method and system for residue cleaning of a wheat harvester

Abstract

The present application relates to a kind of wheat harvester residue cleaning method and system, the method includes the following steps: collecting material bin bottom image and pre-processing;Residue target identification model is constructed;The image after pre-processing is input into model and target identification is carried out, the coordinate of the predicted bounding box of residue target is obtained;Coordinate conversion is carried out, the coordinate of the center point of predicted bounding box is converted to three-dimensional coordinates of mechanical arm base coordinate system, and then mechanical arm motion trajectory planning is carried out, and the motion trajectory of time optimization is obtained;Control mechanical arm movement drives cleaning device to move and clean the residue of material bin bottom.The present application can automatically identify, locate and completely remove the residue in each corner of material bin bottom, realize the automation and intelligentization of wheat harvester material bin residue cleaning, effectively guarantee the purity of breeding and reliability.

Description

Technical Field

[0001] This invention belongs to the field of intelligent agricultural equipment technology, and in particular relates to a method and system for cleaning residues from a wheat harvester. Background Technology

[0002] During the process of wheat seed production in the field, seeds of different strains need to be harvested separately by region, and the harvester needs to be thoroughly cleaned to ensure seed purity and avoid mixing of varieties.

[0003] However, the core functions of commonly used grain combine harvesters are focused on efficient harvesting and threshing, with insufficient ability to clean residual grains inside the machine. The main shortcomings are as follows: Structurally, there is a lack of integrated cleaning devices specifically designed for seed production scenarios. Existing cleaning equipment, such as fixed grain cleaners or mobile grain suction machines, is often bulky and cannot be directly installed inside the harvester's hopper, making it difficult to address residue issues in corners. Regarding intelligence, existing hopper cleaning methods mostly rely on manual operation or simple mechanical actions, failing to achieve automatic identification and positioning of residual grains. Even when using methods such as air suction, the lack of intelligent control results in inaccurate airflow distribution and coverage, creating blind spots in the cleaning process. Summary of the Invention

[0004] The purpose of this invention is to solve one of the above-mentioned technical problems and to provide a method and system for cleaning residues in a wheat harvester that is integrated inside the harvester and can automatically identify, locate, and thoroughly remove residues in the collection bin.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for cleaning residue from a wheat harvester, wherein a robotic arm is installed in the harvester's hopper, and a residue cleaning device is mounted on the robotic arm, the residue cleaning method comprising the following steps:

[0007] Industrial cameras were used to capture images of the bottom of the material collection bins, and the captured images were preprocessed.

[0008] Improve the YOLOv11 model and construct a residual target recognition model based on YOLOv11;

[0009] The preprocessed image is input into the residue target recognition model to perform residue target recognition on the preprocessed image and obtain the predicted bounding boxes of each residue target at the bottom of the collection bin.

[0010] The pixel coordinates of the center point of the predicted bounding box of each residual target are transformed to three-dimensional coordinates in the robot arm's base coordinate system.

[0011] Based on the three-dimensional coordinates of each residual target in the robot arm's base coordinate system, the robot arm's motion trajectory is planned to obtain the time-optimal motion trajectory.

[0012] The robotic arm moves based on the optimal motion trajectory, driving the cleaning device to move and clean the residue at the bottom of the collection bin.

[0013] In some embodiments of the present invention, the specific method for improving the YOLOv11 model includes the following steps:

[0014] A C2PSA cross-stage local self-attention module is introduced into the backbone network of the YOLOv11 model;

[0015] A CGLU gating mechanism is introduced into the C3k2 module of the backbone network of the YOLOv11 model to obtain the C3k2_SMPCGLU module;

[0016] The SPPF spatial pyramid pooling module of the backbone network of the YOLOv11 model is replaced with the FPSConv feature pyramid shared convolution module; the FPSConv feature pyramid shared convolution module has three convolution branches with different dilation rates, which are used to extract features of different receptive fields.

[0017] Replace the regular convolutional layers of the neck network in the YOLO11 model with CBS modules.

[0018] In some embodiments of the present invention, the method for identifying residual targets in a preprocessed image specifically includes the following steps:

[0019] The preprocessed image is input into the backbone network of the residual object target recognition model. The shallow and deep features of the image are extracted by the C2PSA cross-stage local self-attention module and the C3k2_SMPCGLU module. The FPSConv feature pyramid shared convolution module is used to extract features of different receptive fields to output multi-layer features of different receptive fields.

[0020] The multi-layer features output from the backbone network are input into the neck network of the residual target recognition model. The deep features are upsampled step by step, and the upsampled deep features are concatenated with the shallow features and then input into the CBS module for feature fusion to output multi-scale fused features.

[0021] The multi-scale fusion features output from the neck network are input into the detection head for target localization and classification, and the predicted bounding box, class label, class confidence and depth value of each residual target are output.

[0022] In some embodiments of the present invention, the method for identifying residual targets in a preprocessed image further includes the following steps:

[0023] Predicted bounding boxes are filtered based on a predetermined confidence threshold to remove bounding boxes whose class confidence is lower than the predetermined confidence threshold.

[0024] The non-maximum suppression algorithm is used to remove overlapping predicted bounding boxes, retain a unique prediction result for each residual region, and output the predicted bounding box, class confidence and depth value of the unique prediction result.

[0025] In some embodiments of the present invention, the image acquisition device includes an RGB camera and a depth camera; the image at the bottom of the material collection silo includes an RGB image and a depth image, and the method for coordinate transformation specifically includes the following steps:

[0026] Perform intrinsic and extrinsic parameter calibration on the RGB camera and the depth camera to obtain the intrinsic parameter matrix K and the extrinsic parameter matrix [R|t] between them;

[0027] Obtain the coordinates (u,v) of the center point of the predicted bounding box of the residual target and its corresponding depth value z;

[0028] The 3D coordinates (X, Y, Z) of the predicted bounding box center point in the camera coordinate system are calculated based on the coordinates (u, v) and the depth value z. The calculation formula is as follows:

[0029] ;

[0030] in, , , , All of these are camera intrinsic parameters in the intrinsic parameter matrix K;

[0031] The 3D coordinates (X,Y,Z) in the camera coordinate system are transformed to the 3D coordinates in the working coordinate system of the robotic arm based on the extrinsic parameter matrix [R|t].

[0032] In some embodiments of the present invention, the specific method for planning the motion trajectory of a robotic arm includes the following steps:

[0033] Based on the three-dimensional coordinates of each residual target in the robot arm's base coordinate system, the Fireworks Algorithm is used to perform a wide-area search for the motion trajectory. After each iteration of the Fireworks Algorithm, the Particle Swarm Algorithm is used to locally optimize the Fireworks swarm generated by the Fireworks Algorithm, and finally obtains the time-optimal motion trajectory.

[0034] In some embodiments of the present invention, the method for performing local optimization specifically includes:

[0035] Calculate the fitness value of each firework in the firework population;

[0036] A predetermined number of elite sparks are selected from the fireworks population based on their fitness values.

[0037] Elite Sparks are used as the initial particle population for the Particle Swarm Optimization (PSO) algorithm. During the iteration of the PSO algorithm, the position of each particle is updated to the individual optimal position and the global optimal position. The optimized particle population is then output as the initial fireworks population for the next iteration of the fireworks algorithm.

[0038] In some embodiments of the present invention, the method for preprocessing the acquired images specifically includes the following steps:

[0039] The bottom image of the collection silo is scaled proportionally to the predetermined size, and blank space is filled around the image to ensure that the aspect ratio of the image remains unchanged;

[0040] The bottom image of the aggregate bin is normalized and the pixel values ​​are scaled to a predetermined range.

[0041] Adjust the image channels of the bottom image of the aggregate bin according to the predetermined sequence.

[0042] Some embodiments of the present invention further provide a residue cleaning system for a wheat harvester, used to implement the above-mentioned residue cleaning method for a wheat harvester, including an image acquisition device, a residue cleaning device, and a decision control module;

[0043] The image acquisition device is installed inside the feed bin of the wheat harvester to acquire images of the bottom of the feed bin;

[0044] The residue cleaning device is installed inside the collection bin of the wheat harvester, including a robotic arm and an air suction pipe; the robotic arm is mounted on the inner wall of the collection bin via a slide rail; one end of the air suction pipe is connected to the end of the robotic arm away from the slide rail, and the other end extends to the outside of the collection bin and is electrically connected to a negative pressure fan.

[0045] The decision control module is communicatively connected to both the image acquisition device and the residue cleaning device. The decision control module includes a residue target recognition model, a trajectory planning unit, and a control unit. The residue target recognition model is used to identify targets in the images of the bottom of the silo acquired by the image acquisition device and outputs the predicted bounding boxes of each residue target at the bottom of the silo. The trajectory planning unit is used to perform coordinate transformation on the pixel coordinates of the center point of the predicted bounding box of each residue target, thereby planning the robotic arm's motion trajectory to obtain the time-optimal motion trajectory. The control unit is used to drive the robotic arm of the residue cleaning device to move based on the time-optimal motion trajectory.

[0046] In some embodiments of the present invention, a filtration and collection device is further included;

[0047] The filter collection device is connected to the air suction pipe and the negative pressure fan respectively through a spiral separator, and is used to receive the residue sucked out by the air suction pipe.

[0048] The beneficial effects of this invention are as follows:

[0049] 1. By constructing a target recognition model and improving the trajectory planning algorithm, this invention can automatically identify, locate and thoroughly remove the residues in every corner of the bottom of the collection bin, realizing the automation and intelligence of cleaning residues in the collection bin of wheat harvesters. It solves the problem of mixed seeds caused by relying on manual experience or a single mechanical structure in traditional cleaning methods, and effectively ensures the purity and reliability of seed production.

[0050] 2. This invention introduces the C2PSA attention module and CBS module into the YOLOv11 model to construct a residual target recognition model, which improves the target detection effect in complex scenarios such as light changes, warehouse bottom shadows, rust backgrounds and small grains, improves detection accuracy and robustness, and effectively avoids false detection and missed detection.

[0051] 3. This invention combines the Fireworks Algorithm and the Particle Swarm Optimization Algorithm for planning the motion trajectory of the robotic arm. It effectively combines the powerful global search capability of the Fireworks Algorithm with the efficient local convergence capability of the Particle Swarm Optimization Algorithm, balancing the global exploration and local mining capabilities, improving the convergence speed and optimization accuracy of the algorithm, and effectively avoiding getting trapped in local optima, thereby improving the working efficiency of the harvester.

[0052] 4. The invention employs a foldable robotic arm and a distributed installation design, which allows the residue cleaning device to be completely retracted when not in operation, without affecting the original harvesting function of the harvester. Furthermore, its modules are independent, making it easy to integrate into new machines or add to existing equipment. The modification cost is low, the applicability is wide, and it is easy to promote.

[0053] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims, and drawings. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart of a method for cleaning residue from a wheat harvester;

[0056] Figure 2 This is a schematic diagram of the structure of a residue cleaning system for a wheat harvester;

[0057] Figure 3 This is a schematic diagram of the material collection bin of a wheat harvester;

[0058] The attached figures are labeled as follows:

[0059] 1. Inner wall of the collection bin; 2. Slide rail; 3. Air suction pipe; 4. Spiral separator; 5. Fan connecting pipe; 6. Negative pressure fan; 7. Filter collection device; 8. Robotic arm; 9. Monocular depth industrial camera; 10. Air suction pipe; 11. Spiral auger. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0061] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0062] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0063] The technical solution of the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.

[0064] As attached Figure 1 As shown in the illustrative embodiment of a residue cleaning method for a wheat harvester according to the present invention, the wheat harvester's collection bin is equipped with a residue cleaning device and an image acquisition device, both of which are mounted on the inner wall of the collection bin via a foldable robotic arm. The residue includes, but is not limited to, wheat ears, straw, wheat bran, and wheat grains.

[0065] The residue cleaning method includes the following steps.

[0066] S1: After the wheat in the seed-producing wheat harvester's hopper is discharged from the grain bin through the auger at the bottom of the hopper, the robotic arm is extended by a servo motor, which moves the image acquisition device installed at the end of the foldable robotic arm to the best observation point in the hopper's working area. The device takes multiple pictures of the bottom of the hopper from one side to the other to acquire color RGB images. At the same time, it forms a depth image by using the distance information of each pixel from the camera.

[0067] Preprocess the acquired raw images to avoid inconsistent sizes among the acquired raw images.

[0068] In some embodiments of the present invention, the method for preprocessing the acquired raw images includes lens distortion correction based on camera intrinsic parameters, image size normalization, and pixel value normalization to obtain an image tensor that meets the input requirements of a neural network. Specifically, it includes the following steps.

[0069] The image of the bottom of the material silo is scaled proportionally to a predetermined size, and blank space is filled around the image to ensure that the aspect ratio remains unchanged. In this embodiment, the predetermined size is set to 600×600 pixels.

[0070] The formula for proportionally scaling the image of the bottom of the silo is:

[0071] ;

[0072] ;

[0073] ;

[0074] in, Input dimensions, , These are the height and width of the original image, respectively. Scaling factor , These are the fill width and fill height, respectively.

[0075] Normalize the layout of the bottom image of the aggregate bin and scale the pixel values ​​to the range of 0 to 1.

[0076] The image channels of the bottom image of the aggregate bin are adjusted in a predetermined order. The predetermined order of the image channels is determined based on the needs of the residue target recognition model.

[0077] S2: Improve the YOLOv11 model and construct a residue target recognition model based on YOLOv11. The structure of the residue target recognition model is shown in the attached figure. Figure 2 As shown.

[0078] In some embodiments of the present invention, the specific method for improving the YOLOv11 model includes the following steps:

[0079] A C2PSA cross-stage local self-attention module is introduced into the backbone of the YOLOv11 model. This module enhances the model's attention to fine-grained features of wheat grains through a self-attention mechanism, effectively suppressing interference from the warehouse background, rust, and shadows, and improving the robustness of feature extraction.

[0080] Introducing the CGLU gating mechanism into the C3k2 module of the YOLOv11 model backbone network, we obtain the C3k2_SMPCGLU module. Compared to the traditional C3k2 module, this module introduces SMPConv, which acquires features from different receptive fields through multi-branch convolutions. The CGLU gating mechanism selectively enhances these features, which helps to highlight key details such as seed edges and cracks. Furthermore, it can combine DropPath regularization to suppress overfitting, ensuring lightweight design while effectively enhancing the model's ability to represent small targets and complex backgrounds.

[0081] The SPPF spatial pyramid pooling module of the YOLOv11 model backbone network is replaced with the FPSConv feature pyramid shared convolutional module. A parallel structure of multi-dilation rate convolutions is used to replace multi-scale pooling, thereby better extracting multi-scale features of the detected target. The FPSConv feature pyramid shared convolutional module has three convolutional branches with different dilation rates, pre-set to 1, 3, and 5, respectively, to extract features from different receptive fields.

[0082] The conventional convolutional layers of the neck network in the YOLO11 model are replaced with CBS modules. The CBS module is composed of convolutional (conv), batch normalized (BN), and SiLU activation function. This structure ensures training stability through batch normalization and better preserves the detailed features of small targets by utilizing the smoothness of the SiLU activation function. Combined with a multi-scale feature fusion strategy, it ensures that residual seed features are not easily lost during cross-scale transmission.

[0083] In the above illustrative embodiments, the YOLOv11 model was specifically improved to address the characteristics of small targets and multiple interferences in complex agricultural scenarios. By introducing a C2PSA (Cross-Stage Local Self-Attention) module into the backbone network, the model's ability to select attention for fine-grained features of wheat grains was enhanced, effectively suppressing interference from the warehouse background, rust, and shadows. In the neck network, conventional convolutional layers were replaced with CBS modules (Conv+BatchNorm+SiLU), and combined with multi-scale feature fusion, the expressive power and transmission efficiency of small target features were significantly improved, effectively preventing the loss of residual grain features.

[0084] S3: Input the preprocessed image into the residue target recognition model, perform residue target recognition on the preprocessed image, and obtain the predicted bounding boxes of each residue target at the bottom of the collection bin.

[0085] In some embodiments of the present invention, the method for identifying residual targets in a preprocessed image specifically includes the following steps.

[0086] S31: Backbone Feature Extraction. The preprocessed image is input into the backbone network of the residual object recognition model. Shallow and deep features of the image are extracted through the C2PSA cross-stage local self-attention module and the C3k2_SMPCGLU module. Specifically, in the backbone network, the image first undergoes two convolutions (Conv) to extract shallow features such as edges and textures; then, it is stacked with multiple C3k2 modules and convolutional layers (CBS modules) to progressively extract deeper semantic information.

[0087] In the backbone network, the FPSConv feature pyramid shared convolution module is used to extract features from different receptive fields. The outputs of each branch are concatenated in the channel dimension and then fused by 1×1 convolution. The final output multi-layer features retain global semantic information while taking into account local details. Compared with pooling operations, FPSConv does not lose spatial location information, so it performs more stably when dealing with complex backgrounds in the material collection bin and small target detection.

[0088] S32: Feature Fusion. Feature fusion adopts a multi-scale feature interaction structure based on FPN+PAN, implemented through a lighter CBS module.

[0089] The multi-layer features output from the backbone network are input into the neck network of the residue target recognition model. The deep features are upsampled and concatenated step by step, and then fused with the high-resolution features of the shallow layer. This enhances semantic expression while preserving detailed information, enabling the model to better perceive small targets such as wheat grains.

[0090] The upsampled deep features are concatenated with the shallow features and then input into the CBS module for feature fusion. This process suppresses channel redundancy and enhances nonlinear expressive power, resulting in more compact and discriminative features for outputting multi-scale fused features. During the fusion process, the C3k2 module is introduced to extract deep semantic information with lower computational cost, thereby enhancing the model's ability to represent large-scale targets such as straw and miscellaneous leaves.

[0091] The fusion process of the CBS module adopts a bidirectional path design, which includes top-down information transmission (high-level semantics diffuses to shallow layers) to supplement semantic features, and bottom-up feature integration (shallow details feed back to deep layers) to maintain spatial resolution and local texture information. Through multiple iterations of upsampling, stitching, and CBS and C3k2 combination operations, the neck network of the residual object recognition model significantly improves the efficiency and expressive power of multi-scale feature interaction while ensuring controllable computational complexity. This enables the model to meet the detection needs of different-sized targets such as wheat grains, straw, and miscellaneous leaves in complex silo environments.

[0092] S33: Input the multi-scale fusion features output by the neck network into the detection head for target localization and classification, and output the predicted bounding box, class label, class confidence and depth value for each residual target.

[0093] In this embodiment, the detection head uses an anchor-free decoupled prediction structure. Multi-scale fusion features (P3, P4, P5) from the neck network are input to three detection branches, each independently predicting the target bounding box and category information at its corresponding scale. The anchor-free detection head does not rely on predefined anchor boxes, but directly predicts the offset and width / height parameters of the target center point on the feature map grid. This not only reduces the computational overhead of anchor box matching, but also improves the adaptability to targets of different sizes.

[0094] Furthermore, to further reduce the mutual interference between classification and localization, the detection head adopts a decoupled design: the classification branch and the regression branch are independent in feature extraction. Specifically, the classification branch outputs the class probability at each location through convolutional layers, while the regression branch predicts the location parameters of the bounding box and the target confidence. This structure ensures that categories such as wheat grains, straw, and miscellaneous leaves can obtain clearer feature separation and optimization targets during the detection process, thereby achieving more stable and efficient multi-target detection in complex silo environments.

[0095] In this embodiment, the detection head maintains a lightweight design to ensure that the detection speed meets real-time requirements. The model synchronously outputs the bounding box, class confidence, and key depth estimate for each residual wheat target in the image through end-to-end forward inference.

[0096] In some embodiments of the present invention, the method for identifying residual targets in a preprocessed image further includes the following steps.

[0097] S34: Post-processing. Calculate the final score based on the category probability and object confidence of different identified targets, determine overlapping boxes by rotating IoU, identify the optimal prediction result, and remove redundant boxes.

[0098] Specifically, in the post-processing steps, the predicted bounding boxes are first filtered based on a predetermined confidence threshold to remove bounding boxes whose class confidence is lower than the predetermined confidence threshold.

[0099] The non-maximum suppression (NMS) algorithm is used to remove overlapping predicted bounding boxes, retain a unique optimal prediction result for each residual region, and output the unique optimal prediction result containing the predicted bounding box information (u,v,w,h), the class confidence (conf), and the detection list of depth values.

[0100] In some embodiments of the present invention, the method for identifying residual targets in a preprocessed image further includes the following steps.

[0101] S35: Fusion of RGB and depth images.

[0102] Intrinsic and extrinsic parameter calibrations are performed on the RGB and depth cameras to obtain the intrinsic parameter matrix K (focal length and principal point coordinates) and the extrinsic parameter matrix [R|t] (rotation and translation relationship) between them, ensuring that the RGB and depth images can be aligned in the same coordinate system. After calibration, projection formulas can be established between pixel coordinates (u,v), depth value Z, and 3D world coordinates (X,Y,Z).

[0103] During the fusion process, the model performs 2D detection in the RGB image to obtain the bounding box and center point pixel coordinates of the residual target. Subsequently, the corresponding depth value is read from the depth map at the same location, and the 2D coordinates are mapped to 3D space using the aforementioned projection formula, obtaining the target's 3D coordinates (X, Y, Z) in the camera coordinate system. Furthermore, the extrinsic parameter matrix [R|t] can be used to transform it to the world coordinate system or the robotic arm's working coordinate system, thus providing precise spatial location information for subsequent grasping or cleaning operations.

[0104] S4: Perform coordinate transformation on the pixel coordinates of the center point of the predicted bounding box of each residual target, and transform them to the three-dimensional coordinates of the robot arm base coordinate system.

[0105] In some embodiments of the present invention, the method for performing coordinate transformation specifically includes the following steps.

[0106] Intrinsic and extrinsic parameter calibrations are performed on the RGB and depth cameras to obtain the intrinsic parameter matrix K (focal length and principal point coordinates) and the extrinsic parameter matrix [R|t] (rotation and translation relationship) between them, ensuring that the RGB and depth images can be aligned in the same coordinate system. After calibration, projection formulas can be established between pixel coordinates (u,v), depth value Z, and 3D world coordinates (X,Y,Z).

[0107] Based on the coordinates (u,v) of the center point of the predicted bounding box of the residual target output by the model, the corresponding depth value z is read from the depth map at the same location, and the two-dimensional coordinates are mapped to three-dimensional space using the projection formula to obtain the three-dimensional coordinates (X,Y,Z) of the target in the camera coordinate system.

[0108] In this embodiment, the projection formula is:

[0109] ;

[0110] in, , , , All of these are camera intrinsic parameters in the intrinsic parameter matrix K.

[0111] Based on the extrinsic parameter matrix [R|t], the 3D coordinates (X,Y,Z) in the camera coordinate system are transformed to the world coordinate system or the working coordinate system of the robotic arm, providing accurate spatial position information for subsequent grasping or cleaning operations.

[0112] S5: Based on the three-dimensional coordinates of each residual target in the robot arm's base coordinate system, plan the robot arm's motion trajectory to obtain the optimal motion trajectory in time.

[0113] In some embodiments of the present invention, the specific method for planning the motion trajectory of a robotic arm includes the following steps:

[0114] Based on the three-dimensional coordinates of each residual target in the robot arm's base coordinate system, the Fireworks Algorithm is used to perform a wide-area search for the motion trajectory. After each iteration of the Fireworks Algorithm, the Particle Swarm Algorithm is used to locally optimize the Fireworks swarm generated by the Fireworks Algorithm, and finally obtains the time-optimal motion trajectory.

[0115] In some embodiments of the present invention, the method for performing local optimization specifically includes:

[0116] Calculate the fitness value of each firework in the firework population;

[0117] A predetermined number of elite sparks are selected from the fireworks population based on their fitness values.

[0118] Elite Sparks are used as the initial particle population for the Particle Swarm Optimization (PSO) algorithm. During the iteration of the PSO algorithm, the position of each particle is updated to the individual optimal position and the global optimal position. The optimized particle population is then output as the initial fireworks population for the next iteration of the fireworks algorithm.

[0119] In some embodiments of the present invention, step S5 specifically includes the following steps:

[0120] S51: Algorithm initialization.

[0121] Initialize the predefined parameters N, M, D, MaxIter, and [Tmin, Tmax]. Here, N is the firework swarm size; M is the particle swarm size, which is usually related to the total number of sparks; D is the dimension of the optimization problem, corresponding to the number of trajectory segments (e.g., D=3 for 3 trajectory segments); MaxIter is the maximum number of iterations; and [Tmin, Tmax] is the feasible solution range constraint for each trajectory segment's time.

[0122] Initialize the fireworks algorithm parameters, including: upper limit Smax of the number of explosion sparks, lower limit Smin of the number of explosion sparks, initial explosion amplitude Ainitial, elite spark ratio coefficient, and Cauchy distribution parameters.

[0123] Initialize the particle swarm optimization algorithm parameters, including: maximum and minimum inertia weights w_max and w_min, and initial learning factors c1_initial, c2_initial, c1_final, and c2_final.

[0124] A firework population, Fireworks, is randomly initialized within the solution space. The position vector Ti = [t_{i1}, t_{i2}, ..., t_{iD}] of each firework represents a set of piecewise time solutions for the trajectory. The fitness value f(Ti) of each firework is calculated, which is the total time Ttotal.

[0125] S52: Execute the Fireworks Algorithm (IFWA) explosion operation.

[0126] For each firework Ti in the current iteration, perform the following steps:

[0127] S52.1: Calculate the number of explosion sparks Si and the explosion radius A_i.

[0128] The formula for calculating the number of explosion sparks, Si, is as follows:

[0129] Si = round(Smax * (f(Ti) - f_min + ε) / (Σ (f(Ti) - f_min) + ε));

[0130] Where f_min is the optimal fitness value in the current population, and ε is a very small constant to prevent division by zero.

[0131] The number of explosion sparks Si is limited to the range [Smin, Smax] by the formula Si = max(min(Si, Smax), Smin).

[0132] The formula for calculating the explosion radius A_i is:

[0133] A_i(t) = A_initial * (1 - t / MaxIter)^k * (f(T_i) - f_min + ε) / (f_max - f_min + ε);

[0134] Where t is the current iteration number and k is the decay coefficient.

[0135] This formula introduces a time decay mechanism, which allows the algorithm to maintain a large exploration range in the early stages and accelerate convergence in the later stages.

[0136] S52.2: Generates explosive sparks.

[0137] At the current position of the firework Ti, within the hypersphere defined by the explosion radius Ai, Si explosion sparks (Spark_explosion) are randomly generated. The formula for calculating the position of the new spark for each dimension d is:

[0138] spark_d = Ti[d] + uniform(-1, 1) * Ai.

[0139] S52.3: Generate Cauchy mutant sparks.

[0140] To increase population diversity, the Cauchy mutation operator is introduced to select some non-elite sparks in the current population and generate Gaussian mutated sparks (Spark_cauchy) according to the following formula:

[0141] spark_cauchy = Ti + Cauchy(0, 1) * (Tbest - Ti);

[0142] Where Cauchy(0,1) represents a standard Cauchy distribution random number, and Tbest is the current global optimal solution.

[0143] The long-tailed property of the Cauchy distribution enhances the algorithm's global escape capability.

[0144] S52.4: Select Elite Spark.

[0145] Based on fitness values, a certain number (e.g., Top-N) of EliteSparks are selected from all fireworks, explosive sparks, and mutated sparks to enter the next generation of fireworks population.

[0146] S53: Local optimization is performed using the Particle Swarm Optimization (PSO) algorithm.

[0147] The EliteSparks obtained in step S52.4 are considered as the initial particle population for the particle swarm optimization algorithm. The following steps are performed:

[0148] S53.1: Initialize particle information: The position of each particle X_i = T_i (i.e., the position of the firework spark), its individual historical best position Pbest_i = X_i, and its velocity V_i is initialized to 0 or randomly generated within a small range.

[0149] S53.2: Update particle state.

[0150] For each particle i and each dimension d, update its velocity and position.

[0151] The formula for updating the speed is:

[0152] Vi[d]^{t+1} = w(t) * Vi[d]^t + c1(t)*r1*(Pbest_i[d] - X_i[d]^t) + c2(t)*r2*(Gbest[d] - X_i[d]^t).

[0153] The formula for updating the position is:

[0154] Xi[d]^{t+1} = Xi[d]^t + Vi[d]^{t+1}.

[0155] in,

[0156] w(t) is the dynamic inertia weight:

[0157] w(t) = w_max - (w_max - w_min) * (t / MaxIter)^2 (non-linear decreasing).

[0158] Both c1(t) and c2(t) are dynamic learning factors:

[0159] c1(t) = c1_initial - (c1_initial - c1_final) * (t / MaxIter);

[0160] c2(t) = c2_initial + (c2_final - c2_initial) * (t / MaxIter).

[0161] r1 and r2 are random numbers in the range [0,1], and Gbest is the current global optimal position.

[0162] S53.3: Update the optimal position. Calculate the fitness of each new particle and update its individual historical best position Pbest_i and global best position Gbest.

[0163] S54: Iteration and Termination. Perform the following steps:

[0164] Population update: The particle swarm optimized by PSO is used as the initial firework population Fireworks_{new} for the next iteration of the firework algorithm.

[0165] Termination Check: Determine if the maximum number of iterations MaxIter or the fitness value Gbest meets the convergence accuracy requirement. If so, the algorithm terminates and outputs the global optimal solution Gbest, i.e., the optimal time allocation scheme; otherwise, return to step S52 to continue iteration.

[0166] In the above illustrative embodiment, the Fireworks Algorithm (IFWA) and the Particle Swarm Optimization (PSO) algorithm are sequentially fused. First, IFWA is used for wide-area exploration, and then PSO is used for local mining, effectively combining the powerful global search capability of the former with the efficient local convergence capability of the latter.

[0167] The IFWA algorithm introduces a mechanism that reduces the explosion radius over time, enabling the algorithm to maintain a wide search range in the early and middle stages and accelerate convergence in the later stages. The Cauchy mutation operator is used instead of the traditional Gaussian mutation, and its long-tail characteristics are used to enhance the global escape capability and avoid getting trapped in local optima.

[0168] The Particle Swarm Optimization (PSO) algorithm incorporates a nonlinear inertia weight reduction strategy and a dynamically adjusted learning factor. This allows the algorithm to focus on global exploration in the early stages of iteration and on fine-grained local search in the later stages, thereby accelerating the convergence speed and improving the optimization accuracy.

[0169] The combination of the two algorithms can directly optimize the time nodes of the 3-5-3 polynomial interpolation trajectory, with the goal of minimizing the total time and strictly satisfying the joint kinematic constraints, ultimately generating a smooth, efficient, and collision-free time-optimal motion trajectory, which directly improves the working efficiency of the cleaning device.

[0170] S6: Based on the optimal time-based motion trajectory control, the robotic arm moves to drive the cleaning device to clean the residue at the bottom of the collection bin.

[0171] Specifically, during the cleaning process, the optimal motion trajectory command is transmitted to the motion control unit of the robotic arm. The control unit controls the drive device in the execution module through the optimal command, so that the foldable robotic arm drives the air suction pipe 10 in the pneumatic module to move along the planned path. The air suction pipe installed on the foldable arm continuously suctions and cleans the remaining grain at the bottom of the auger through the negative pressure suction provided by the negative pressure fan 6. The remaining grain collected through the air suction pipe is separated by the spiral separator 4 and enters the remaining grain filter collection device 7 through the air suction pipe 3.

[0172] Some embodiments of the present invention further provide a residue cleaning system for a wheat harvester, used to implement the above-described residue cleaning method for a wheat harvester, as shown in the attached figure. Figure 2 -Appendix Figure 3 As shown, the residue cleaning system includes a sensing module, an execution module, a decision control module, and a pneumatic module.

[0173] Image acquisition device, residue cleaning device, and decision control module.

[0174] The sensing module includes an image acquisition device and a data storage module. The image acquisition device is installed inside the feed hopper of the wheat harvester to capture images of the bottom of the feed hopper, and the data storage module stores the image data acquired by the image acquisition device. Specifically, the image acquisition device is a monocular depth industrial camera, which includes an RGB camera and a depth camera.

[0175] The execution module includes a residue cleaning device and a corresponding drive device for the residue cleaning device. The residue cleaning device is set inside the collection bin of the wheat harvester and includes a robotic arm and an air suction pipe. The robotic arm is mounted on the inner wall 1 of the collection bin via a slide rail 2. One end of the air suction pipe is connected to the end of the robotic arm away from the slide rail, and the other end extends to the outside of the collection bin and is electrically connected to the negative pressure fan 6 via an air suction pipe 3.

[0176] The decision control module is communicatively connected to both the image acquisition device and the residue cleaning device. The decision control module includes a residue target recognition model, a trajectory planning unit, and a control unit; the target recognition model is deployed on a Jetson Nano controller, while the trajectory planning unit and control unit are deployed on an STM32 controller.

[0177] The residual object recognition model is used to identify objects in the images of the bottom of the collection bin acquired by the image acquisition device and output the predicted bounding boxes of each residual object at the bottom of the collection bin; the trajectory planning unit is used to perform coordinate transformation on the pixel coordinates of the center point of the predicted bounding box of each residual object and then perform robotic arm motion trajectory planning to obtain the time-optimal motion trajectory; the control unit is used to drive the robotic arm of the residual cleaning device to move based on the time-optimal motion trajectory.

[0178] In some embodiments of the present invention, a pneumatic module is further included. The pneumatic module includes a negative pressure fan 6 and a filter collection device 7.

[0179] The filter collection device is connected to the air suction pipe and the fan connection pipe 5 of the negative pressure fan through a spiral separator, and is used to receive the residue sucked out by the air suction pipe.

[0180] In the above illustrative embodiment, the residue cleaning system, through a closed-loop design of "perception-decision-execution-pneumatics", organically integrates monocular vision, Jetson Nano and STM32 embedded AI computing, multi-degree-of-freedom foldable robotic arm, and negative pressure air suction system to form a complete autonomous operation system. It can automatically complete the entire process of "image acquisition-residue identification and positioning-trajectory planning-motion control-suction cleaning", achieving complete automation without human intervention.

[0181] Furthermore, the actuator features a foldable robotic arm design, which can be retracted when not in operation, without interfering with the normal harvesting operation of the harvester, thus solving the problem of functional conflicts between specialized equipment and the main unit of the wheat harvester. Simultaneously, the system supports the distributed installation of multiple cleaning units within the grain silo, fundamentally eliminating cleaning blind spots, ensuring thorough cleaning, and meeting the near-stringent requirements for seed purity in seed production operations.

[0182] In a specific embodiment of the present invention, the workflow of the residue cleaning system is as follows:

[0183] After the wheat in the seed-producing wheat harvester's hopper is discharged through the bottom auger 11, the foldable robotic arm 8 mounted above the hopper unfolds via a servo motor, moving its end-mounted monocular depth industrial camera 10 to the optimal observation point in the hopper's working area. The camera takes multiple shots of the hopper's bottom from one side to the other, acquiring high-quality RGB color images. Simultaneously, it uses the distance information of each pixel from the camera to form a depth map. In the complex environment of the hopper, it accurately and automatically identifies and locates the three-dimensional spatial position of residual wheat, completing the initial image data acquisition function and providing a data foundation for subsequent intelligent trajectory planning. The image and depth data acquired by the sensing module are then processed and analyzed by a Jetson Nano controller that deploys a residual target recognition model to obtain the residual's location information.

[0184] The trajectory planning unit calculates the robotic arm's motion trajectory based on the location information data of the residue, obtains the optimal motion trajectory in time, and transmits the trajectory command to the control unit. The control unit controls the drive device in the execution module through the optimal command, so that the foldable robotic arm drives the air suction pipe 10 in the pneumatic module to move along the planned path. The air suction pipe installed on the foldable arm continuously cleans the residue at the bottom of the auger through the negative pressure suction provided by the negative pressure fan 6. The residue collected through the air suction pipe is separated by the spiral separator 4 and enters the filter collection device 7 through the air suction pipe 3.

[0185] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0186] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A method for cleaning residue from a wheat harvester, characterized in that, The wheat harvester's hopper is equipped with a residue cleaning device and an image acquisition device. Both the residue cleaning device and the image acquisition device are mounted on the inner wall of the hopper via a foldable robotic arm. The residue cleaning method includes the following steps: Images of the bottom of the material silo are acquired using an image acquisition device, and the acquired images are preprocessed. Improve the YOLOv11 model and construct a residual target recognition model based on the YOLOv11 model; The preprocessed image is input into the residue target recognition model to perform residue target recognition on the preprocessed image and obtain the predicted bounding boxes of each residue target at the bottom of the collection bin. The coordinates of the center points of the predicted bounding boxes of each residual target are transformed to the three-dimensional coordinates of the robot arm's base coordinate system. Based on the three-dimensional coordinates of each residual target in the robot arm's base coordinate system, the robot arm's motion trajectory is planned to obtain the time-optimal motion trajectory. Based on the optimal motion trajectory, the robotic arm moves to drive the residue cleaning device to clean the residue at the bottom of the collection bin. The specific methods for improving the YOLOv11 model include the following steps: A C2PSA cross-stage local self-attention module is introduced into the backbone network of the YOLOv11 model; A CGLU gating mechanism is introduced into the C3k2 module of the backbone network of the YOLOv11 model to obtain the C3k2_SMPCGLU module; The SPPF spatial pyramid pooling module of the backbone network of the YOLOv11 model is replaced with the FPSConv feature pyramid shared convolution module; the FPSConv feature pyramid shared convolution module has three convolution branches with different dilation rates, which are used to extract features of different receptive fields. Replace the regular convolutional layers of the neck network in the YOLO11 model with CBS modules; The specific methods for planning the motion trajectory of a robotic arm include the following steps: Based on the three-dimensional coordinates of each residual target in the robot arm's base coordinate system, a fireworks algorithm is used to perform a wide-area search for the motion trajectory. After each iteration of the fireworks algorithm, a particle swarm algorithm is used to locally optimize the fireworks population generated by the fireworks algorithm, and finally, the time-optimal motion trajectory is obtained.

2. The method for cleaning residue from a wheat harvester according to claim 1, characterized in that, The method for identifying residual targets in preprocessed images specifically includes the following steps: The preprocessed image is input into the backbone network of the residual object target recognition model. The shallow and deep features of the image are extracted by the C2PSA cross-stage local self-attention module and the C3k2_SMPCGLU module. The FPSConv feature pyramid shared convolution module is used to extract features of different receptive fields to output multi-layer features of different receptive fields. The multi-layer features output from the backbone network of the residue target recognition model are input into the neck network of the residue target recognition model. The deep features are upsampled step by step, and the upsampled deep features are concatenated with the shallow features and then input into the CBS module for feature fusion to output multi-scale fused features. The multi-scale fusion features output from the neck network of the residue target recognition model are input into the detection head for target localization and classification, and the predicted bounding box, category label, category confidence and depth value of each residue target are output.

3. The method for cleaning residue from a wheat harvester according to claim 2, characterized in that, The method for residual target identification in the preprocessed image further includes the following steps: Predicted bounding boxes are filtered based on a predetermined confidence threshold to remove bounding boxes whose class confidence is lower than the predetermined confidence threshold. The non-maximum suppression algorithm is used to remove overlapping predicted bounding boxes, retain a unique prediction result for each residual region, and output the predicted bounding box, class confidence and depth value of the unique prediction result.

4. The method for cleaning residue from a wheat harvester according to claim 3, characterized in that, The image acquisition device includes an RGB camera and a depth camera; the image at the bottom of the material collection hopper includes an RGB image and a depth image, and the method for coordinate transformation specifically includes the following steps: The intrinsic and extrinsic parameters of the RGB camera and the depth camera are calibrated to obtain the intrinsic parameter matrix K and the extrinsic parameter matrix [R|t] between them; Obtain the coordinates (u,v) of the center point of the predicted bounding box of the residual target and its corresponding depth value z; Based on the coordinates (u, v) of the predicted bounding box center point and the depth value z, the three-dimensional coordinates (X, Y, Z) of the predicted bounding box center point in the camera coordinate system are calculated using the following formula: ； in, , , , All of these are camera intrinsic parameters in the intrinsic parameter matrix K; Based on the extrinsic parameter matrix [R|t], the three-dimensional coordinates (X,Y,Z) in the camera coordinate system are transformed to the three-dimensional coordinates in the robot arm base coordinate system.

5. The method for cleaning residue from a wheat harvester according to claim 1, characterized in that, The method for performing local optimization specifically includes: Calculate the fitness value of each firework in the firework population; Based on the fitness value, a predetermined number of elite sparks are selected from the fireworks population; Elite Sparks are used as the initial particle population for the Particle Swarm Optimization (PSO) algorithm. During the iteration of the PSO algorithm, the position of each particle is updated to the individual optimal position and the global optimal position. The optimized particle population is then output as the initial fireworks population for the next iteration of the fireworks algorithm.

6. The method for cleaning residue from a wheat harvester according to claim 1, characterized in that, The method for preprocessing the acquired images specifically includes the following steps: The image of the bottom of the collection bin is scaled proportionally to the predetermined size, and blank space is filled around the image to ensure that the aspect ratio of the image remains unchanged; The image at the bottom of the collection bin is normalized and the pixel values ​​are scaled to a predetermined range. Adjust the image channels of the bottom of the aggregate bin according to the predetermined sequence.

7. A residue cleaning system for a wheat harvester, used to implement the residue cleaning method for a wheat harvester according to any one of claims 1-6, characterized in that, It includes an image acquisition device, a residue cleaning device, and a decision control module; The image acquisition device is installed inside the collection bin of the wheat harvester and is used to acquire images of the bottom of the collection bin; The residue cleaning device is installed inside the collection bin of the wheat harvester and includes a robotic arm and an air suction pipe; the robotic arm is mounted on the inner wall of the collection bin via a slide rail. One end of the air suction pipe is connected to the end of the robotic arm away from the slide rail, and the other end extends to the outside of the collection bin and is electrically connected to the negative pressure fan. The decision control module is communicatively connected to both the image acquisition device and the residue cleaning device. The decision control module includes a residue target recognition model, a trajectory planning unit, and a control unit. The residue target recognition model is used to identify targets in the image of the bottom of the collection hopper acquired by the image acquisition device and output predicted bounding boxes for each residue target at the bottom of the collection hopper. The trajectory planning unit is used to perform coordinate transformation on the coordinates of the center point of the predicted bounding box of each residue target, thereby planning the robotic arm's motion trajectory to obtain the time-optimal motion trajectory. The control unit is used to drive the robotic arm of the residue cleaning device to move based on the time-optimal motion trajectory.

8. The residue cleaning system for a wheat harvester according to claim 7, characterized in that, Further includes a filter collection device; The filter collection device is connected to the air suction pipe and the negative pressure fan respectively through a spiral separator, and is used to receive the residue sucked out by the air suction pipe.

Images