Combine harvester grain impurity rate and breakage rate monitoring system and method

Abstract

The invention provides a grain impurity rate and breakage rate monitoring system and method for a combine harvester. The grain impurity rate and breakage rate monitoring system comprises a grain sampling device, an image acquisition device and a control device, the grain sampling device is installed on the side wall of the vertical grain conveying barrel, a grain inlet of the grain sampling device is connected with a grain outlet hole in the side wall of the vertical grain conveying barrel through a hose, and a grain outlet of the grain sampling device is connected with a feeding port of the image acquisition device through a connecting pipe; the image acquisition device is connected with the control device, an image acquisition module of the control device controls the image acquisition device to acquire a grain image, and online calculation of the impurity rate and the breakage rate of the grain is realized on the grain image through a YOLOv8-Seg instance segmentation model of the image processing module in combination with an impurity rate and breakage rate mathematical model; finally, a detection result is displayed on a display screen in real time, visual numerical values can be provided for an operator of the combine harvester, and the operator is helped to adjust working parameters of the harvester. The method can be used for harvesting rice, wheat, oilseed rape and soybeans, and has wide applicability.

Description

Technical Field

[0001] The present invention belongs to the field of operating performance monitoring of combine harvesters for grains such as rice and wheat, and particularly relates to a system and method for monitoring the impurity content and breakage rate of grains of a combine harvester. Background Art

[0002] Combine harvesters can complete multiple operations such as harvesting, threshing, separating and cleaning grains in the field at one time with high operating efficiency. At present, rice harvesting has become an indispensable agricultural machinery in my country's rice production. The impurity / breakage rate of grain in the grain box is an important indicator to measure the operating performance of the combine harvester. If the impurity rate of grain is too high, it will not only affect the fluidity of grains (rice, wheat, etc.) during the unloading process and thus affect the unloading efficiency, but also directly affect the price of grain sold by farmers to the grain station; a high grain breakage rate will affect the whole rice rate of rice grain polishing, and after the grain is broken, the natural protective effect of the shell is lost. The grain is very easy to absorb water, clump, mildew and oxidize and rancidify fatty acids[7]. It is difficult to store and seriously affects the germination rate of seed grains, resulting in a decrease in the seedling rate. At present, some studies have tried to install a detection device at the grain outlet, but due to the high grain ejection speed, sampling is difficult and grains are piled up, affecting the grain output efficiency. In addition, there is serious dust in the grain box, and the unprotected camera lens is prone to dust accumulation, resulting in a decrease in image quality and restricting the stability and reliability of the monitoring system. Summary of the Invention

[0003] To address the above technical issues, the present invention provides a system and method for monitoring the impurity and breakage rates of grains in a combine harvester, thereby improving monitoring accuracy and operating efficiency. The present invention can be used for harvesting grains such as rice and wheat and has wide applicability.

[0004] Note that the inclusion of these objectives does not preclude the existence of other objectives. One embodiment of the present invention does not necessarily achieve all of the above objectives. Objectives other than the above objectives may be extracted from the description of the specification, drawings, and claims.

[0005] The present invention achieves the above technical objectives through the following technical means.

[0006] A combine harvester grain impurity rate and breakage rate monitoring system includes a grain sampling device, an image acquisition device and a control device;

[0007] The grain sampling device is installed on the side wall of the vertical grain conveying cylinder, the grain inlet of the grain sampling device is connected to the grain outlet hole on the side wall of the vertical grain conveying cylinder through a hose, and the grain outlet of the grain sampling device is connected to the feed inlet of the image acquisition device through a connecting pipe; the image acquisition device is connected to the control device, and the image acquisition module of the control device controls the image acquisition device to acquire grain images, and processes the grain images through the YOLOv8-Seg instance segmentation model of the image processing module, extracts the pixel values ​​of various components, and uses them as inputs of the mathematical model of impurity content and breakage rate to calculate and output the grain impurity content and breakage rate.

[0008] In the above scheme, the grain sampling device includes a grain barrel, an auger shaft and a reduction motor; a first gap sleeve is provided at one end of the grain barrel, and a second gap sleeve is provided at the other end, a first bearing is provided in the first gap sleeve, and a second bearing is provided in the second gap sleeve, one end of the auger shaft is connected to the first bearing, and the other end is connected to the second bearing; the grain inlet is arranged at one end of the grain barrel, and the grain outlet is arranged at the other end of the grain barrel; a motor sleeve is provided on the second bearing, and the reduction motor is installed on the motor sleeve, and the output shaft of the reduction motor is connected to the auger shaft to drive the auger shaft to rotate, thereby transporting the grains from the grain inlet to the grain outlet.

[0009] In the above solution, the image acquisition device includes a housing, a feeder, a receiving trough, a camera bracket, a light source bracket, a transparent glass plate, an L-shaped bracket and an optical acquisition system;

[0010] The feeder, receiving trough, camera bracket, light source bracket, transparent glass plate, L-shaped bracket and optical acquisition system are all installed in the housing;

[0011] The camera bracket, light source bracket and L-shaped bracket are installed on the inner wall of the shell from top to bottom in sequence by bolts; the optical acquisition system includes an industrial camera and a ring light source; the ring light source is installed on the light source bracket, the glass plate is fixed on the L-shaped bracket, the center of the industrial camera and the ring light source are on the same axis, and the industrial camera is located above the ring light source; by adjusting the light source bracket, the ring light source is located at the center of the glass plate and above it, providing fill light for the industrial camera when collecting pictures; a material receiving trough is provided below the optical acquisition system, one end of the material receiving trough is facing the material inlet of the shell, and the other end is facing the material outlet of the shell, the feeder is installed on the bottom plate of the shell of the image acquisition device, the feeder is connected to the bottom of the material receiving trough, and is used to transfer the grains in the material receiving trough to the field of view of the optical acquisition system, and finally fall into the recovery trough through the material outlet of the shell; the industrial camera selects a high frame rate color industrial camera under the premise of ensuring pixels, and the industrial camera shoots the grain flow entering the material receiving trough during operation through the transparent glass plate through the middle hole of the ring light source, obtains the image of the impurity information of the grain during movement, and transmits it to the control device.

[0012] Furthermore, the control device includes an embedded processor, a light source controller and a speed regulator;

[0013] The embedded processor is connected to the industrial camera, the light source controller is connected to the annular light source, and the speed regulator is connected to the reduction motor of the grain sampling device to control the flow of grains falling into the image acquisition device.

[0014] Furthermore, the control device also includes a display screen; the display screen is connected to the embedded processor.

[0015] In the above scheme, the YOLOv8-Seg instance segmentation model is: YOLOv8-Seg+C2f-Ghost+ShuffleAttention+Focal Loss instance segmentation model;

[0016] The C2f module in the original YOLOv8-Seg model is replaced with C2f-Ghost, which uses the GhostBottleneck module.

[0017] The Shuffle Attention module is added after the three effective feature layers in the CSPDarknet53 backbone network of the original YOLOv8-Seg model;

[0018] Replace the BCE Loss classification loss function in the original YOLOv8-Seg model with the Focal Loss loss function.

[0019] In the above scheme, the mathematical model of the impurity content and breakage rate is:

[0020]

[0021] Where P i Indicates rice impurity rate, %; m i Indicates the mass of rice residues, which include rice stems and rice branches, m h Indicates the mass of rice stalks, in g; m s Indicates the mass of rice stems, unit is g; m g Indicates the weight of complete rice grains. Since the shape, size and mass of complete rice grains are similar, the complete rice mass is expressed by multiplying the number of rice grains by the thousand-grain weight of rice. N represents the number of rice grains, and m represents the total weight of rice grains. q Indicates the thousand-grain weight of rice, in g; m b Indicates the mass of broken rice grains, unit is g; S h Represents the pixel area of ​​rice stems, unit is pix; S s Indicates the pixel area of ​​rice stems, unit is pix; S b Indicates the pixel area of ​​broken grains, unit is pix; Pb Indicates the rice breakage rate, %; m g Indicates the weight of complete rice grains, in g; N indicates the number of complete rice grains, in m q Indicates the thousand-grain weight of rice, in g; S b Indicates the pixel area of ​​broken grains, unit is pix.

[0022] A combine harvester comprises the above-mentioned combine harvester grain impurity rate and breakage rate monitoring system.

[0023] A control method according to the above-mentioned combine harvester grain impurity rate and breakage rate monitoring system comprises the following steps:

[0024] The grain sampling device collects grains from the vertical grain conveyor and transmits them to the image acquisition device.

[0025] The image acquisition device takes pictures of the grain flow entering during the operation, obtains images of grain impurity information during the movement, and transmits them to the control device;

[0026] The image acquisition module of the control device controls the image acquisition device to acquire grain images, and processes the grain images through the YOLOv8-Seg instance segmentation model of the image processing module, extracts the pixel values ​​of various components (the pixel values ​​of broken grains and debris and the number of intact grains), and uses them as inputs of the mathematical model of the impurity content and breakage rate to calculate and output the impurity content and breakage rate of the grains.

[0027] In the above scheme, the YOLOv8-Seg instance segmentation model is: YOLOv8-Seg+C2f-Ghost+ShuffleAttention+Focal Loss instance segmentation model; the YOLOv8-Seg+C2f-Ghost+ShuffleAttention+Focal Loss instance segmentation model processes the grain image specifically including the following steps:

[0028] The backbone network CSPDarknet53 in the YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model performs multi-scale feature extraction on images collected by the image acquisition device through the CBS (Conv+BN+SiLU) module, the C2f-Ghost module, and the SPPF (Spatial Pyramid Pooling) module. The depth-separable convolution of Ghost Bottleneck in the C2f-Ghost module significantly reduces the computational complexity.

[0029] Subsequently, the neck network (Neck) fuses deep and shallow features, integrating deep semantic features (high-level features) with shallow detail features (low-level features). The SA (ShuffleAttention) mechanism is embedded behind the three feature maps extracted by the backbone network to enhance the ability to distinguish small objects and complex backgrounds. The SA module uses channel grouping, dual-branch attention calculation, and feature reordering operations to enable the model to focus more on the edges of rice grains and small impurities.

[0030] In the detection head stage, the Box branch predicts the bounding box coordinates (x, y, w, h) and confidence (Confidence), the Cls branch predicts the target category (such as complete grains, broken grains, branches and stems), and the Mask branch outputs the mask coefficient (MaskCoefficients) for subsequent mask generation; the specific steps of subsequent mask generation are: on the multi-scale feature map output by Neck, 1×1 convolution is used to generate K prototype masks Proto (Prototype Masks), in the detection head Head stage, the Mask branch predicts a set of K-dimensional mask coefficients (Mask Coefficients) for each detected target, which are used to weightedly combine the prototype mask Proto. For example, if a rice grain is detected, the Mask branch will output K weight values ​​of the rice grain to adjust the contribution of the prototype mask Proto, and combine the mask coefficients (Mask Coefficients) with the prototype mask Proto (Prototype Masks) are subjected to matrix multiplication + Sigmoid activation to generate the instance mask of the target. Finally, the classification result optimized by the FocalLoss loss function and the detection box regressed by the CIoU-DFL loss function are combined, and the masked instance segmentation result is output after screening by the NMS non-maximum suppression module. The mask resolution is restored to the original image size through bilinear interpolation, realizing the accurate separation and statistics of rice grains and impurities.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The grain sampling device of the present invention is connected to the grain discharge hole on the side wall of the vertical grain conveyor, and the grain discharge port of the grain sampling device is connected to the feed port of the image acquisition device. The image acquisition module of the control device controls the image acquisition device to capture grain images. The image processing module processes the grain images using the YOLOv8-Seg instance segmentation model, extracting pixel values ​​of various components as input to the mathematical model for calculating and outputting the grain impurity and breakage rates. This improves monitoring accuracy and operational efficiency.

[0033] 2. The combine harvester's impurity and breakage rate monitoring system, installed on the side wall of the vertical grain conveyor, can avoid sample accumulation at the grain outlet due to high-speed grain ejection, ensuring the smooth flow of the grain conveying system. It can also overcome the interference of the high dust environment in the closed grain tank on the imaging equipment, preventing the adhesion of fine particles and affecting the image acquisition quality.

[0034] 3. The grain sampling device of this invention transports grains to the receiving trough of the image acquisition device via an auger shaft. A speed regulator controls the flow of grains into the image acquisition device. Grains enter the angled receiving trough through the grain sampling device's outlet. Adjusting the vibration frequency of the feeder allows precise control of the grain flow rate. Grains enter the imaging field of view through a transparent glass-sealed channel. This enclosed design effectively prevents dust contamination, providing high-quality image acquisition conditions for visual inspection and achieving controllable processing from sampling to imaging.

[0035] 3. The control method of the combine harvester grain impurity and breakage rate monitoring system of the present invention constructs an image acquisition module through the YOLOv8-Seg instance segmentation model combined with a mathematical model for calculating the grain impurity and breakage rate based on the quality-pixel mapping relationship. The embedded processor calls the industrial camera to capture images of the grains. The image acquisition module processes the images and obtains the values ​​of the grain impurity and breakage rate, realizing a rapid conversion from visual information to quality parameters. The processing results are displayed in real time through the human-computer interaction interface of the display screen, providing intuitive values ​​to the combine harvester operator to help the operator adjust the working parameters of the harvester.

[0036] 4. The present invention can be used for harvesting rice, wheat, rapeseed and soybean, and has wide applicability.

[0037] Note that the description of these effects does not preclude the existence of other effects. One embodiment of the present invention does not necessarily have all of the above effects. Effects other than the above can be clearly seen and extracted from the description of the specification, drawings, claims, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The present invention is a front view of a device for monitoring the impurity content and breakage rate of grains according to an embodiment of the present invention, shown in FIG.

[0039] Figure 2 This is a schematic structural diagram of a grain sampling device according to one embodiment of the present invention, wherein: Figure 2 (a) is a cross-sectional view of the grain sampling device. Figure 2 (b) Schematic diagram of the explosion of the grain sampling device structure.

[0040] Figure 3 This is a control principle and flow chart of an embodiment of the present invention, wherein: Figure 3(a) is the control principle diagram, Figure 3 (b) is a flow chart.

[0041] Figure 4 is a Canny method for extracting grain pixels according to an embodiment of the present invention, wherein: Figure 4 (a) is the grain-original image, Figure 4 (b) is the original image of haulm, Figure 4 (c) is the stem-original image, Figure 4 (d) is the broken original image, Figure 4 (e) is grain-Canny, Figure 4 (f) is haulm–Canny, Figure 4 (g) is stem–Canny, Figure 4 (h) is broken–Canny;

[0042] Figure 5 is a function fitting image of the mass of each component of cereals versus pixel area according to one embodiment of the present invention, wherein: Figure 5 (a) is the function fitting image of rice broken grain mass-pixel area, where Figure 5 (b) Function fitting image of rice stalk mass-pixel area, Figure 5 (c) Function fitting image of rice stalk mass-pixel area;

[0043] Figure 6 It is a typical sample of a monitoring device according to one embodiment of the present invention.

[0044] Figure 7 The Labelme annotation interface of one embodiment of the present invention, wherein: Figure 7 (a) is the Labelme annotation interface, (b) is the Labelme annotation image;

[0045] Figure 8 : is a data expansion effect diagram of an embodiment of the present invention, wherein: Figure 8 (a) is the original image, Figure 8 (b) is the mirror image. Figure 8 (c) is a vertical flip image. Figure 8 (d) is the Gaussian blurred image.

[0046] Figure 9 This is a diagram of the improved YOLOv8-Seg structure according to one embodiment of the present invention.

[0047] Figure 10 This is a Mosaic data enhancement strategy according to one embodiment of the present invention.

[0048] Figure 11This is a comparison chart of model training loss according to one embodiment of the present invention, where: Figure 11 (a) is Box-loss, Figure 11 (b) is seg-loss, Figure 11 (c) is cls-loss, Figure 11 (d) is dfl-loss.

[0049] Figure 12 is a YOLOv8-SegPR graph according to an embodiment of the present invention, Figure 12 (a) is Box(PR), Figure 12 (b) is Mask (PR).

[0050] Figure 13 is an improved YOLOv8-SegPR graph according to an embodiment of the present invention, Figure 13 (a) is Box(PR), Figure 13 (b) is Mask (PR).

[0051] Figure 14 This is a comparison chart of the improved grain recognition effect of one embodiment of the present invention. Figure 14 (a) is the original image, Figure 14 (b) is the YOLOv8-Seg model, Figure 14 (c) is the improved model.

[0052] Figure 15 PR diagrams of different models according to one embodiment of the present invention. Figure 15 (a) is Box-PR, Figure 15 (b)Mask-PR.

[0053] Figure 16 This is a monitoring interface according to an embodiment of the present invention.

[0054] The figure includes 1 - grain box, 2 - feeder, 3 - receiving trough, 4 - L-shaped bracket, 5 - glass plate, 6 - ring light source, 7 - light source bracket, 8 - industrial camera, 9 - camera bracket, 10 - embedded processor, 11 - light source controller, 12 - speed regulator, 13 - control device, 14 - grain outlet, 15 - connecting pipe, 16 - flexible pipe, 17 - grain sampling device, 1701 - grain outlet, 1703 - first gap sleeve, 1708 - second gap sleeve, 1702 - first bearing, 1710 - second bearing, 1704 - spiral blade, 1705 - auger shaft, 1706 - grain barrel, 1707 - grain inlet, 1709 - fixing plate, 1711 - motor sleeve. 18 - clamp, 19 - reduction motor, 20 - image acquisition device, 21 - vertical grain conveyor. DETAILED DESCRIPTION

[0055] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0056] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "front", "back", "left", "right", "up", "down", "axial", "radial", "vertical", "horizontal", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0057] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0058] Figure 1 FIG. 1 shows a preferred embodiment of the combined harvester grain impurity rate and breakage rate monitoring system, which includes a grain sampling device 17, an image acquisition device 20, and a control device 13.

[0059] The grain sampling device 17 is installed on the side wall of the vertical grain conveying cylinder 21 through a clamp 18, and the grain inlet 1707 of the grain sampling device 17 is connected to the grain outlet 14 on the side wall of the vertical grain conveying cylinder 21 through a hose 16, and the grain outlet 1701 of the grain sampling device 17 is connected to the feeding port of the image acquisition device 20 through a connecting pipe 15; the image acquisition device 20 is connected to the control device 13, and the image acquisition device 20 and the control device 13 are respectively fixed to the upper and lower parts of the outer wall of the grain box 1. The image acquisition module of the control device 13 controls the image acquisition device 20 to collect grain images, and processes the grain images through the YOLOv8-Seg instance segmentation model of the image processing module, extracts the pixel values ​​of various components, and uses them as inputs of the mathematical model of impurity content and breakage rate to calculate and output the grain impurity content and breakage rate.

[0060] like Figure 2 As shown, Figure 2 (a) is a cross-sectional view of the grain sampling device. Figure 2 (b) is an exploded schematic diagram of the grain sampling device structure. The grain sampling device 17 includes a grain barrel 1706, an auger shaft 1705, and a reduction motor 19. One end of the grain barrel 1706 is provided with a first clearance sleeve 1703, and the other end is provided with a second clearance sleeve 1708. The first clearance sleeve 1703 is provided with a first bearing 1702, and the second clearance sleeve 1708 is provided with a second bearing 1710. One end of the auger shaft 1705 is connected to the first bearing 1702, and the other end is connected to the second bearing 1710. Grain inlet 1707 is located at one end of grain barrel 1706, and grain outlet 1701 is located at the other end of grain barrel 1706. A motor housing 1711 is mounted on second bearing 1710, and reduction motor 19 is mounted on motor housing 1711. The output shaft of reduction motor 19 is connected to auger shaft 1705, which is welded with spiral blades 1704. The reduction motor 19 drives auger shaft 1705 to rotate, transporting grain from grain inlet 1707 to grain outlet 1701. Grain barrel 1706 is mounted on clamp 18 via fixing plate 1709.

[0061] Because the inner diameter of grain barrel 1706 does not match the outer diameters of first bearing 1702 and second bearing 1710, making them incompatible, two matching sleeves 1703 and 1708 are designed. The outer diameters of these sleeves 1703 and 1708 match the inner diameter of grain barrel 1706, and the inner diameters of these sleeves 1703 and 1708 match the outer diameters of first bearing 1702 and second bearing 1710. First bearing 1702 and second bearing 1710 are bolted to these sleeves 1703 and 1708, which are then bolted to grain barrel 1706. The inner diameter of motor sleeve 1711 matches the outer diameter of second bearing 1710 and is bolted to grain barrel 1706. By controlling the speed reduction motor 19 to rotate, the sampling device 17 can ensure that the grain sample falls from the grain inlet 1707 and is stably transported to the grain outlet 1701 .

[0062] The image acquisition device 20 includes a housing, a feeder 2, a receiving trough 3, a camera bracket 9, a light source bracket 7, a glass plate 5, an L-shaped bracket 4 and an optical acquisition system;

[0063] The feeder 2, the receiving trough 3, the camera bracket 9, the light source bracket 7, the transparent glass plate 5, the L-shaped bracket 4 and the optical acquisition system are all installed in the housing;

[0064] The camera bracket 9, light source bracket 7 and L-shaped bracket 4 are sequentially mounted on the inner wall of the shell by bolts from top to bottom; the optical acquisition system includes an industrial camera 8 and a ring light source 6; the ring light source 6 is mounted on the light source bracket 7 by bolts, the glass plate 5 is fixed on the L-shaped bracket 4, the industrial camera 8 and the ring light source 6 are centered on the same axis, and the industrial camera 8 is located above the ring light source 6; by adjusting the light source bracket 7, the ring light source 6 is located at the center of the glass plate 5 and above it, providing fill light for the industrial camera 8 when collecting pictures; a material receiving trough 3 is provided below the optical acquisition system, and the material receiving trough One end of 3 faces the feed port of the shell, and the other end faces the discharge port of the shell. The feeder 2 is installed on the inner bottom plate of the shell of the image acquisition device 20. The feeder 2 is connected to the bottom of the receiving trough 3, and is used to transfer the grains in the receiving trough 3 to the field of view of the optical acquisition system, and finally fall into the recovery trough through the discharge port of the shell; the industrial camera 8 selects a high frame rate color industrial camera while ensuring the pixel. The industrial camera 8 shoots the grain flow entering the receiving trough 3 during the operation through the transparent glass plate 5 through the middle hole of the annular light source 6, obtains the image of the impurity information of the grain during the movement, and transmits it to the control device 13.

[0065] By opening a window on the side wall of the vertical grain conveying cylinder 21 and installing a grain collecting hose 16, the centrifugal separation characteristics of the grain in the vertical grain conveying cylinder 21 are utilized, and the grains that move to the grain outlet 14 enter the hose 16 under the action of their own centrifugal force. The samples entering the hose 16 enter the grain sampling device 17, and the reduction motor 19 is fixed to the front end of the grain sampling device 17. The reduction motor 19 is controlled by the speed regulator 12 to control the flow of the grains. The grains fall continuously and stably into the receiving trough 3 in the image acquisition device 20 through the connecting pipe 15. The feeder 2 transports the grains in the receiving trough 3 to the shooting range of the industrial camera 8 through vibration. The embedded processor 11 calls the image acquisition module to capture images of the grains within the shooting range and transmits them to the image acquisition module. The grain image is processed by the YOLOv8-Seg instance segmentation model of the image processing module, and the pixel values ​​of various components are extracted. As the input of the mathematical model of the impurity content and breakage rate, the impurity content and breakage rate of the grain are calculated and output. The online calculation of the impurity rate and breakage rate of the grains is realized, and the test results and processed images are finally displayed on the display screen in real time.

[0066] like Figure 3 As shown, the control device 13 includes an embedded processor 10, a light source controller 11 and a speed regulator 12; Figure 3 As shown in (a), the USB port of the embedded processor 10 is connected to the industrial camera 8. The light source controller 11 is connected to the ring light source 6, and the light source intensity is manually adjusted to provide fill lighting for the shooting environment. The speed regulator 12 is connected to the reduction motor 19 of the grain sampling device 17, and the speed of the reduction motor 19 is manually adjusted to control the flow rate of grains falling into the image acquisition device 20. The control device 13 also includes a display screen; the display screen is connected to the HDMI port of the embedded processor 10. Preferably, the display screen is installed in the combine harvester cab and connected to the embedded processor 10 via an HDMI signal cable. The impurity / breakage rate and the processed image are displayed on the display screen.

[0067] Preferably, the embedded processor 10, based on the Ubuntu system and Python development environment, integrates a mathematical model for impurity and breakage rates based on the Canny edge extraction algorithm to ultimately calculate the impurity and breakage rates of the kernels. The system utilizes a multi-threaded architecture, enabling parallel operation and data isolation of functional modules such as image acquisition, processing, calculation, storage, and GUI interaction. Each module collaborates through main program scheduling, achieving a stable single processing cycle of less than 2.5 seconds.

[0068] The embedded processor 10 calls the image acquisition module to capture images of the grains within the shooting range and transmits them to the image processing system. The improved model YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss combines the impurity / breakage rate mathematical model with the image processing algorithm to realize the online calculation of the impurity rate and breakage rate of the grains. Finally, the detection results and the processed images are displayed on the display screen in real time, which can provide intuitive numerical values ​​for the combine harvester operator to help the operator adjust the working parameters of the harvester.

[0069] The system is equipped with an adjustable light source controller 11, which allows manual adjustment of the ring light source 6 intensity according to environmental conditions to ensure image quality. Simultaneously, the speed of the reduction motor 19 is manually adjusted via the speed regulator 12, precisely controlling the flow of grains into the image acquisition area. The workflow of this invention is as follows: First, the camera SDK is initialized and the image acquisition channel is established. After verifying that the image acquisition is normal, the system switches to automatic acquisition mode. Subsequently, the image processing program sequentially accesses the captured rice sample images. The image processing module in this example accurately segmentes and classifies the grains, stalks, branches, and broken grains. Based on a mathematical model for impurity and breakage rates, the pixel features of each component are converted into quality parameters, ultimately calculating the impurity and breakage rates. Image comparison, calculated numerical results, and a visual bar chart are displayed in real time on the human-computer interface. During operation, the system simultaneously stores the original image, processed image, and related data (number of categories, pixel values, and calculation results) in TXT format to an SD card. The entire processing process consists of a working cycle of approximately 2.5 seconds, and the system operates continuously through a loop mechanism until the task is completed.

[0070] The YOLOv8-Seg instance segmentation model is: YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model;

[0071] The C2f module in the original YOLOv8-Seg model is replaced with C2f-Ghost, which uses the GhostBottleneck module.

[0072] The Shuffle Attention module is added after the three effective feature layers in the CSPDarknet53 backbone network of the original YOLOv8-Seg model;

[0073] Replace the BCE Loss classification loss function in the original YOLOv8-Seg model with the Focal Loss loss function.

[0074] The mathematical model of the impurity content and breakage rate is:

[0075]

[0076]

[0077] Where, P i Indicates rice impurity rate, %; m i Indicates the mass of rice residues, which include rice stems and rice branches, m h Indicates the mass of rice stalks, in g; m s Indicates the mass of rice stems, unit is g; m g Indicates the weight of complete rice grains. Since the shape, size and mass of complete rice grains are similar, the complete rice mass is expressed by multiplying the number of rice grains by the thousand-grain weight of rice. N represents the number of rice grains, and m represents the total weight of rice grains. q Indicates the thousand-grain weight of rice, in g; m b Indicates the mass of broken rice grains, unit is g; S h Represents the pixel area of ​​rice stems, unit is pix; S s Indicates the pixel area of ​​rice stems, unit is pix; S b Indicates the pixel area of ​​broken grains, unit is pix; P b Indicates the rice breakage rate, %; m g Indicates the weight of complete rice grains, in g; N indicates the number of complete rice grains, in m q Indicates the thousand-grain weight of rice, in g; S b Indicates the pixel area of ​​broken grains, unit is pix.

[0078] A combine harvester comprises the above-mentioned combine harvester grain impurity rate and breakage rate monitoring system.

[0079] like Figure 3 As shown in (b), a control method of the combined harvester grain impurity rate and breakage rate monitoring system includes the following steps:

[0080] The grain sampling device 17 collects grains from the vertical grain conveying cylinder 21 and transmits them to the image acquisition device 20. The image acquisition device 20 photographs the grain flow entering during the operation, obtains images of grain impurity information during the movement, and transmits them to the control device 13.

[0081] The image acquisition module of the control device 13 controls the image acquisition device 20 to acquire grain images, and processes the grain images through the YOLOv8-Seg instance segmentation model of the image processing module, extracts the pixel values ​​of various components, the pixel values ​​of broken grains and residuals, and the number of complete grains, and uses them as inputs of the mathematical model of impurity content and breakage rate to calculate and output the impurity content and breakage rate of the grains.

[0082] The YOLOv8-Seg instance segmentation model is: YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model; the YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model processes the grain image specifically including the following steps:

[0083] The backbone network CSPDarknet53 in the YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model performs multi-scale feature extraction on the images collected by the image acquisition device 20 through the CBS (Conv+BN+SiLU) module, the C2f-Ghost module, and the SPPF (Spatial Pyramid Pooling) module. The depthwise separable convolution of GhostBottleneck in the C2f-Ghost module greatly reduces the computational complexity.

[0084] Subsequently, the neck network (Neck) fuses deep and shallow features, integrating deep semantic features (high-level features) with shallow detail features (low-level features). The SA (ShuffleAttention) mechanism is embedded behind the three feature maps extracted by the backbone network to enhance the ability to distinguish small objects and complex backgrounds. The SA module uses channel grouping, dual-branch attention calculation, and feature reordering operations to enable the model to focus more on the edges of rice grains and small impurities.

[0085] In the detection head stage, the Box branch predicts the bounding box coordinates (x, y, w, h) and confidence (Confidence), the Cls branch predicts the target category (such as complete grains, broken grains, branches and stems), and the Mask branch outputs the mask coefficient (MaskCoefficients) for subsequent mask generation; the specific steps of subsequent mask generation are: on the multi-scale feature map output by Neck, 1×1 convolution is used to generate K prototype masks Proto (Prototype Masks), in the detection head Head stage, the Mask branch predicts a set of K-dimensional mask coefficients (Mask Coefficients) for each detected target, which are used to weightedly combine the prototype mask Proto. For example, if a rice grain is detected, the Mask branch will output K weight values ​​of the rice grain to adjust the contribution of the prototype mask Proto, and combine the mask coefficients (Mask Coefficients) with the prototype mask Proto (Prototype Masks) are subjected to matrix multiplication + Sigmoid activation to generate the instance mask of the target. Finally, the classification result optimized by the FocalLoss loss function and the detection box regressed by the CIoU-DFL loss function are combined, and the masked instance segmentation result is output after screening by the NMS non-maximum suppression module. The mask resolution is restored to the original image size through bilinear interpolation, realizing the accurate separation and statistics of rice grains and impurities.

[0086] In a specific embodiment of the present invention, a control method according to the above-mentioned combine harvester grain impurity rate and breakage rate monitoring system comprises the following steps:

[0087] S1: A sample of rice mixture harvested from the field was selected from the grain tank 1 of the combine harvester. 1000 intact and healthy rice grains were selected and their thousand-grain weight was measured. q The weight of the rice stems was 27.55g, and 168 groups of rice stalks, 168 groups of broken grains, and 120 groups of stalks were selected. First, a calibration test bench was constructed. An appropriate amount of rice stem samples were evenly spread on a black background, ensuring that the samples were within the field of view of the industrial camera 8. The test bench equipment, such as the relevant parameters of the industrial camera 8, such as aperture, focal length, and object distance, were adjusted to ensure clear images could be captured using the computer acquisition software. Finally, the sample mass was measured using a precision electronic balance for each captured image and recorded in an Excel spreadsheet for use in establishing a mathematical model for impurity and breakage rates.

[0088] Specifically, the method for establishing a mathematical model for calculating the impurity / breakage rate of grains comprises the following steps:

[0089] like Figure 4 The above is a Canny method for extracting grain pixels according to an embodiment of the present invention. Figure 4(a) is the grain-original image, Figure 4 (b) is the original image of haulm, Figure 4 (c) is the stem-original image, Figure 4 (d) is the broken original image, Figure 4 (e) is grain-Canny, Figure 4 (f) is haulm–Canny, Figure 4 (g) is stem–Canny, Figure 4 (h) is broken–Canny. Preferably, 168 rice stalk images, 168 sets of broken grain images, and 120 sets of branch images were collected and the pixel area of ​​each image was extracted in batches using edge detection technology based on the Canny algorithm. The Canny algorithm can completely extract the boundaries of each target area, with clear and continuous edges and almost no excess noise, demonstrating excellent noise resistance and segmentation accuracy. The extracted pixel values ​​are highly consistent with the actual situation.

[0090] like Figure 5 The above is a function fitting image of the mass of each component of cereals-pixel area according to one embodiment of the present invention, wherein: Figure 5 (a) is the function fitting image of rice broken grain mass-pixel area, where Figure 5 (b) Function fitting image of rice stalk mass-pixel area, Figure 5 (c) Function fitting image of rice stalk mass versus pixel area; preferably, the data of the broken grain and residual mass versus pixel area calibration records are imported into MATLAB, and linear fitting is performed using the least squares method to obtain a function image of the mass of each component versus pixel area. The function of the mass of each component versus pixel area is obtained as follows:

[0091] m b =9.2488×10 -6 S b -0.0083#(1)

[0092] m h =8.0918×10 -6 S h +0.048#(2)

[0093] m s =1.4279×10 -6 S s +0.0036#(3)

[0094] where m b is the mass of broken rice grains (g), S b is the pixel area of ​​broken rice grains (pix); m h is the mass of rice stalk (g), S his the pixel area of ​​rice stalk (pix); m s is the mass of rice stems (g), S s is the pixel area of ​​rice stalks (pix); the function is used to calculate the impurity rate of grains and breakage rate The mathematical models of impurity rate and breakage rate are obtained from the calculation formula as follows:

[0095]

[0096] Where, P i Indicates rice impurity rate, %; m i Indicates the mass of rice residues, which include rice stems and rice branches, m h Indicates the mass of rice stalks, in g; m s Indicates the mass of rice stems, unit is g; m g Indicates the weight of complete rice grains. Since the shape, size and mass of complete rice grains are similar, the complete rice mass is expressed by multiplying the number of rice grains by the thousand-grain weight of rice. N represents the number of rice grains, and m represents the total weight of rice grains. q Indicates the thousand-grain weight of rice, in g; m b Indicates the mass of broken rice grains, unit is g; S h Represents the pixel area of ​​rice stems, unit is pix; S s Indicates the pixel area of ​​rice stems, unit is pix; S b Indicates the pixel area of ​​broken grains, unit is pix; P b Indicates the rice breakage rate, %; m g Indicates the weight of complete rice grains, in g; N indicates the number of complete rice grains, in m q Indicates the thousand-grain weight of rice, in g; S b Indicates the pixel area of ​​broken grains, unit is pix.

[0097] S2: The grain sampling device 17 obtains samples from the vertical grain conveying drum 21 of the combine harvester and transports them to the receiving trough 3 of the image acquisition device 20 under the action of the spiral auger 1705. The grains are transported to the field of view of the industrial camera 7 through vibration. The embedded processor 10 uses the industrial camera 7 to photograph the grains.

[0098] S3: The mixed / broken images taken in S2 are finely annotated using the Labelme annotation tool to create a self-made grain dataset, which is used for the instance segmentation model adopted in this example.

[0099] Specifically, the method for making a homemade grain dataset is as follows:

[0100] like Figure 6As shown, preferably, a self-made dataset of impurity / broken images taken in S2 is used, and the collected images contain broken kernels and other impurities other than kernels, totaling 2458 images, as the original dataset.

[0101] like Figure 7 As shown, Figure 7 (a) is the Labelme annotation interface, and (b) is the Labelme annotation image. Preferably, Labelme software is used to annotate the data. During the annotation process of the rice sample image, the polygon tool in Labelme is used to draw closed polygons for intact rice grains, stalks, branches, and broken grains to accurately mark the target area. After drawing the polygons, a dialog box will pop up, and the corresponding target category name will be entered. Use haulm for stalks, stem for branches, grain for intact grains, and broken for broken grains. After annotating an image, a JSON annotation file will be generated. The annotation data is saved in .json format and contains information such as the annotation coordinates and category.

[0102] like Figure 8 The figure shows the data expansion effect diagram of an embodiment of the present invention, wherein: Figure 8 (a) is the original image, Figure 8 (b) is the mirror image. Figure 8 (c) is a vertical flip image. Figure 8 (d) is a Gaussian blurred image. Preferably, in order to prevent the overfitting problem caused by the insufficient number of images in the data set, the training image data set is expanded by contrast enhancement, mirror inversion, vertical flipping, Gaussian blurring and other methods to improve the generalization ability of the network. The original 2458 original images are expanded to 9832, and the corresponding .json annotation files are expanded in the same way to ensure that the expanded images correspond to the annotation data to meet the needs of model training. The .json file generated by Labelme stores the coordinates of the annotated polygons, but YOLOv8-Seg training requires a normalized TXT format. Therefore, it is necessary to batch convert the .json files into .txt files through a python program, and make a rice data set by yourself and divide the training set, test set, and validation set into a ratio of 7:1:2.

[0103] S4: This example uses the improved model (YOLOv8-Seg + C2f-Ghost + Shuffle Attention + Focal Loss) for image processing. The dataset constructed in S3 is used for training this example model. The C2f-Ghost module reduces computational overhead, Shuffle Attention enhances the detection of small fragmented areas, and Focal Loss alleviates the problem of class imbalance.

[0104] like Figure 9 As shown, preferably, this example uses the improved model (YOLOv8-Seg+C2f-Ghost+ShuffleAttention+Focal Loss) method for image processing to accurately identify the various components of grains.

[0105] The original YOLOv8-Seg model includes the CSPDarknet53 backbone network, which includes the CBS (Conv+BN+SiLU) module, the C2f module, and the SPPF (Spatial Pyramid Pooling) module for extracting multi-scale features. This module suffers from high parameter count and computational complexity. Improvements to the original YOLOv8-Seg model include replacing the original C2f module with C2f-Ghost. C2f-Ghost uses the Ghost Bottleneck module, which uses a hybrid structure of stacked GhostConv and depthwise separable convolution to reduce the computational burden while maintaining feature extraction capabilities. Improvements to the original YOLOv8-Seg model also include adding the Shuffle Attention module after the three effective feature layers in the CSPDarknet53 backbone network. Improvements to the original YOLOv8-Seg model also include replacing the classification loss BCE Loss, which gives equal weight to each category in the YOLOv8-Seg model, with Focal Loss, a loss function that targets the problem of category imbalance.

[0106] Specifically, this study lightweighted the standard Bottleneck architecture in the original model for applications with limited computing resources in embedded devices. While the traditional Bottleneck architecture, which employs a dual Convolutional Broadcasting (CBS) architecture (Conv+Batch Normalization (BN)+SiLU), offers strong feature extraction capabilities, its standard convolution operation suffers from high computational complexity and a large number of parameters. To address this, the innovative Ghost Bottleneck module was introduced. This hybrid architecture, stacking GhostConv and depthwise separable convolution, reduces the computational burden while maintaining feature extraction capabilities.

[0107] In rice sample recognition tasks, branches and broken kernels are small targets. YOLOv8-segment (YOLOv8-segment) has difficulty capturing the boundary features and categories of small targets in these scenarios. By studying the working principles of various attention mechanisms, such as the SENet model, STN module, CA attention mechanism, CBAM, and ShuffleAttention module, we found that Shuffle Attention, through cross-channel information interaction and feature reorganization, enables the model to focus more on key information channels and suppress irrelevant or noisy features. Its computational complexity is also lower, making it suitable for edge computing tasks embedded in digital devices to optimize rice debris and broken kernel detection, improving the model's real-time performance and detection accuracy in embedded environments.

[0108] During model training, a loss function is required to continuously optimize model parameters. Different loss functions have different effects on the speed of model convergence. The YOLOv8-Seg model loss consists of classification loss, bounding box regression loss, and segmentation mask loss, which is a weighted sum of these three types of losses.

[0109] In rice detection tasks, there is often a significant imbalance between intact, broken, and impurity categories. Since intact kernels far outnumber broken kernels and impurities, the BCE loss function in the YOLOv8-Seg model, which assigns equal weight to each category, was replaced with a Focal Loss function that addresses this imbalance.

[0110] Specifically, this instance segmentation model (YOLOv8-Seg+C2f-Ghost+Shuffle Attention+FocalLoss), YOLOv8-Seg

[0111] First, the CSPDarknet53 backbone network extracts multi-scale features through the CBS (Conv+BN+SiLU) module, the improved C2f-Ghost module, and the SPPF (Spatial Pyramid Pooling) module. The depthwise separable convolution of the Ghost Bottleneck significantly reduces computational complexity. Subsequently, the PAN-FPN (Path Aggregation Network + Feature Pyramid) bottleneck network fuses deep and shallow features, fusing deep semantic features (high-level features) with shallow detail features (low-level features). The Shuffle Attention mechanism is embedded behind the three feature maps extracted by the backbone network to enhance the ability to distinguish small objects and complex backgrounds. This module uses channel grouping, dual-branch attention computation, and feature shuffling to focus on rice grain edges and fine impurities. During the prediction phase, the Box branch predicts bounding box coordinates (x, y, w, h) and confidence; the Cls branch predicts the object category (e.g., intact kernel, broken kernel, branch, and stem); and the Mask branch outputs mask coefficients for subsequent mask generation. Specifically, on the multi-scale feature map output by Neck, a 1×1 convolution is used to generate K prototype masks. Each mask represents a basic segmentation mode (such as edge, texture, shape, etc.). The Mask branch of the detection head predicts a set of K-dimensional coefficients for each detected target, which is used to weightedly combine the prototype masks. For example, if a rice grain is detected, the Mask branch will output K weight values ​​of the target to adjust the contribution of the prototype mask. The Mask Coefficients are matrix multiplied with the Prototype Masks + Sigmoid activation to generate the instance mask of the target. Finally, the classification results optimized by Focal Loss and the detection box of CIoU-DFL regression are combined, and the instance segmentation result with mask is output after NMS screening. The mask resolution is restored to the original image size through bilinear interpolation to achieve accurate separation and statistics of rice grains and impurities.

[0112] S5: Before model testing, it is necessary to set up the corresponding experimental environment according to the training hardware configuration, adjust the model training parameters, and prevent the network from overfitting. Operating system: 64-bit Windows 10 Professional, graphics card NVIDIA GeForce GTX 1060, based on the deep learning framework python3.8, pytorch1.9.0+cu111, CUDA11.1 and cuDNN8.0.50, and install the libraries that the model depends on, such as Torchvision0.10.0+cu111, Opencv4.7.0.72, Pillow9.4.0, etc. Use GPU acceleration to improve the speed of model training and inference, and speed up the training and inference speed of deep learning models, data processing and preprocessing, visualization and result analysis. Precision; Recall; Mean Average Precision (mAP); F1-score; Number of floating-point operations (FLOPs) and number of parameters (Paramrters) are selected as evaluation indicators. The formula is as follows:

[0113]

[0114] Where T p is the number of targets correctly identified, F p is the number of positive samples that are not identified or misidentified, F N is the number of targets that are mistakenly classified as negative samples.

[0115]

[0116]

[0117] S6: Parameter configuration for optimizing the image processing method used in this example. First, the input images are uniformly resized to a 640×640 resolution, and the Mosaic data augmentation strategy is enabled. By randomly splicing four images, small object detection is effectively improved. During the training process, 300 epochs are set with a batch size of 8 to ensure both GPU memory utilization and training stability. The optimizer uses SGD with a cosine annealing (Cos) learning rate scheduling strategy, with an initial learning rate of 0.01. The model scale parameter, Phi, is set to n, selecting a medium-sized variant to balance computational efficiency and detection accuracy. Furthermore, the training process employs optimizations such as automatic mixed precision (AMP) acceleration, 0.0005 weight decay, and early stopping (patience = 50) to ensure full model convergence within 300 epochs.

[0118] like Figure 10 As shown in the figure, the preferred Mosaic data enhancement strategy improves the robustness of the model for small target detection and complex background.

[0119] S7: The original model YOLOv8-Seg model is compared with the example model in the ablation test. The experiment uses the original YOLOv8-Seg as the baseline model and gradually introduces three improved modules: Ghost Bottleneck, Shuffle Attention and Focal Loss. Model1 is a lightweight test to verify the model. The C2f module in the YOLOv8-Seg backbone network is modified into the C2f-Ghost module. Model2 verifies whether the Shuffle Attention mechanism can effectively improve the model's ability to extract small target crop features. Shuffle Attention modules are added to the three valid feature maps extracted by the YOLOv8-Seg backbone network. Model3 verifies the effectiveness of the loss function optimization. The original YOLOv8-Seg classification loss function binary cross entropy (BCE Loss) is replaced by the Focal Loss function. Model4 and Model5 respectively verify the Shuffle Attention mechanism and Focal Loss. Ablation experiments show the impact of the loss function on lightweight modules. The original YOLOv8-Seg model achieved a Box mAP of 0.974 and a Mask mAP of 0.964, with 3,011,628 parameters and 8.2 GFLOPS. Model 1 (YOLOv8-Seg + C2f-Ghost) replaces the standard convolutional architecture with a lightweight C2f-Ghost module, effectively reducing model complexity from 3.01M parameters to 2.56M and computational GFLOPs from 8.2 to 6.5, demonstrating excellent lightweighting capabilities. However, this also results in a certain loss in accuracy: Box mAP drops from 0.974 to 0.923, and Mask mAP drops from 0.964 to 0.896. This shows that although Ghost convolution improves computational efficiency, its feature generation method is relatively sparse and its ability to express low-frequency information is weak. In particular, it has poor segmentation effects on rice images with blurred edges, small-scale impurities, or broken targets, resulting in decreased sensitivity. Model 2 (YOLOv8-Seg+Shuffle Attention) introduces a lightweight attention mechanism, which significantly improves model performance without significantly increasing the amount of computation (the number of parameters remains almost unchanged), with Box mAP increased to 0.981 and Mask mAP increased to 0.971.This module effectively enhances the feature representation capability through inter-channel information rearrangement and attention weighting. It is particularly suitable for rice images with complex backgrounds and difficult-to-distinguish foreground targets, and improves the ability to recognize cluttered and fragmented areas. Model 3 (YOLOv8-seg+Focal Loss) targets tasks with severely uneven target-background ratios by dynamically adjusting the focus of the loss function to enhance the model's learning ability for difficult-to-distinguish samples. Although the overall improvement in mAP is not significant, the Box mAP still basically maintains the level of the original model, indicating that it has a positive effect on improving the detection performance of "difficult-to-detect" samples such as branches and broken grains, and is suitable for situations where the contrast between the foreground and background in rice images is not high; on this basis, the module combination Model4 (YOLOv8-Seg+C2f-Ghost+Shuffle Attention) and Model5 (YOLOv8-seg+C2f-Ghost+Focal Loss) experiments further verified the complementarity of the improved strategies. The results show that after adding the attention mechanism or Focal Loss to the C2f-Ghost module, the model accuracy has recovered to a certain extent. Model4 partially compensates for the shortcomings of the Ghost module in feature expression by introducing the attention mechanism, and the Mask mAP is increased to 0.942; Model5 (YOLOv8-Seg+C2f-Ghost+Focal Loss) improves the model's recognition ability for difficult samples, and the Mask mAP is restored to 0.951. The improved model (YOLOv8-Seg + C2f-Ghost + Shuffle Attention + Focal Loss) achieves an optimal balance between accuracy and efficiency: Box mAP increases to 0.987, and Mask mAP increases to 0.985. Meanwhile, the model has only 2.69M parameters and 6.4 GFLOPs, outperforming the original YOLOv8-seg model. This demonstrates that the improved image processing method in this example can effectively improve the model's overall performance in segmentation tasks.

[0120] like Figure 11 The figure shows a comparison diagram of model training loss according to an embodiment of the present invention, wherein: Figure 11 (a) is Box-loss, Figure 11 (b) is seg-loss, Figure 11 (c) is cls-loss, Figure 11(d) is dfl-loss. Preferably, in order to deeply compare the performance differences between the original YOLOv8-Seg model and the improved model, a systematic analysis of the changes in various loss functions during the training phase of the original YOLOv8-Seg image processing model and the improved image processing model (YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss) used in this example was conducted, mainly including key indicators such as box_loss, cls_loss, dfl_loss, and seg_loss. With the increase in the number of training rounds, the box_loss, cls_loss, dfl_loss, and seg_loss of both models showed a relatively obvious downward trend. Specifically, the improved model showed faster convergence speed and lower stable value in both bounding box regression loss box_loss and segmentation loss seg_loss, indicating that the lightweight feature extraction capability of the C2f-Ghost module and the attention mechanism of Shuffle Attention effectively improved target positioning and segmentation accuracy. The classification loss curve, cls_loss, shows that the improved model achieves lower loss values ​​and less volatility after 150 epochs of training, demonstrating that Focal Loss alleviates class imbalance and enhances classification robustness in complex scenarios. The distribution focal loss, dfl_loss, further verifies the improved model's stability on boundary prediction tasks, with its curve consistently below the baseline and the final convergence value reduced by approximately 15%. Overall, the introduced modules and loss function optimizations significantly improve model performance, striking a balance between accuracy and efficiency.

[0121] like Figure 12 is a YOLOv8-SegPR graph according to an embodiment of the present invention, Figure 12 (a) is Box(PR), Figure 12 (b) is Mask (PR). Figure 13 is an improved YOLOv8-SegPR graph according to an embodiment of the present invention, Figure 13 (a) is Box(PR), Figure 13(b) is Mask (PR). Preferably, the PR (Precision-Recall) curves of the original YOLOv8-Seg model and the improved image processing model used in this example on four types of rice targets are shown respectively. The PR graph (Precision-Recall Curve) is an important tool for evaluating the performance of target detection and image segmentation models. By plotting the relationship between the changes in precision (Precision) and recall (Recall) under different confidence thresholds, the comprehensive detection capability of the model can be intuitively reflected. It can be clearly observed from the figure that the PR curves of the improved model on various targets are smoother and the area under the overall curve is larger, indicating that it has higher accuracy while maintaining a high recall rate. Especially for targets such as "stem" and "broken", where the recall ability of the original model is slightly weaker, the PR curve of the improved model is significantly closer to the upper right corner, showing stronger robustness and generalization ability. This further verifies that on the basis of lightweight, the improved module combination of this example significantly improves the detection and segmentation performance of the model in real complex scenes.

[0122] like Figure 14 The figure shows a comparison of the improved grain recognition effect of an embodiment of the present invention. Figure 14 (a) is the original image, Figure 14 (b) is the YOLOv8-Seg model, Figure 14 (c) is the improved model. Preferably, in order to more intuitively demonstrate the effectiveness of the improved YOLOv8-Seg model in detecting rice debris and broken targets, the recognition effect diagram of the test set is compared. For the convenience of comparison, the "grain" class label is removed. The figure shows three original images. Figure 14 (a) The effect of original model recognition Figure 14 (b) and the recognition effect of the improved YOLOv8-Seg model Figure 14 (c) As can be seen from the figure, the YOLOv8-Seg original model performs well in detecting intact rice grains (unlabeled) and stalks (labeled as haulm) with regular shapes and clear boundaries, and the target mask has clear contours and accurate coverage. However, for broken grains (labeled as broken) with fuzzy edges and irregular shapes and slender and low-contrast stalks (labeled as stem), there are problems such as missing stalks, misdetecting stalks as stalks, and misdetecting intact grains as broken grains. Figure 14(b) is highlighted with a black box. By introducing the attention mechanism Shuffle Attention, the model's perception of low-contrast and morphologically complex areas is enhanced. At the same time, the feature fusion structure is adjusted to improve the expression of fine-grained targets. The introduction of Focal Loss allows the model to focus on these difficult-to-classify targets (branches and broken grains) during training. The improved model Figure 14 (c) demonstrates significant advantages, effectively reducing missed and false detections of branches and stalks and significantly improving the accuracy of broken kernel recognition. The object mask boundary more closely matches the true morphology, enhancing overall segmentation accuracy. In particular, in areas with blurred boundaries between objects, the improved model demonstrates stronger discrimination capabilities, demonstrating greater robustness and generalization.

[0123] like Figure 15 Shown are different model PR diagrams of an embodiment of the present invention, Figure 15 (a) is Box-PR, Figure 15 (b) Mask-PR. Preferably, the improved YOLOv8-Seg model is compared with the current mainstream segmentation models YOLACT, Mask R-CNN, YOLOv5-Seg and the original YOLOv8-Seg on a self-built rice image dataset. To ensure the fairness and comparability of the experiment, all models are run in the same experimental environment, as described in S5. The specific parameter settings refer to the experimental parameters described in S6. The improved YOLOv8-Seg model shows significant advantages in the instance segmentation task. Both the Box-PR and Mask-PR curve areas are significantly better than the original YOLOv8-Seg, YOLOv5-Seg, the two-stage classic model Mask R-CNN and the single-stage model YOLACT, indicating that the improved model has been effectively optimized in feature extraction, target positioning and mask prediction, and can significantly reduce false detections while ensuring a high recall rate. It is particularly noteworthy that in the high recall region (Recall>0.8), the improved model maintains a relatively stable performance decay trend, while the precision of other models (especially YOLACT and YOLOv5-Seg) shows a significant decline, which further verifies the robustness of the improved model in complex scenarios. The experimental results fully demonstrate the effectiveness of the improved model.

[0124] S8: Deploy the improved YOLOv8-Seg instance segmentation model used in this example to the embedded processor 10. This model, combined with the impurity and breakage rate mathematical models obtained in S1, constructs an image detection system that combines image processing methods with the impurity / breakage rate mathematical models. Specifically, images captured by the industrial camera 7 are first segmented using the YOLOv8-Seg model to extract pixel values ​​for each component. This pixel data is then used as input for the mathematical model to calculate and output the impurity and breakage rates of the kernels. The processed image and calculation results are displayed on the screen, enabling rapid conversion from visual information to quality parameters.

[0125] like Figure 16 As shown, preferably, a human-computer interaction interface is designed to visualize the change of the impurity rate / breakage rate in a bar graph, providing intuitive numerical values ​​for the combine harvester operator to help the operator adjust the working parameters of the harvester.

[0126] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0127] The series of detailed descriptions listed above are only specific descriptions of feasible embodiments of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent embodiments or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A monitoring system for the impurity content and breakage rate of grains in a combine harvester, characterized in that: It includes a grain sampling device (17), an image acquisition device (20) and a control device (13); The grain sampling device (17) is installed on the side wall of the vertical grain conveying cylinder (21), the grain inlet (1707) of the grain sampling device (17) is connected to the grain outlet (14) on the side wall of the vertical grain conveying cylinder (21) through a hose (16), and the grain outlet (1701) of the grain sampling device (17) is connected to the feed port of the image acquisition device (20) through a connecting pipe (15); the image acquisition device (20) is connected to the control device (13), and the image acquisition module of the control device (13) controls the image acquisition device (20) to acquire grain images, and processes the grain images through the YOLOv8-Seg instance segmentation model of the image processing module, extracts the pixel values ​​of various components, and uses them as inputs of the mathematical model of impurity content and breakage rate to calculate and output the impurity content and breakage rate of the grains.

2. The combine harvester grain impurity rate and breakage rate monitoring system according to claim 1, characterized in that: The grain sampling device (17) comprises a grain barrel (1706), an auger shaft (1705) and a reduction motor (19); one end of the grain barrel (1706) is provided with a first gap sleeve (1703), and the other end is provided with a second gap sleeve (1708); a first bearing (1702) is provided in the first gap sleeve (1703), and a second bearing (1710) is provided in the second gap sleeve (1708); one end of the auger shaft (1705) is connected to the first bearing (1702), and the other end is connected to the second bearing (1710). The second bearing (1710) is connected; the grain inlet (1707) is arranged at one end of the grain barrel (1706), and the grain outlet (1701) is arranged at the other end of the grain barrel (1706); a motor sleeve (1711) is sleeved on the second bearing (1710), and a reduction motor (19) is installed on the motor sleeve (1711); the output shaft of the reduction motor (19) is connected to the auger shaft (1705), driving the auger shaft (1705) to rotate, thereby transporting the grains from the grain inlet (1707) to the grain outlet (1701).

3. The system for monitoring the impurity content and breakage rate of grains in a combine harvester according to claim 1, characterized in that: The image acquisition device (20) comprises a housing, a feeder (2), a receiving trough (3), a camera bracket (9), a light source bracket (7), a transparent glass plate (5), an L-shaped bracket (4), and an optical acquisition system; The feeder (2), the receiving trough (3), the camera bracket (9), the light source bracket (7), the transparent glass plate (5), the L-shaped bracket (4) and the optical collection system are all installed in the housing; The camera bracket (9), the light source bracket (7) and the L-shaped bracket (4) are sequentially mounted on the inner wall of the housing from top to bottom by bolts; the optical acquisition system comprises an industrial camera (8) and an annular light source (6); the annular light source (6) is mounted on the light source bracket (7), the glass plate (5) is fixed on the L-shaped bracket (4), the industrial camera (8) and the annular light source (6) are coaxial in center, and the industrial camera (8) is located above the annular light source (6); a material receiving trough (3) is provided below the optical acquisition system, and the material receiving trough ( One end of the feeder (3) faces the feed inlet of the shell, and the other end faces the discharge outlet of the shell. The feeder (2) is installed on the bottom plate of the shell of the image acquisition device (20). The feeder (2) is connected to the bottom of the receiving trough (3) and is used to transfer the grains in the receiving trough (3) to the field of view of the optical acquisition system. The industrial camera (8) takes pictures of the grain flow entering the receiving trough (3) during the operation through the middle hole of the annular light source (6) through the transparent glass plate (5), obtains the impurity information image of the grains during the movement, and transmits it to the control device (13).

4. The system for monitoring the impurity content and breakage rate of grains in a combine harvester according to claim 3, characterized in that: The control device (13) includes an embedded processor (10), a light source controller (11) and a speed regulator (12); The embedded processor (10) is connected to the industrial camera (8), the light source controller (11) is connected to the annular light source (6), and the speed regulator (12) is connected to the reduction motor (19) of the grain sampling device (17) to control the flow rate of the grains falling into the image acquisition device (20).

5. The system for monitoring the impurity content and breakage rate of grains in a combine harvester according to claim 4, characterized in that: The control device (13) further includes a display screen; The display screen is connected to the embedded processor (10).

6. The combined harvester grain impurity rate and breakage rate monitoring system according to claim 1, characterized in that: The YOLOv8-Seg instance segmentation model is: YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model; The C2f module in the original YOLOv8-Seg model is replaced with C2f-Ghost, which uses the GhostBottleneck module. The ShuffleAttention module is added after the three effective feature layers in the backbone network CSPDarknet53 of the original YOLOv8-Seg model; Replace the BCE Loss classification loss function in the original YOLOv8-Seg model with the Focal Loss loss function.

7. The system for monitoring the impurity content and breakage rate of grains in a combine harvester according to claim 1, characterized in that: The mathematical model of the impurity content and breakage rate is: Where P i Indicates rice impurity rate, %; m i Indicates the mass of rice residues, which include rice stems and rice branches, m h Indicates the mass of rice stalks, unit is g; m s Indicates the mass of rice stems, unit is g; m g Indicates the weight of complete rice grains. Since the shape, size and mass of complete rice grains are similar, the complete rice mass is expressed by multiplying the number of rice grains by the thousand-grain weight of rice. N represents the number of rice grains, and m represents the total weight of rice grains. q Indicates the thousand-grain weight of rice, in g; m b Indicates the mass of broken rice grains, unit: g; S h Represents the pixel area of ​​rice stems, unit is pix; S s Represents the pixel area of ​​rice stalks, unit is pix; S b Indicates the pixel area of ​​broken grains, unit is pix; P b It represents the rice breakage rate, %; m g Indicates the weight of complete rice grains, in g; N indicates the number of complete rice grains, in m q Indicates the thousand-grain weight of rice, in g; S b Indicates the pixel area of ​​broken grains, unit is pix.

8. A combine harvester, characterized in that: The invention comprises the combined harvester grain impurity rate and breakage rate monitoring system according to any one of claims 1 to 7.

9. A control method for a combine harvester grain impurity and breakage rate monitoring system according to any one of claims 1 to 7, characterized in that: The following steps are involved: The grain sampling device (17) collects grains from the vertical grain conveying drum (21) and transmits them to the image acquisition device (20). The image acquisition device (20) photographs the grain flow entering during the operation, obtains images of impurity information of the grains during the movement, and transmits them to the control device (13); The image acquisition module of the control device (13) controls the image acquisition device (20) to acquire grain images, and processes the grain images through the YOLOv8-Seg instance segmentation model of the image processing module, extracts pixel values ​​of various components, and uses them as inputs of the impurity content and breakage rate mathematical model to calculate and output the impurity content and breakage rate of the grains.

10. The control method of the combine harvester grain impurity rate and breakage rate monitoring system according to claim 8, characterized in that: The YOLOv8-Seg instance segmentation model is: YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model; the YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model processes the grain image specifically including the following steps: The backbone network CSPDarknet53 in the YOLOv8-Seg+C2f-Ghost+Shuffle Attention+Focal Loss instance segmentation model performs multi-scale feature extraction on the image collected by the image acquisition device (20) through the CBS module, the C2f-Ghost module and the SPPF module, and the depth-separable convolution of the Ghost Bottleneck in the C2f-Ghost module greatly reduces the computational complexity; Subsequently, the neck network (Neck) fuses deep and shallow features, integrating deep semantic features with shallow detail features. It also embeds a SA mechanism behind the three feature maps extracted by the backbone network to enhance the ability to distinguish small objects and complex backgrounds. The SA module uses channel grouping, dual-branch attention calculation, and feature reordering operations to enable the model to focus more on rice grain edges and fine impurity features. In the detection head stage, the Box branch predicts the bounding box coordinates (x, y, w, h) and confidence, the Cls branch predicts the target category, and the Mask branch outputs the mask coefficient for subsequent mask generation; the specific steps of subsequent mask generation are: on the multi-scale feature map output by Neck, 1×1 convolution is used to generate K prototype masks Proto, and in the detection head stage, the Mask branch predicts a set of K-dimensional mask coefficients for each detected target, which are used to weightedly combine the prototype mask Proto. For example, if a rice grain is detected, the Mask branch will output K weight values ​​of the rice grain to adjust the contribution of the prototype mask Proto, and perform matrix multiplication + Sigmoid activation on the mask coefficient and the prototype mask Proto to generate the instance mask of the target. Finally, combined with Focal The classification results optimized by the Loss loss function and the detection boxes regressed by the CIoU-DFL loss function are filtered by the NMS non-maximum suppression module to output the instance segmentation results with masks. The mask resolution is restored to the original image size through bilinear interpolation, realizing the accurate separation and statistics of rice grains and impurities.

Images