Method and system for detecting impurity rate and breakage rate of wheat

Abstract

The invention relates to a wheat impurity rate and damage rate detection method and system, and the method comprises the following steps: collecting wheat sample images in single-layer distribution, and carrying out unified preprocessing; a YOLOv8-FCD model is constructed; inputting the collected wheat sample image into the YOLOv8-FCD model, performing target identification on the wheat sample image, and outputting category labels and bounding box information of different targets in the image; and extracting the length and number of the diagonal lines of the bounding box of each target, and calculating the mass of the wheat ears, straws, wheat bran and crushed grains based on the extracted length and number of the diagonal lines of the bounding box of each target in a fitting manner, so as to calculate the impurity rate and the crushing rate of the wheat sample. According to the method, target identification is performed on the wheat sample image by constructing the YOLOv8-FCD model, the accuracy of target identification is improved, and meanwhile, accurate calculation of the impurity rate and the damage rate is realized through the pre-established regression model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligentization of agricultural equipment, and particularly relates to a wheat impurity content and breakage rate detection method and system. BACKGROUND

[0002] The breakage rate and the impurity content are important indexes for measuring the operation quality of the wheat combine harvester. The breakage rate refers to the percentage of broken grains in the total grains in the harvested wheat. The impurity content refers to the percentage of impurities such as wheat chaff and wheat straw in the total harvested wheat.

[0003] In the prior art, the visual detection method for the wheat breakage rate and the impurity content usually relies on artificial feature design, and has significant errors under light changes and grain overlapping scenes. The near-infrared spectrum technology is not sensitive to physical properties, and it is difficult to distinguish the spectral differences between the broken grains and the intact grains, which has significant defects in detection accuracy. Moreover, the existing processing algorithm has a long processing chain, and the single-frame processing takes more than 2 seconds, which cannot match the grain conveying speed of the harvester. The frame rate of the deep learning model on the embedded platform is only 5 FPS, which increases the synchronization delay and cannot meet the requirements of real-time monitoring. In addition, the existing mechanical sampling device is prone to jamming in a vibrating environment, and the grain outlet is prone to blockage, so the system adaptability is weak, and it is difficult to meet the continuous operation requirements in the whole harvesting season. Therefore, the prior art has obvious defects in detection accuracy, real-time performance and system adaptability, and cannot meet the requirements of high precision, high real-time performance and strong robustness for the detection of the breakage rate and the impurity content of the wheat combine harvester in the field operation. Therefore, there is an urgent need for a detection method that can accurately and quickly detect the breakage rate and the impurity content of the wheat harvested material in a complex field environment. SUMMARY

[0004] The purpose of the present application is to solve one of the above technical problems, and to provide a wheat impurity content and breakage rate detection method and system. The YOLOv8-FCD model is constructed to identify the target of the wheat sample image, and reliable data basis is provided for the calculation of the wheat breakage rate and the impurity content. At the same time, the regression model is established in advance to calculate the wheat impurity content and the breakage rate, so as to realize the accurate calculation of the impurity content and the breakage rate.

[0005] To achieve the above purpose, the technical solution adopted by the present application is: A wheat impurity content and breakage rate detection method, comprising: Collecting a single-layer distributed wheat sample image, and uniformly preprocessing the collected wheat sample image; Improving the YOLOv8 model to construct a YOLOv8-FCD model; input the collected wheat sample image into the YOLOv8-FCD model, perform target recognition on the wheat sample image, and output the class label and boundary box information of different targets in the image; the different targets include but are not limited to complete grains, broken grains, wheat spikes, straws and wheat bran; Based on the output result of the YOLOv8-FCD model, the diagonal line length and the number of each target are extracted, and the mass of the wheat spike, the straw, the wheat bran and the broken grain is calculated based on the extracted diagonal line length and the number of each target; Based on the calculated mass of the wheat spike, the straw, the wheat bran and the broken grain, and the total mass of the wheat sample, the impurity content and the broken rate of the wheat sample are calculated. In some embodiments of the present application, the method for improving the YOLOv8 model specifically includes the following steps: The SPPF spatial pyramid pooling module of the backbone network of the YOLOv8 model is replaced by the FPSConv feature pyramid shared convolution module; three convolution branches with different expansion rates are set in the FPSConv feature pyramid shared convolution module, which are respectively used to extract features with different receptive fields; A CSFCN context-sensitive fully convolutional network module is introduced in the neck network of the YOLOv8 model; the CSFCN context-sensitive fully convolutional network module includes a CFC context calibration module and an SFC spatial calibration module; The detection head of the YOLOv8 model is replaced by a DyHead dynamic detection head; The LAMP light-weight automatic mixed precision pruning strategy is used to compress the model.

[0006] In some embodiments of the present application, the method for recognizing the target of the wheat sample image specifically includes the following steps: The collected wheat sample image is input into the backbone network of the YOLOv8-FCD model; Two convolution modules are used in the backbone network to extract shallow features, and a plurality of C2f modules and convolution modules are stacked to extract deep features; The FPSConv feature pyramid shared convolution module is used in the key layer of the backbone network to extract features with different receptive fields, and the features with different receptive fields are spliced in the channel direction and then fused by using a convolution module; The wheat sample image is output after passing through the backbone network of the YOLOv8-FCD model.

[0007] In some embodiments of the present application, the method for recognizing the target of the wheat sample image further includes the following steps: The multi-layer features output by the backbone network are input into the neck network, deep layer features are up-sampled, shallow layer features are down-sampled, and the PAN-FPN fusion mode is used to fuse the features after sampling; The CFC context calibration module is used to enhance the semantic consistency between different scale features while performing feature fusion, and the SFC space calibration module is used to align the spatial positions of deep layer features and shallow layer features.

[0008] In some embodiments of the application, the method for target recognition of the wheat sample image further comprises the following steps: The fusion features output by the neck network are input into the DyHead dynamic detection head, the scale perception attention mechanism is used to assign different weights to feature maps of different scales, the spatial perception attention mechanism is used to correct the offset of the detection frame through deformable convolution, and the task perception attention mechanism is used to distinguish the classification branch and the regression branch, and finally the detection frame parameters and class probability of different targets are output.

[0009] In some embodiments of the application, the method further comprises the following steps: The final score is calculated based on the class probability and object confidence of different targets, and the redundant frames are removed after judging the overlapping frames by rotating IoU; In some embodiments of the application, the method for uniform preprocessing of the collected wheat sample image comprises the following steps: The collected wheat sample image is scaled to a predetermined size by equal proportion, and the blank is filled around the image to ensure that the aspect ratio of the image is unchanged; The wheat sample image is normalized and arranged, and the pixel value is scaled to a predetermined interval; The image channel of the wheat sample image is adjusted in a predetermined order.

[0010] In some embodiments of the application, the calculation formula of the impurity content of the wheat sample is: ; Wherein, is the total mass of the straw, is the total mass of the wheat bran, is the total mass of the measured wheat sample; The calculation formula of the total mass of the straw is: ; Wherein, is the actual mass of the straw, is the length of the diagonal line of the straw detection frame, is the number of straws, is the detection frame diagonal line length of the i th straw.​ The calculation formula of the total mass of the wheat bran is: ; Wherein, is the mass of a single wheat bran, is the number of wheat brans.

[0011] In some embodiments of the present application, the calculation formula of the breakage rate of the wheat sample is: ; Wherein, is the total mass of the broken wheat, is the total mass of the measured wheat sample; The calculation formula of the total mass of the broken wheat is: ; Wherein, is the mass of a single broken wheat, is the number of broken wheat.

[0012] Some embodiments of the present application further provide a wheat impurity rate and breakage rate detection system for implementing the above-mentioned wheat impurity rate and breakage rate detection method, comprising: A mechanical sampling module comprising a sample collection area, an image collection area and a conveyor belt; the sample collection area is connected to the shunt of the pneumatic conveying pipeline of the wheat harvester, and the conveyor belt is arranged below the sample collection area and the image collection area, for conveying the wheat sample collected in the sample collection area to the image collection area; An image collection module arranged at the top of the image collection area, comprising an industrial camera and a light supplement source, for continuously taking pictures of the sample dynamically entering the image collection area to obtain a wheat sample image; A data processing module for target recognition on the obtained wheat sample image, and fitting calculation of the mass of the wheat ear, straw, wheat bran and broken grain based on the target recognition result, and further calculation of the impurity rate and breakage rate of the wheat sample.

[0013] In some embodiments of the present application, the data processing module comprises: A YOLOv8-FCD model for target recognition on the obtained wheat sample image, outputting the class label and bounding box information of different targets in the image; different targets include but are not limited to complete grain, broken grain, wheat ear, straw and wheat bran; ​The regression model is used for extracting the diagonal line length and the number of the boundary box of each target based on the output result of the YOLOv8-FCD model, and fitting and calculating the mass of the wheat ear, the straw, the wheat bran and the broken grain based on the extracted diagonal line length and the number of the boundary box of each target, and then calculating the impurity content and the breakage rate of the wheat sample based on the calculated mass of the wheat ear, the straw, the wheat bran and the broken grain and the total mass of the wheat sample.

[0014] The present application has the advantages that: 1. The present application improves the accuracy of target recognition by constructing a YOLOv8-FCD model for target recognition of a wheat sample image, solves the problem of low recognition rate of micro-damaged grains and wheat with unclear features in the prior art, and provides a reliable data basis for calculation of the breakage rate and the impurity content of wheat. 2. The present application directly converts the diagonal line pixel length of the boundary box of the wheat ear, the straw and other impurities recognized by the YOLOv8-FCD model into its physical length through a pre-established regression model, and further calculates its mass through a length-mass regression model, and for the wheat bran and the broken grain, a quantity-average mass calculation model is used to calculate its mass, without the need to integrate any weighing unit in the detection device, so that accurate calculation of the impurity content and the breakage rate is realized, the system structure of the detection system is effectively simplified, and dynamic real-time detection of the breakage rate and the impurity content of wheat is realized. 3. The image acquisition device provided by the present application completely cancels the complex screening mechanism, the weighing sensor and the turnover part, adopts a combination of the shunt idea and the micro conveyor belt, and reduces the mechanical moving parts by more than 70%, thereby fundamentally eliminating the jamming phenomenon and effectively reducing the failure rate. 4. The imaging cavity of the image acquisition device provided by the present application adopts a closed design and is equipped with a dust cover and an LED light supplement, effectively resisting the interference of field dust and light changes, ensuring the stability of image acquisition quality, enabling the detection system provided by the present application to stably operate in the harsh vibration environment of a combine harvester for a long time, and having high reliability.

[0015] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and achieved by the structures indicated in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, specific embodiments of the present application will be described in detail below with reference to the drawings. Those skilled in the art can also obtain other drawings without creative labor on the premise that the drawings are not disclosed.

[0017] Figure 1 A flow chart of a wheat impurity and broken rate detection method; Figure 2 A structural diagram of a YOLOv8-FCD model provided by an embodiment of the application is shown in the figure; Figure 3 A structural diagram of a DyHead dynamic detection head provided by an embodiment of the application is shown in the figure; Figure 4 A structural diagram of three types of attention mechanisms provided by an embodiment of the application is shown in the figure; Figure 5 A side view of an image acquisition device provided by an embodiment of the application is shown in the figure; Figure 6 A front view of an image acquisition device provided by an embodiment of the application is shown in the figure; Figure 7 A working flow chart of a wheat impurity and broken rate detection system is shown in the figure; Among them, the reference signs are: 1, transmission base; 2, material port cover; 3, material passing adjusting plate; 4, dust cover; 5, light supplement source; 6, industrial camera; 7, bolt and nut; 8, motor; 9, conveying belt. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is described and explained below in combination with the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and not to limit the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0019] It should be noted that the terms used here are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0020] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0021] The technical solutions of the present application will be described in detail below in combination with specific embodiments and the drawings of the specification.

[0022] As shown in the accompanying drawings, Figure 1 -attached Figure 4 In one illustrative embodiment of the detection method for the foreign matter content and broken rate of wheat, the detection method comprises the following steps.

[0023] S1: Use an image acquisition device to acquire a machine-harvested wheat sample blown by a fan. After the machine-harvested wheat enters the image acquisition device, it falls onto a conveyor belt and passes through a single-layered scraper, causing the wheat sample to be distributed in a single layer. An industrial camera is used to take a photo of the machine-harvested wheat sample and impurities on the conveyor belt every second to obtain an image of the wheat sample distributed in a single layer.

[0024] The acquired wheat sample image is uniformly preprocessed.

[0025] In some embodiments of the present application, since the sizes of the acquired original images are different, the method for uniformly preprocessing the acquired wheat sample image comprises the following steps.

[0026] First, scaling and padding are performed to scale the acquired wheat sample image to a predetermined size in proportion and fill the blank around the image to ensure that the aspect ratio of the image remains unchanged. In this embodiment, the predetermined size is set to 640x640 pixels.

[0027] The formula for scaling the wheat sample image in proportion is: ,; ; ; wherein, is the input size, , are the height and width of the original image, respectively, is the scaling factor, , are the padding width and height, respectively.

[0028] The wheat sample image is normalized and arranged, and the pixel value is scaled to the interval of 0-1.

[0029] The image channels of the wheat sample image are adjusted in a predetermined order. The predetermined order of the image channels is determined based on the needs of the YOLOv8-FCD model.

[0030] S2: Improve the YOLOv8 model to build a YOLOv8-FCD model, and the structure diagram of the YOLOv8-FCD model is shown in the accompanying drawings. Figure 2

[0031] ​In some embodiments of the present application, the method for improving the YOLOv8 model in step S2 specifically includes the following steps: The SPPF spatial pyramid pooling module of the backbone network of the YOLOv8 model is replaced by the FPSConv feature pyramid shared convolution module to better extract multi-scale features of the detected target. Three convolution branches with different expansion rates are set in the FPSConv feature pyramid shared convolution module, which are respectively used to extract features with different receptive fields.

[0032] A CSFCN context-sensitive fully convolutional network module is introduced in the neck network of the YOLOv8 model to enhance the model's ability to capture local details of wheat. The CSFCN context-sensitive fully convolutional network module includes a CFC context calibration module and an SFC spatial calibration module. The detection head of the YOLOv8 model is replaced by a DyHead dynamic detection head to further improve the performance of small target detection. In order to support subsequent embedded deployment, a LAMP lightweight automatic mixed precision pruning strategy is used to compress the model. In this embodiment, the size of the YOLOV8-FCD model is about 4.2MB, and the single-frame detection time is about 4.2ms.

[0033] S3: input the collected wheat sample image into the YOLOv8-FCD model, perform target recognition on the wheat sample image, and output the class label and bounding box information of different targets in the image. Different targets include but are not limited to complete kernels, broken kernels, wheat spikes, straw and wheat bran.

[0034] In some embodiments of the present application, the method for performing target recognition on the wheat sample image specifically includes the following steps.

[0035] The collected wheat sample image is input into the backbone network (Backbone) of the YOLOv8-FCD model for backbone feature extraction.

[0036] In the backbone network, two convolution (Conv) modules are first used to extract shallow features, and multiple C2f modules and convolution (Conv) modules are stacked to gradually extract deep features. Among them, the shallow features include edge, texture and other features.

[0037] In the key layer of the backbone network, the three convolution branches of the FPSConv feature pyramid shared convolution module are used to extract features with different receptive fields, and the features with different receptive fields are spliced in the channel direction and then fused using a convolution module. Among them, the three convolution branches are 1, 3 and 5.

[0038] The formula for splicing and fusion is: ; wherein, , , respectively represent three branch outputs, is a shared convolution kernel.

[0039] Using the FPSConv, both the wide-range background information and the fine-grained kernel crack and embryo defect features can be covered.

[0040] After the wheat sample image passes through the backbone network of the YOLOv8-FCD model, multi-layer features are output.

[0041] In some embodiments of the present application, the method for target recognition of the wheat sample image in step S2 further includes the following steps.

[0042] The multi-layer features output by the backbone network are input into the neck network, and the deep layer features are up-sampled and spliced with the shallow layer features. The shallow layer features are down-sampled and fused with the deep layer features. The PAN-FPN fusion method is used to fuse the features after sampling, which can utilize large-scale context information and maintain the details of small targets.

[0043] The CSFCN context-sensitive fully convolutional network module is used to effectively suppress the boundary blur caused by vibration or overlap while performing feature fusion.

[0044] Specifically, the CFC context calibration module is used to enhance the semantic consistency between different scale features, and the SFC spatial calibration module is used to align the spatial positions of the deep layer features and the shallow layer features.

[0045] The specific formula of this step is: ; wherein, is the up-sampling result of the deep layer features; is the shallow layer features; ; is sigmoid.

[0046] In some embodiments of the present application, the method for target recognition of the wheat sample image in step S2 further includes the following steps.

[0047] The fusion features output by the neck network are input into a DyHead dynamic detection head for target positioning and classification. In this embodiment, the DyHead dynamic detection head comprises three types of attention mechanisms, namely, a scale-aware attention mechanism, a spatial-aware attention mechanism, and a task-aware attention mechanism. The scale-aware attention is used to assign different weights to feature maps of different scales. The spatial-aware attention is used to correct the offset of the detection frame through deformable convolution. The task-aware attention is used to distinguish the classification branch and the regression branch. The network structure of the DyHead dynamic detection head is shown in FIG. 2. The network structures of the scale-aware attention mechanism, the spatial-aware attention mechanism, and the task-aware attention mechanism are shown in FIGS. 3, 4, and 5, respectively. Figure 3 Figure 4

[0048] In the DyHead dynamic detection head, the scale-aware attention mechanism is used to assign different weights to feature maps of different scales, the spatial-aware attention mechanism is used to correct the offset of the detection frame through deformable convolution, and the task-aware attention mechanism is used to distinguish the classification branch and the regression branch, so as to finally output the detection frame parameters and the class probability of different targets. wherein, represents the center point coordinates; represents the width and height of the detection frame; represents the rotation angle; represents different classes.

[0049] wherein, the output of the feature map obtained through the three attention modules is: wherein, represents the scale-aware attention mechanism, represents the spatial-aware attention mechanism, is the channel-aware attention mechanism.

[0050] In this embodiment, the scale-aware attention (Scale-aware Attention) is composed of global pooling, 1x1 convolution, ReLU activation function, and hard-sigmoid activation function. The Adaptive-AvgPool2d performs global pooling on HxW to obtain the maximum value of the feature map, and the convolution integrates the channels together to form L numbers.

[0051] The specific formula is as follows: wherein, is a linear function using a 1x1 convolution layer, is a hard-sigmoid activation function, and its formula is: ​​​​.

[0052] In this embodiment, the spatial-aware attention (Spatial-aware Attention) uses deformable convolution v2. The formula is as follows: ; wherein K is the number of sparse sampling positions, is the position moved by learning the offset, learned by deformable convolution v2, focusing on the problem area, indicates the position importance of learning.

[0053] In this embodiment, the task-aware attention (Task-aware Attention) uses two layers of fully connected neural network to model the channel, and controls the super parameter whether to learn the threshold of activation to automatically select to open or close the channel to adapt to different task requirements.

[0054] The formula is as follows: ; wherein, is the threshold of learning activation Channel, is the feature divided on the Cth channel.

[0055] In some embodiments of the present application, step S2 further comprises the following steps.

[0056] Based on the category probability and object confidence of different targets, the final score is calculated, and the optimal prediction result is determined by judging the overlapping frame through rotation IoU, and removing the redundant frame.

[0057] S4: Based on the output result of the YOLOv8-FCD model, the diagonal line length and number of each target boundary box are extracted, and the mass of the ear, straw, wheat bran, broken grain and complete grain is calculated based on the diagonal line length and number of each target boundary box extracted.

[0058] Wherein, due to the similar size and weight of complete grains, the average mass of a single complete wheat is measured by experiment to calculate the overall intact wheat mass. By establishing the quantity-mass relationship, the rapid estimation of intact wheat mass is realized.

[0059] The mass formula of complete grains is: ; wherein: is the total mass of intact wheat, unit gram, g; For the quality of a single intact wheat, unit gram, g; For the number of intact wheat, unit pcs.

[0060] The fitting method of the weight of the straw is to fit the relationship between the length and the weight of the straw, so as to obtain the weight of all the straws in the detection picture. The calculation formula of the total weight of the straw is:

[0061] Among them, The actual weight of the straw, unit gram, g; The length of the diagonal line of the detection anchor frame of the straw, unit centimeter, cm; The number of straws, unit pcs; The diagonal line length of the detection frame of the i-th straw, unit centimeter, cm; The total weight of all straws, that is, the total weight of all detected straws in the image, unit gram, g.

[0062] The average weight of a single wheat bran is determined by experiment to establish the quantity-quality relationship, so as to realize the rapid estimation of the weight of the wheat bran. The calculation formula of the total weight of the wheat bran is: ; Among them, The total weight of the wheat bran, unit gram, g; The weight of a single wheat bran, unit gram, g; The number of wheat bran, unit pcs.

[0063] S5: Based on the calculated weight of the spike, straw, wheat bran, broken grain and complete, and the total weight of the wheat sample, the impurity content and the broken rate of the wheat sample are calculated.

[0064] In some embodiments of the application, the impurity content of the wheat sample The calculation formula is: ; Among them, The total weight of the straw, The total weight of the wheat bran, The total weight of the measured wheat sample; In the calculation process of the broken rate of wheat, the weight of broken wheat determines the final broken rate, but its size is smaller or equivalent to that of intact wheat. According to the diagonal fitting weight method, it is difficult to efficiently and accurately determine the weight, therefore, in some embodiments of the application, the average weight of a single broken wheat is determined by experiment to establish the quantity-quality relationship, so as to realize the rapid estimation of the weight of the broken wheat.

[0065] Specifically, the broken rate of the wheat sample The calculation formula is: ; in, The total mass of the crushed wheat. The total mass of the wheat sample being tested; The formula for calculating the total mass of broken wheat is as follows: ; in, For the mass of a single broken wheat grain, This refers to the quantity of broken wheat.

[0066] In some embodiments of the present invention, the following steps are further included.

[0067] S6: After the data processing and analysis are completed, the system outputs a visual detection image, impurity rate, and breakage rate value. The analysis results are displayed on the display screen in the cab, allowing the harvester to adjust the actual harvesting parameters. When the ratio exceeds the threshold, an alarm prompt is displayed.

[0068] After testing, the harvested wheat falls into the grain collection bin through the terminal bulk outlet.

[0069] Some embodiments of the present invention further provide a wheat impurity and damage rate detection system for implementing the above-described wheat impurity and damage rate detection method. The detection system includes an image acquisition device and a data processing terminal. The workflow of the detection system is shown in the attached figure. Figure 7 As shown.

[0070] Among them, the image acquisition device is as follows Figure 5 -Appendix Figure 6 As shown, it includes a mechanical sampling module and an image acquisition module.

[0071] The mechanical sampling module includes a sample collection area, an image acquisition area, and a conveyor belt.

[0072] The sample collection area is connected to the branch port of the pneumatic conveying pipe of the wheat harvester via the material mask 2. The conveyor belt is set below the sample collection area and the image acquisition area and is installed on the transmission base 1 to transport the wheat samples collected in the sample collection area to the image acquisition area. The image acquisition area has a closed imaging cavity.

[0073] For details, see attached. Figure 5As shown, during the process of harvesting wheat by the wheat harvester, the wheat enters the sample collection area along direction A after entering the material port cover 2 through the shunt of the pneumatic conveying pipe of the harvester, and part of the grains and impurities enter the sample collection port through the shunt effect, and then enter the image collection area along direction B through the miniature conveying belt 9 powered by the motor 8. The amount of wheat entering the image collection unit along direction B can be adjusted by adjusting the screw nut 7 to adjust the distance between the material adjusting plate 3 and the conveying belt, so as to adjust the amount of harvested materials entering the image collection area. When too much harvested material enters the mechanical sampling module through the shunt, the excess harvested material can be discharged from the collection device through the bulk grain outlet in direction C, effectively preventing the accumulation of harvested materials in the collection port.

[0074] The image collection module is shown in the attached Figure 5 As shown, the image collection area is provided at the top of the image collection area, including an industrial camera 6, a light supplement source 5 and a transparent acrylic plate dust cover, which is used for continuously photographing the samples dynamically entering the image collection area to obtain the wheat sample image.

[0075] Specifically, in the embodiment, the industrial camera 6 is a Hikvision MV-CS060-10GC industrial camera, equipped with an MVL-HF0828-6MPE lens. The focal length of the lens is 8mm, the aperture is f2.8, the shooting distance between the lens and the wheat is fixed at 205mm, and the output image resolution is 2048x2048 pixels.

[0076] The data processing terminal includes a data processing module, which includes a YOLOv8-FCD model running on a Jetson Nano calculator, which is used for target recognition on the obtained wheat sample image, and fitting calculation of the mass of wheat ear, straw, bran and broken grain based on the target recognition result, and then calculating the impurity content and broken rate of the wheat sample.

[0077] In some embodiments of the application, the data processing module includes a YOLOv8-FCD model and a regression model.

[0078] The YOLOv8-FCD model is used for target recognition on the obtained wheat sample image, and outputs the class label and bounding box information of different targets in the image; different targets include but are not limited to whole grains, broken grains, wheat ears, straws and bran; The regression model is used to extract the diagonal line length and number of each target based on the output result of the YOLOv8-FCD model, and to fit and calculate the mass of wheat ear, straw, bran and broken grain based on the extracted diagonal line length and number of each target, and then to calculate the impurity content and broken rate of the wheat sample based on the calculated mass of wheat ear, straw, bran and broken grain and the total mass of the wheat sample.

[0079] In the above schematic embodiment, unlike the complex and failure-prone mechanical screening mechanism (such as a waste box, a combing mechanism, a weighing sensor, etc.) in the prior art, the detection system provided by the application innovatively designs a dynamic sampling mechanism, which directly intercepts part of the dynamic grain flow as a sample by controlling the speed of the gap adjustment plate and the micro conveyor belt, and the sample is sent back to the grain bin after being sent into the imaging cavity through the flow guide gap and the micro conveyor belt. This structure completely eliminates the complex steps of traditional screening, weighing and the like, greatly simplifies the mechanical device, fundamentally solves the technical problems of easy jamming and easy blocking of the existing system in a vibrating environment, and significantly improves the reliability and adaptability of the system.

[0080] In view of the problems of large calculation amount of the existing deep learning model and difficulty in real-time deployment on an agricultural machine embedded platform, the detection system and the detection method provided by the application make four targeted improvements to the YOLOv8n model to form a special YOLOv8-FCD model.

[0081] The above four targeted improvements include: The SPPF module in the backbone network is replaced by FPSConv (feature pyramid shared convolution), which utilizes multi-dilation rate convolution to work cooperatively and enhances the feature extraction capability of the model for multi-scale targets (from small broken kernels to large straws).

[0082] The CSFCN (context-sensitive fully convolutional network) module is introduced into the Neck network, which significantly improves the local detail capture capability and recognition accuracy of the "lost germ wheat" and small broken kernels with inconspicuous features through its context feature calibration (CFC) and spatial feature calibration (SFC) mechanisms.

[0083] The original detection head is replaced by DyHead (dynamic detection head), which dynamically fuses features through scale, space and task triple attention mechanisms, and enhances the detection robustness for dense, overlapping and multi-scale targets.

[0084] The improved model is subjected to lightweight processing by using the LAMP (layer adaptive sparse pruning) strategy, which compresses the model volume to 4.2MB without losing accuracy (mAP@0.5 remains 97.3%) and improves the inference speed to 4.2ms / frame, so that it can stably run on low-power embedded devices such as Jetson Nano, meeting the real-time detection requirements of the harvester.

[0085] In order to overcome the problems of high hardware cost, complex synchronization and difficult image registration in the existing double-camera scheme, the detection system provided by the present application adopts an imaging scheme of single camera combined with a reflecting prism group. Only one industrial camera is used, and through reasonable arrangement of the optical prism in the imaging cavity, the front and back images of the grain can be synchronously collected without the need of complex mechanical or electronic synchronization devices. This method greatly reduces the hardware cost and structural complexity of the system while ensuring that the key feature information is not lost.

[0086] At the same time, unlike the traditional direct measurement method relying on a weighing sensor, the present application provides a non-contact mass estimation method based on the size of the detection frame. Through a pre-established precise regression model, the diagonal pixel length of the boundary frame of the impurities (wheat ear and straw) identified by the model is directly converted into the physical length, and then the mass is calculated through a length-mass regression model. For wheat bran and broken grains, a quantity-average mass calculation model is used. This method does not need to integrate any weighing unit in the detection device, and can realize accurate calculation of the impurity content and the breakage rate, which is a key technical breakthrough for realizing simplified system structure and dynamic real-time detection.

[0087] Finally, it should be noted that: the embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to.

[0088] The above embodiments are only used to illustrate the technical solutions of the present application rather than limit them; although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the specific embodiments of the present application can be modified or some technical features can be replaced by equivalent ones; without departing from the spirit of the technical solutions of the present application, they should be covered in the technical solution range of the present application.

Claims

1. A method for detecting the foreign matter content and the broken rate of wheat, characterized by, The application relates to a wheat sample image recognition method and device. The method comprises the following steps: Collecting a wheat sample image in a single-layer distribution, and uniformly preprocessing the collected wheat sample image; Improving a YOLOv8 model to construct a YOLOv8-FCD model; Inputting the collected wheat sample image into the YOLOv8-FCD model, identifying the target of the wheat sample image, and outputting the class label and the boundary box information of different targets in the image; the different targets include but are not limited to complete grains, broken grains, wheat spikes, straws and wheat bran; Based on the output result of the YOLOv8-FCD model, the diagonal line length and the number of each target boundary box are extracted, and the mass of the wheat spike, the straw, the wheat bran and the broken grain is calculated based on the extracted diagonal line length and the number of each target boundary box; 2. The wheat foreign matter and broken rate detection method according to claim 1, characterized by, Based on the calculated mass of the wheat spike, the straw, the wheat bran and the broken grain and the total mass of the wheat sample, the impurity content and the broken rate of the wheat sample are calculated. The method for improving the YOLOv8 model comprises the following steps: Replacing the SPPF spatial pyramid pooling module of the backbone network of the YOLOv8 model with an FPSConv feature pyramid shared convolution module; three convolution branches with different expansion rates are arranged in the FPSConv feature pyramid shared convolution module, and are used for extracting features of different receptive fields; Introducing a CSFCN context-sensitive fully convolutional network module into the neck network of the YOLOv8 model; the CSFCN context-sensitive fully convolutional network module comprises a CFC context calibration module and a SFC spatial calibration module; Replacing the detection head of the YOLOv8 model with a DyHead dynamic detection head; 3. The wheat foreign matter and broken rate detection method according to claim 2, characterized by, Using a LAMP lightweight automatic mixed precision pruning strategy to compress the model. The method for identifying the target of the wheat sample image comprises the following steps: Inputting the collected wheat sample image into the backbone network of the YOLOv8-FCD model; Using two convolution modules to extract shallow layer features in the backbone network, and using a plurality of C2f modules and convolution modules to stack and extract deep layer features; Using the FPSConv feature pyramid shared convolution module to extract features of different receptive fields at the key layer of the backbone network, and using a convolution module to fuse the features of different receptive fields after the features of different receptive fields are spliced in the channel direction; 4. The wheat foreign matter and broken rate detection method according to claim 3, characterized by, The wheat sample image is outputted after passing through the backbone network of the YOLOv8-FCD model. The method for identifying the target of the wheat sample image further comprises the following steps: Inputting the multi-layer features outputted by the backbone network into the neck network, up-sampling the deep layer features, down-sampling the shallow layer features, and using a PAN-FPN fusion mode to fuse the features after sampling; 5. The wheat foreign matter and broken rate detection method according to claim 4, characterized by, Using the CFC context calibration module to enhance the semantic consistency between different scale features while fusing the features, and using the SFC spatial calibration module to align the spatial positions of the deep layer features and the shallow layer features. The method for identifying the target of the wheat sample image further comprises the following steps: The fusion features output by the neck network are input into a DyHead dynamic detection head, in which a scale-aware attention mechanism is used to assign different weights to feature maps of different scales, a spatial-aware attention mechanism is used to correct the offset of the detection frame through deformable convolution, and a task-aware attention mechanism is used to distinguish the classification branch and the regression branch, and finally the detection frame parameters and class probability of different targets are output.

6. The wheat foreign matter and broken rate detection method according to claim 5, characterized by, The method for identifying the target of the wheat sample image further includes the following steps: The final score is calculated based on the class probability and object confidence of different targets, and redundant frames are removed after judging the overlapping frames by rotating IoU.

7. The method of claim 1, wherein the method is a method of detecting the foreign matter content and the broken rate of wheat. The method for uniformly preprocessing the collected wheat sample image specifically includes the following steps: The collected wheat sample image is scaled to a predetermined size in proportion, and blank is filled around the image to ensure that the aspect ratio of the image is unchanged; The wheat sample image is normalized and arranged, and the pixel value is scaled to a predetermined interval; The image channel of the wheat sample image is adjusted in a predetermined order.

8. The method according to any one of claims 1 to 7, wherein, The formula for calculating the foreign matter content of the wheat sample is: The formula for calculating the foreign matter content of the wheat sample is: ； wherein, is the total mass of the straw, is the total mass of the wheat bran, is the total mass of the wheat sample under test; The calculation formula of the total mass of the straw is: ； wherein, is the actual mass of the straw, is the length of the diagonal of the detection frame of the straw, is the number of straws, is the length of the diagonal of the detection frame of the i-th straw; The calculation formula of the total mass of the bran is: ； wherein, is the mass of the single wheat bran, is the quantity of wheat bran.

9. The method according to any one of claims 1 to 7, wherein the method is a method for detecting the foreign matter content and the broken rate of wheat, characterized by, The breakage rate of the wheat sample The formula for calculating the breakage rate is: ； wherein, is the total mass of the wheat to be broken, is the total mass of the wheat sample being measured; The calculation formula of the total mass of the broken wheat is: ； wherein, is the quality of the single broken wheat, is the quantity of the broken wheat.

10. A wheat foreign matter and broken end detection system for implementing the wheat foreign matter and broken end detection method according to any one of claims 1 to 9, characterized by, It comprises: A mechanical sampling module comprising a sample collection area, an image acquisition area and a conveyor belt; The sample collection area is connected with the shunt port of the pneumatic conveying pipeline of the wheat harvester, and the conveyor belt is arranged below the sample collection area and the image acquisition area, used to convey the wheat samples collected in the sample collection area to the image acquisition area; An image acquisition module is arranged at the top of the image acquisition area, comprising an industrial camera and a light source, used to continuously take pictures of the samples entering the image acquisition area dynamically to obtain wheat sample images; A data processing module running a YOLOv8-FCD model is used to identify the target of the obtained wheat sample image, and based on the target identification result, the mass of the wheat ear, straw, bran and broken grain is fitted and calculated, and then the impurity content and the breakage rate of the wheat sample are calculated.

Images