A method and system for detecting debris and crushing rate of a hilly and mountainous combine harvester and a harvester

Abstract

The present invention provides a method and system for detecting the impurity content and breakage rate of a combined harvester for hilly and mountainous areas, and a harvester, including the following steps: Step S7, judging the image quality: establishing an image quality detection standard, comparing the area #imgabs0# of the similarity domain obtained in combination with Step S6 with the area of the detected image, classifying the image into a standard-compliant image and a piled-up image. When the judged image is a standard-compliant image, proceed to Step S8. When the judged image is a piled-up image, fuse the image stream within the sampling time t with Mask still and substitute it into Steps S1 - S7, then make an image quality judgment again until the judged image is a standard-compliant image, and then proceed to Step S8. The present invention solves, from the perspective of visual processing, the situation of falsely high impurity content caused by the occurrence of accumulation and sticking of impurities in the grain conveyor auger of the harvesting machinery due to the undulating terrain of hilly and mountainous areas.

Description

Technical Field

[0001] The present invention belongs to the field of mechanical intelligence, and in particular relates to a method and system for detecting the debris and crushing rate of a combine harvester in hilly and mountainous areas, and the harvester. Background Art

[0002] The planting areas in my country's hilly and mountainous areas are mainly trapezoidal planting slopes. Agricultural machinery suitable for harvesting in hilly and mountainous areas is in the development stage and has a low level of intelligence. Due to the rugged ground and large slope difference in hilly and mountainous areas, grains are easily accumulated at the grain feeding auger mouth during the operation of hilly and mountainous combine harvesters, and even machine failures such as blockage of the grain feeding mouth may occur.

[0003] The existing online target detection technology for grains uses Mask R-CNN as the target detection network to perform online detection of grain bin grains. Although performing two quantization operations on the target object can improve the detection accuracy, the multiple traversal and iterative operations for the huge amount of data make the real-time performance slightly insufficient. In addition, there is no corresponding solution to the problem of grain accumulation caused by the blockage caused by the unique terrain and landforms in hilly and mountainous areas. The impurities and broken data generated in the grain accumulation area will seriously affect the impurity rate and the calculation accuracy of the breakage rate. Summary of the invention

[0004] In view of the above technical problems, the present invention provides a method for detecting the debris and breakage rate of a combine harvester in hilly and mountainous areas. While completing the real-time detection of debris and breakage, the present invention solves the problem of falsely high debris content due to the accumulation of debris and stickiness in the grain conveying auger of the harvester caused by the undulating topography of hilly and mountainous areas from the perspective of visual processing, thereby improving the calculation accuracy of the debris content and breakage rate.

[0005] The present invention also provides a system for realizing the method for detecting the debris and crushing rate of the hilly and mountainous combine harvester.

[0006] The present invention also provides a harvester comprising the impurity and breakage rate detection system for the hilly and mountainous combine harvester.

[0007] The technical solution of the present invention is:

[0008] A method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas comprises the following steps:

[0009] Step S1, image acquisition: The image acquisition device collects the photo stream of rice in the grain conveying auger and transmits it to the control unit. The control unit pre-sets the sampling time of two adjacent photos in the photo stream as t, and uses the sampling time t as the subscript to mark the photo stream as k represents the number of photos in the photo stream;

[0010] Step S2, image preprocessing: perform ROI processing on the photo collected in step S1, perform color inversion on the photo, and perform proportional calculation on the photo pixels;

[0011] Step S3, obtain the fused grayscale image: intercept the photo in the photo stream preprocessed in step S2 Get the grayscale image of the photo Will Fusion, get the fused grayscale image

[0012] Step S4, obtaining the grayscale image basic threshold: arranging sampling quadrat, obtaining The basic threshold th bass ;

[0013] Step S5, distinguishing the foreground and background of the grayscale image: The grayscale image fused in step S3 The basic threshold th in step S4 bass Perform comparative processing to distinguish the foreground P of the grayscale image pro With background P back part;

[0014] Step S6: Create a static object capture mask still And: by the foreground P in step S5 pro Get the similarity evaluation matrix Mat eva , via Mat eva Get the improved similarity evaluation matrix Mat * eva , via Mat * eva Get the similarity domain area Black Mask still0 and white mask still1 ;

[0015] Step S7, judging image quality: establishing an image quality detection standard, combining the similarity domain area obtained in step S6 The image is compared with the detected image area, and the image is divided into a standard image and an accumulation image. When the image is judged to be a standard image, step S8 is entered. When the image is judged to be an accumulation image, the image stream within the sampling time t is compared with the Mask still After fusion, the image is entered into step S1 to step S7, and the image quality is judged again until the image is judged to meet the standard image, and then the image is entered into step S8;

[0016] Step S8, detection of impurity and broken rice grain rate: The images meeting the standard in step S7 are subjected to deeplab segmentation and Yolo target detection to obtain the impurity mass g(za) and the broken rice grain mass G(A a) and rice grain mass G(A), and the impurity rate and breakage rate of the corresponding rice grains were obtained by calculation.

[0017] In the above scheme, the specific steps of obtaining the fused grayscale image in step S3 are as follows:

[0018] Step S3.1: Change the sensitivity of the three colors R, G, and B in the image according to the brightness reflected by the different grain colors in the image, specifically:

[0019]

[0020] in And n>m>l, n, m, l are the influencing factors of RGB channel proportion respectively;

[0021] Get the grayscale image:

[0022] Step S3.2: convert the image grayscale image obtained in step S3.1 into: Fusion, get the fused grayscale image The specific steps are:

[0023] Will The overall gray value is recently integerized and averaged to obtain

[0024] Will Grayscale values ​​are linearly fused to obtain the fused grayscale image

[0025] In the above scheme, the specific steps of step S4 to obtain the grayscale image basic threshold are:

[0026] Step S4.1, arrange sampling plots: According to the N*N grid uniform differentiation, N is the side length of the square inscribed in the circular field of view, M is the number of sample squares, the side length of the sample square is equal to N / M, and it is rounded down, and the entire grayscale image is reset to M*M basic sample squares; take the sample square numbered y, where y=3x+1(x∈[0,(M*M-1) / 3],x∈Z), and obtain (M*M-1) / 3 sampling sample squares with uniform distribution: q1...q k ……q 21 ;

[0027] Step S4.2: Obtain The basic threshold th bass:Slide (M*M-1) / 3 sampling plots horizontally according to one frame and one grid to obtain 3 groups of plot groups. In each group, the grayscale values ​​in (M*M-1) / 3 sampling plots are slid according to Gaussian convolution to obtain the reference basic threshold. The (M*M-1) / 3 reference basic thresholds are averaged to obtain th1, th2, and th3. The three groups of data th1, th2, and th3 are averaged to obtain the basic threshold th bass .

[0028] In the above scheme, the specific steps of S5 for distinguishing the foreground and background of the grayscale image are as follows:

[0029] The fused grayscale image With the basic threshold th bass The comparison is as follows:

[0030]

[0031] Among them, P1 is the grayscale image obtained after comparison, P back For the background part, P pro Foreground part.

[0032] In the above scheme, the S6 prepares a stationary object capture mask and;

[0033] S6.1, through P pro and Proportional operation to obtain the similarity evaluation matrix Mat eva :Will The mask Mask1 of the same size filled with 1 is linearly added and fused to process the pixels as non-zero. pro Divided by the pixel after non-zero processing Get the similarity evaluation matrix Mat composed of similarity coefficients eva ;

[0034] S6.2, similarity evaluation matrix Mat eva Perform bilateral filtering with boundaries A and B, where A and B are the thresholds of bilateral filtering, A is 1.8-2, B is 2-2.2, and the similarity coefficient within the filtering range is assigned to 0, and the range is recorded as similar domain area0; the similarity coefficient that does not belong to the filtering range is assigned to 1, and the range is recorded as non-similar domain area1; an improved similarity evaluation matrix Mat is obtained, which marks the similar domain area0 as 0 and the non-similar domain area1 as 1 * eva ;

[0035] S6.3. Traverse and improve the similarity evaluation matrix Mat * eva , get the similarity domain area

[0036] S6.4. Traverse and improve the similarity evaluation matrix Mat * eva , save the relative position coordinates of the 0 and 1 values ​​in the matrix relative to the image; assign the RGB three channels of the mask corresponding to the 0 value relative coordinates to 0, and assign the RGB three channels of the mask corresponding to the 1 value relative coordinates to 255; process the similarity domain area0 as a black mask to obtain a black mask Mask still0 , treat the non-similar domain area1 as a white mask to obtain a white mask Mask still1 .

[0037] In the above scheme, the image in the step S7 judging the image quality also includes a blocked image; when the image is judged to be a blocked image, the control unit issues an alarm.

[0038] In the above solution, the specific steps of S7 for judging the image quality are:

[0039] Establish image quality detection standards and classify images into three categories: images that meet the standards, blocked images, and accumulated images. The specific rules are as follows:

[0040]

[0041] Among them, S1 is the judgment area, and D1 is the flag bit of whether it is judged as a standard image; if the flag bit D1 is 0, it means that the image meets the sample detection standard and is brought into the mixed and broken detection calculation; if the flag bit D1 is 1, it means that the image does not meet the sample detection standard, and enters the steps S1-S7 again to make another image quality judgment.

[0042] In the above scheme, the S8 impurity and breakage rate detection includes the following steps:

[0043] When the flag D1=0, the image is subjected to deeplab segmentation and Yolo target detection respectively, where:

[0044] The image is processed by deeplab semantic segmentation to obtain the pixel area of ​​the residual and broken rice grain samples. Then, according to the mass-pixel correspondence of the residual, the residual mass g(za) is obtained; according to the mass-pixel correspondence of the broken rice grains, the broken rice grain G(A) is obtained. a )’s quality;

[0045] The rice grain information is input into the Yolo neural network for target detection to obtain the number information of the rice grains, and the rice grain mass G(A) is obtained through the number information of the rice grains;

[0046] Enter the calculation formula:

[0047]

[0048]

[0049] Obtain the impurity rate and breakage rate of the corresponding rice grains.

[0050] In the above scheme, the S8 impurity and breakage rate detection also includes the following steps:

[0051] When the flag bit D1 = 1, the following judgment is made for images that do not meet the detection requirements:

[0052]

[0053] Where S2 is the blockage judgment area, and D2 is the flag bit for judging whether the image is blocked or piled up; if the flag bit D2 is 1, the blockage alarm state is entered and an alarm image is generated; if the flag bit D2 is 0, it is judged to be piled up;

[0054] When the flag D2 = 0, the image is judged as a pile-up phenomenon, and the image stream within the sampling time t is compared with the Mask still After fusion, the image is input into step S1 to step S7, and then the image quality is judged;

[0055] When the flag D2=1, the machine enters the jam alarm state and stops operating.

[0056] A control system according to the method for detecting the debris and brokenness rate of a hilly combine harvester, comprising an image acquisition device and a control unit; the control unit comprises a photo stream marking module, an image preprocessing module, a fused grayscale image acquisition module, a grayscale image basic threshold acquisition module, a grayscale image foreground and background distinction module, a stationary object capture mask and acquisition module, an image quality detection module and a debris and brokenness rate detection module;

[0057] The image acquisition device is used to collect the photo stream of rice in the grain conveying auger and transmit it to the control unit. The photo stream marking module is used to pre-set the sampling time of two adjacent photos in the photo stream as t, and

[0058] The space between is a subscript, marking the photo stream as

[0059] The image preprocessing module is used to perform ROI processing on the collected photos, invert the colors of the photos, and perform proportional calculation on the pixels of the photos;

[0060] The fused grayscale image acquisition module is used to capture photos in the photo stream Get the grayscale image of the photo Will Fusion, get the fused grayscale image

[0061] The grayscale image basic threshold acquisition module is used to arrange sampling quadrangles to obtain The basic threshold th bass ;

[0062] The grayscale image foreground and background distinguishing module is used to distinguish the foreground and background of the grayscale image: the fused grayscale image With the basic threshold th bass Perform comparative processing to distinguish the foreground P of the grayscale image pro With background P back part;

[0063] The stationary object capture mask and acquisition module is used to capture the foreground P pro Get the similarity evaluation matrix Mat eva , via Mat eva Get the improved similarity evaluation matrix Mat * eva , via Mat * eva Get the similarity domain area Mask still0 and Mask still1 ;

[0064] The image quality detection module is used to establish image quality detection standards and classify images into images that meet the standards, blocked images, and accumulated images;

[0065] The impurity and broken rice grain detection module is used to perform deeplab segmentation and Yolo target detection on the images that meet the standards, and obtain the impurity mass g(za), the broken rice grain mass G(A a ) and rice grain mass G(A), and the impurity rate and breakage rate of the corresponding rice grains were obtained by calculation.

[0066] In the above solution, the control unit further comprises an alarm module; the alarm module is used to control the alarm device to sound an alarm when the image is judged to be a blocking image.

[0067] A harvester comprises a control system of the method for detecting the debris and crushing rate of a hilly and mountainous combine harvester.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] 1. The present invention solves the problem of excessive impurity content in the grain conveying auger of the harvesting machinery due to the undulating topography of hilly and mountainous areas, which causes accumulation of impurities and stickiness.

[0070] 2. The present invention improves the detection accuracy of impurity content and breakage rate, and can reflect the grain transportation situation during the operation of the harvester in hilly and mountainous areas.

[0071] 3. When the working space is limited in hilly and mountainous areas, the present invention can also issue an alarm when the feed amount is too large or the cleaning effect is poor, resulting in grain blockage. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a schematic diagram of a flow chart of a method for detecting the debris and crushing rate of a combine harvester in hilly and mountainous areas according to one embodiment of the present invention.

[0073] Figure 2 It is a schematic diagram of three groups of sample plots of a method for detecting the debris and breakage rate of a combine harvester in hilly and mountainous areas according to one embodiment of the present invention.

[0074] Figure 3 It is a schematic diagram of image processing of a method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to one embodiment of the present invention.

[0075] Figure 4 It is a specific application schematic diagram of an embodiment of the present invention.

[0076] In the figure: 1. Grain conveying auger; 3. Observation window; 4. Lighting device; 5. Image acquisition device; 6. Control unit. DETAILED DESCRIPTION

[0077] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0078] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "front", "back", "left", "right", "up", "down", "axial", "radial", "vertical", "horizontal", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0079] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0080] Example 1

[0081] like Figure 1 , Figure 2 As shown, a method for detecting the debris and crushing rate of a combine harvester in hilly and mountainous areas comprises the following steps:

[0082] Step S1, image acquisition: The image acquisition device collects the photo stream of rice in the grain conveying auger and transmits it to the control unit. The control unit pre-sets the sampling time of two adjacent photos in the photo stream as t, and uses the sampling time t as the subscript to mark the photo stream as k represents the number of photos in the photo stream;

[0083] Step S2, image preprocessing: perform ROI processing on the photo collected in step S1, perform color inversion on the photo, and perform proportional calculation on the photo pixels;

[0084] Step S3, obtain the fused grayscale image: intercept the photo in the photo stream preprocessed in step S2 Get the grayscale image of the photo Will Fusion, get the fused grayscale image

[0085] Step S4, obtaining the grayscale image basic threshold: arranging sampling quadrat, obtaining The basic threshold th bass ;

[0086] Step S5, distinguishing the foreground and background of the grayscale image: The grayscale image fused in step S3 The basic threshold th in step S4 bass Perform comparative processing to distinguish the foreground P of the grayscale image pro With background P back part;

[0087] Step S6: Create a static object capture mask still And: by the foreground P in step S5 pro Get the similarity evaluation matrix Mat eva , via Mat eva Get the improved similarity evaluation matrix Mat * eva , via Mat * eva Get the similarity domain area Black Mask still0 and white mask still1 ;

[0088] Step S7, judging image quality: establishing an image quality detection standard, combining the similarity domain area obtained in step S6 The image is compared with the detected image area, and the image is divided into a standard image and an accumulation image. When the image is judged to be a standard image, step S8 is entered. When the image is judged to be an accumulation image, the image stream within the sampling time t is compared with the Mask still After fusion, the image is entered into step S1 to step S7, and the image quality is judged again until the image is judged to meet the standard image, and then the image is entered into step S8;

[0089] Step S8, detection of impurity and broken rice grain rate: The images meeting the standard in step S7 are subjected to deeplab segmentation and Yolo target detection to obtain the impurity mass g(za) and the broken rice grain mass G(A a ) and rice grain mass G(A), and the impurity rate and breakage rate of the corresponding rice grains were obtained by calculation.

[0090] Preferably, the specific steps of obtaining the fused grayscale image in step S3 are:

[0091] Step S3.1: Change the sensitivity of the three colors R, G, and B in the image according to the brightness reflected by the different grain colors in the image, specifically:

[0092]

[0093] in And n>m>l, n, m, l are the influencing factors of RGB channel proportion respectively;

[0094] Get the grayscale image:

[0095] Step S3.2: convert the image grayscale image obtained in step S3.1 into: Fusion, get the fused grayscale image The specific steps are:

[0096] Will The overall gray value is recently integerized and averaged to obtain

[0097] Will Grayscale values ​​are linearly fused to obtain the fused grayscale image

[0098] Preferably, the specific steps of obtaining the grayscale image basic threshold in step S4 are:

[0099] Step S4.1, arrange sampling plots: According to the N*N grid uniform differentiation, N is the side length of the square inscribed in the circular field of view, M is the number of sample squares, the side length of the sample square is equal to N / M, and it is rounded down, and the entire grayscale image is reset to M*M basic sample squares; take the sample square numbered y, where y=3x+1(x∈[0,(M*M-1) / 3],x∈Z), and obtain (M*M-1) / 3 sampling sample squares with uniform distribution: q1...q k ……q 21 ;

[0100] Preferably, the value of M is 8;

[0101] Step S4.2: Obtain The basic threshold th bass :Slide (M*M-1) / 3 sampling plots horizontally according to one frame and one grid to obtain 3 groups of plot groups. In each group, the grayscale values ​​in (M*M-1) / 3 sampling plots are slid according to Gaussian convolution to obtain the reference basic threshold. The (M*M-1) / 3 reference basic thresholds are averaged to obtain th1, th2, and th3. The three groups of data th1, th2, and th3 are averaged to obtain the basic threshold th bass .

[0102] Preferably, the specific steps of S5 for distinguishing the foreground and background of the grayscale image are:

[0103] The fused grayscale image With the basic threshold th bassThe comparison is as follows:

[0104]

[0105] Among them, P1 is the grayscale image obtained after comparison, P back For the background part, P pro Foreground part.

[0106] Preferably, the S6 prepares a stationary object capture mask and;

[0107] S6.1, through P pro and Proportional operation to obtain the similarity evaluation matrix Mat eva :Will The mask Mask1 of the same size filled with 1 is linearly added and fused to process the pixels as non-zero. pro Divided by the pixel after non-zero processing Get the similarity evaluation matrix Mat composed of similarity coefficients eva ;

[0108] S6.2, similarity evaluation matrix Mat eva Perform bilateral filtering with boundaries A and B, where A and B are the thresholds of bilateral filtering, A is 1.8-2, B is 2-2.2, and the similarity coefficient within the filtering range is assigned to 0, and the range is recorded as similar domain area0; the similarity coefficient that does not belong to the filtering range is assigned to 1, and the range is recorded as non-similar domain area1; an improved similarity evaluation matrix Mat is obtained, which marks the similar domain area0 as 0 and the non-similar domain area1 as 1 * eva ;

[0109] Preferably, A is 1.9 and B is 2.1;

[0110] S6.3. Traverse and improve the similarity evaluation matrix Mat * eva , get the similarity domain area

[0111] S6.4. Traverse and improve the similarity evaluation matrix Mat * eva , save the relative position coordinates of the 0 and 1 values ​​in the matrix relative to the image; assign the RGB three channels of the mask corresponding to the 0 value relative coordinates to 0, and assign the RGB three channels of the mask corresponding to the 1 value relative coordinates to 255; process the similarity domain area0 as a black mask to obtain a black mask Mask still0 , treat the non-similar domain area1 as a white mask to obtain a white mask Mask still1 .

[0112] Preferably, the image in the image quality determination in step S7 also includes a blocked image; when the image is determined to be a blocked image, the control unit issues an alarm.

[0113] Preferably, the specific steps of judging the image quality in S7 are:

[0114] Establish image quality detection standards and classify images into three categories: images that meet the standards, blocked images, and accumulated images. The specific rules are as follows:

[0115]

[0116] Among them, S1 is the judgment area, and D1 is the flag bit of whether it is judged as a standard image; if the flag bit D1 is 0, it means that the image meets the sample detection standard and is brought into the mixed and broken detection calculation; if the flag bit D1 is 1, it means that the image does not meet the sample detection standard, and enters the steps S1-S7 again to make another image quality judgment.

[0117] Preferably, the S8 impurity and breakage rate detection comprises the following steps:

[0118] When the flag D1=0, the image is subjected to deeplab segmentation and Yolo target detection respectively, where:

[0119] The image is processed by deeplab semantic segmentation to obtain the pixel area of ​​the residual and broken rice grain samples. Then, according to the mass-pixel correspondence of the residual, the residual mass g(za) is obtained; according to the mass-pixel correspondence of the broken rice grains, the broken rice grain G(A) is obtained. a )’s quality;

[0120] The rice grain information is input into the Yolo neural network for target detection to obtain the number information of the rice grains, and the rice grain mass G(A) is obtained through the number information of the rice grains;

[0121] Enter the calculation formula:

[0122]

[0123]

[0124] Obtain the impurity rate and breakage rate of the corresponding rice grains.

[0125] Preferably, the S8 impurity and breakage rate detection further comprises the following steps:

[0126] When the flag bit D1 = 1, the following judgment is made for images that do not meet the detection requirements:

[0127]

[0128] Where S2 is the blockage judgment area, and D2 is the flag bit for judging whether the image is blocked or piled up; if the flag bit D2 is 1, the blockage alarm state is entered and an alarm image is generated; if the flag bit D2 is 0, it is judged to be piled up;

[0129] When the flag D2 = 0, the image is judged as a pile-up phenomenon, and the image stream within the sampling time t is compared with the Mask still After fusion, the image is input into step S1 to step S7, and then the image quality is judged;

[0130] When the flag D2=1, the machine enters the jam alarm state and stops operating.

[0131] Preferably, a non-traversal visual digitization processing method can be used to complete the accumulation capture of partial visual field output.

[0132] Removal processing and blockage alarm further improve the accuracy of debris and crushing detection.

[0133] Combination Figure 3 and 4 As shown, a control system according to the method for detecting the debris and brokenness rate of a hilly combine harvester comprises an image acquisition device 5 and a control unit 6; the control unit comprises a photo stream marking module, an image preprocessing module, a fused grayscale image acquisition module, a grayscale image basic threshold acquisition module, a grayscale image foreground and background distinction module, a stationary object capture mask and acquisition module, an image quality detection module and a debris and brokenness rate detection module;

[0134] The image acquisition device is used to collect the photo stream of rice in the grain conveying auger and transmit it to the control unit. The photo stream marking module is used to pre-set the sampling time of two adjacent photos in the photo stream as t, and use the sampling time as the subscript to mark the photo stream as

[0135] The image preprocessing module is used to perform ROI processing on the collected photos, invert the colors of the photos, and perform proportional calculation on the pixels of the photos;

[0136] The fused grayscale image acquisition module is used to capture photos in the photo stream Get the grayscale image of the photo Will Fusion, get the fused grayscale image

[0137] The grayscale image basic threshold acquisition module is used to arrange sampling quadrangles to obtain The basic threshold th bass ;

[0138] The grayscale image foreground and background distinguishing module is used to distinguish the foreground and background of the grayscale image: the fused grayscale image With the basic threshold th bass Perform comparative processing to distinguish the foreground P of the grayscale image pro With background P back part;

[0139] The stationary object capture mask and acquisition module is used to capture the foreground P pro Get the similarity evaluation matrix Mat eva , via Mat eva Get the improved similarity evaluation matrix Mat * eva , via Mat * eva Get the similarity domain area Mask still0 and Mask still1 ;

[0140] The image quality detection module is used to establish image quality detection standards and classify images into images that meet the standards, blocked images, and accumulated images;

[0141] The impurity and broken rice grain detection module is used to perform deeplab segmentation and Yolo target detection on the images that meet the standards, and obtain the impurity mass g(za), the broken rice grain mass G(A a ) and the rice grain mass G(A), and the impurity rate and breakage rate of the corresponding rice grain 2 are obtained by calculation.

[0142] Preferably, the control unit further comprises an alarm module; the alarm module is used to control the alarm device to sound an alarm when the image is judged to be a blocking image.

[0143] Preferably, it also includes an illumination device 4, and the illumination device 4 is used to illuminate the acquisition area of ​​the image acquisition device 5 with corresponding brightness.

[0144] Preferably, it also includes an observation window 3, and the observation window 3 can be used to observe the rice grains in the grain conveying auger 1.

[0145] According to this embodiment, preferably, the image acquisition device is an industrial camera.

[0146] Example 2

[0147] A harvester includes a control system of the method for detecting the debris and crushing rate of a hilly and mountainous combine harvester, and thus has the beneficial effects described in Example 1, which will not be repeated here.

[0148] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0149] The series of detailed descriptions listed above are only specific descriptions of feasible embodiments of the present invention. They are not intended to limit the scope of protection of the present invention. All equivalent embodiments or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting the debris and crushing rate of a combine harvester in hilly and mountainous areas, characterized in that: The following steps are involved: Step S1, image acquisition: The image acquisition device collects the photo stream of rice in the grain conveying auger and transmits it to the control unit. The control unit pre-sets the sampling time of two adjacent photos in the photo stream as t, and uses the sampling time t as the subscript to mark the photo stream as k represents the number of photos in the photo stream; Step S2, image preprocessing: perform ROI processing on the photo collected in step S1, perform color inversion on the photo, and perform proportional calculation on the photo pixels; Step S3, obtain the fused grayscale image: intercept the photo in the photo stream preprocessed in step S2 Get the grayscale image of the photo Will Fusion, get the fused grayscale image Step S4, obtaining the grayscale image basic threshold: arranging sampling quadrat, obtaining The basic threshold th bass ; Step S5, distinguishing the foreground and background of the grayscale image: The grayscale image fused in step S3 The basic threshold th in step S4 bass Perform comparative processing to distinguish the foreground P of the grayscale image pro With background P back part; Step S6: Create a static object capture mask still And: The foreground P in step S5 pro Get the similarity evaluation matrix Mat eva , via Mat eva Get the improved similarity evaluation matrix Mat * eva , via Mat * eva Get the similarity domain area Black Mask still0 and white mask still1 ; Step S7, judging image quality: establishing an image quality detection standard, combining the similarity domain area obtained in step S6 The image is compared with the detected image area, and the image is divided into a standard image and an accumulation image. When the image is judged to be a standard image, step S8 is entered. When the image is judged to be an accumulation image, the image stream within the sampling time t is compared with the Mask still After fusion, the image is entered into step S1 to step S7, and the image quality is judged again until the image is judged to meet the standard image, and then the image is entered into step S8; Step S8, detection of impurity and broken rice grain rate: The images meeting the standard in step S7 are subjected to deeplab segmentation and Yolo target detection to obtain the impurity mass g(za) and the broken rice grain mass G(A a ) and rice grain mass G(A), and the impurity rate and breakage rate of the corresponding rice grains were obtained by calculation.

2. The method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to claim 1, characterized in that: The specific steps of step S3 to obtain the fused grayscale image are: Step S3.1: Change the sensitivity of the three colors R, G, and B in the image according to the brightness reflected by the different grain colors in the image, specifically: in And n>m>l, n, m, l are the influencing factors of RGB channel proportion respectively; Get the grayscale image: Step S3.2: convert the image grayscale image obtained in step S3.1 into: Fusion, get the fused grayscale image The specific steps are: Will The overall gray value is recently integerized and averaged to obtain Will Grayscale values ​​are linearly fused to obtain the fused grayscale image 3. The method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to claim 1, characterized in that: The specific steps of step S4 for obtaining the grayscale image basic threshold are: Step S4.1, arrange sampling plots: According to the N*N grid uniform differentiation, N is the side length of the square inscribed in the circular field of view, M is the number of sample squares, the side length of the sample square is equal to N / M, and it is rounded down, and the entire grayscale image is reset to M*M basic sample squares; take the sample square numbered y, where y=3x+1(x∈[0,(M*M-1) / 3],x∈Z), and obtain (M*M-1) / 3 sampling sample squares with uniform distribution: q1...q k ……q 21 ; Step S4.2: Obtain The basic threshold th bass :Slide (M*M-1) / 3 sampling plots horizontally according to one frame and one grid to obtain 3 groups of plot groups. In each group, the grayscale values ​​in (M*M-1) / 3 sampling plots are slid according to Gaussian convolution to obtain the reference basic threshold. The (M*M-1) / 3 reference basic thresholds are averaged to obtain th1, th2, and th3. The three groups of data th1, th2, and th3 are averaged to obtain the basic threshold th bass .

4. The method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to claim 1, characterized in that: The specific steps of S5 for distinguishing the foreground and background of the grayscale image are as follows: The fused grayscale image With the basic threshold th bass The comparison is as follows: Among them, P1 is the grayscale image obtained after comparison, P back For the background part, P pro Foreground part.

5. The method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to claim 1, characterized in that: The S6 prepares a stationary object capture mask and; S6.1, through P pro and Proportional operation to obtain the similarity evaluation matrix Mat eva :Will The mask Mask1 of the same size filled with 1 is linearly added and fused to process the pixels as non-zero. pro Divided by the pixel after non-zero processing Get the similarity evaluation matrix Mat composed of similarity coefficients eva ; S6.2, similarity evaluation matrix Mat eva Perform bilateral filtering with boundaries A and B, where A and B are the thresholds of bilateral filtering, A is 1.8-2, B is 2-2.2, and the similarity coefficient within the filtering range is assigned to 0, and the range is recorded as similarity domain area0; the similarity coefficient outside the filtering range is assigned to 1, and the range is recorded as non-similar domain area1; Get an improved similarity evaluation matrix Mat that marks the similar domain area0 as 0 and the non-similar domain area1 as 1 * eva ; S6.

3. Traverse and improve the similarity evaluation matrix Mat * eva , get the similarity domain area S area1 ; S6.

4. Traverse and improve the similarity evaluation matrix Mat * eva , save the relative position coordinates of the 0 and 1 values ​​in the matrix relative to the image; assign the RGB three channels of the mask corresponding to the 0 value relative coordinates to 0, and assign the RGB three channels of the mask corresponding to the 1 value relative coordinates to 255; process the similarity domain area0 as a black mask to obtain a black mask Mask still0 , treat the non-similar domain area1 as a white mask to obtain a white mask Mask still1 .

6. The method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to claim 1, characterized in that: The image in the step S7 judges the image quality also includes a blocked image; when the image is judged to be a blocked image, the control unit issues an alarm.

7. The method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to claim 6, characterized in that: The specific steps of S7 for judging the image quality are as follows: Establish image quality detection standards and classify images into three categories: images that meet the standards, blocked images, and accumulated images. The specific rules are as follows: Among them, S1 is the judgment area, and D1 is the flag bit of whether it is judged as a standard image; if the flag bit D1 is 0, it means that the image meets the sample detection standard and is brought into the mixed and broken detection calculation; if the flag bit D1 is 1, it means that the image does not meet the sample detection standard, and enters the steps S1-S7 again to make another image quality judgment.

8. The method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to claim 7, characterized in that: The S8 impurity and breakage rate detection comprises the following steps: When the flag D1=0, the image is subjected to deeplab segmentation and Yolo target detection respectively, where: The image is processed by deeplab semantic segmentation to obtain the pixel area of ​​the residual and broken rice grain samples. Then, according to the mass-pixel correspondence of the residual, the residual mass g(za) is obtained; according to the mass-pixel correspondence of the broken rice grains, the broken rice grain G(A) is obtained. a )’s quality; The rice grain information is input into the Yolo neural network for target detection to obtain the number information of the rice grains, and the rice grain mass G(A) is obtained through the number information of the rice grains; Enter the calculation formula: Obtain the impurity rate and breakage rate of the corresponding rice grains.

9. The method for detecting the debris and broken rate of a combine harvester in hilly and mountainous areas according to claim 7, characterized in that: The S8 impurity and breakage rate detection also includes the following steps: When the flag bit D1 = 1, the following judgment is made for images that do not meet the detection requirements: Wherein, S1 is the judgment area, S2 is the blockage judgment area, and D2 is the flag bit for judging whether the image is blocked or piled up; if the flag bit D2 is 1, the blockage alarm state is entered and an alarm image is generated; if the flag bit D2 is 0, it is judged to be piled up; When the flag D2 = 0, the image is judged as a pile-up phenomenon, and the image stream within the sampling time t is compared with the Mask still After fusion, the image is input into step S1 to step S7, and then the image quality is judged; When the flag D2=1, the machine enters the jam alarm state and stops operating.

10. A control system for the method for detecting the debris and crushing rate of a combine harvester in hilly and mountainous areas according to any one of claims 1 to 9, characterized in that: It includes an image acquisition device and a control unit; the control unit includes a photo stream marking module, a picture preprocessing module, a fused grayscale image acquisition module, a grayscale image basic threshold acquisition module, a grayscale image foreground and background distinction module, a stationary object capture mask and acquisition module, an image quality detection module and a mixed fragmentation rate detection module; The image acquisition device is used to collect the photo stream of rice in the grain conveying auger and transmit it to the control unit. The photo stream marking module is used to pre-set the sampling time of two adjacent photos in the photo stream as t, and use the sampling time as the subscript to mark the photo stream as The image preprocessing module is used to perform ROI processing on the collected photos, invert the colors of the photos, and perform proportional calculation on the pixels of the photos; The fused grayscale image acquisition module is used to capture photos in the photo stream Get the grayscale image of the photo Will Fusion, get the fused grayscale image The grayscale image basic threshold acquisition module is used to arrange sampling quadrangles to obtain The basic threshold th bass ; The grayscale image foreground and background distinguishing module is used to distinguish the foreground and background of the grayscale image: the fused grayscale image With the basic threshold th bass Perform comparative processing to distinguish the foreground P of the grayscale image pro With background P back part; The stationary object capture mask and acquisition module is used to capture the foreground P pro Get the similarity evaluation matrix Mat eva , via Mat eva Get the improved similarity evaluation matrix Mat * eva , via Mat * eva Get the similarity domain area S area1 、Mask still0 and Mask still1 ; The image quality detection module is used to establish image quality detection standards and classify images into images that meet the standards, blocked images, and accumulated images; The impurity and broken rice grain detection module is used to perform deeplab segmentation and Yolo target detection on the images that meet the standards, and obtain the impurity mass g(za), the broken rice grain mass G(A a ) and rice grain mass G(A), and the impurity rate and breakage rate of the corresponding rice grains were obtained by calculation.

11. The control system of the method for detecting the debris and crushing rate of a combine harvester in hilly and mountainous areas according to claim 10, characterized in that: The control unit also includes an alarm module; the alarm module is used to control the alarm device to alarm when the image is judged to be a blocking image.

12. A harvester, characterized in that: A control system comprising the method for detecting the debris and crushing rate of a hilly and mountainous combine harvester as described in claim 10 or 11.

Images