Detecting crop conditions in front of a harvester

Abstract

A method is provided for an image processor to detect crop conditions in front of a header on a crop harvesting machine. The header includes a ground engaging device mounted on a bottom of the header. The ground engaging device has an adjustable height. The method comprises the steps of obtaining, by an optical sensor, an image of a field in front of the header, using the image from the optical sensor to identify lodged crop in the field, identifying a location of the lodged crop in the field, determining when the header will reach the location of the lodged crop in the field, and when it is determined that the header will reach the location of the lodged crop in the field, sending a signal to a machine controller to retract the ground engaging device.

Description

DETECTING CROP CONDITIONS IN FRONT OF A HARVESTER TECHNICAL FIELD

[0001] The present invention relates to systems and methods for detecting crop conditions in front of a harvester.BACKGROUND OF THE INVENTION

[0002] Combine headers typically cut and gather a width of standing crop which is then transported inwardly to the center of the header where a feed draper and feed drum feed the crop rearward into the combine feederhouse. The crop is then processed by the combine. On a draper header, the inward transport of crop material is typically achieved using a left-hand and a right-hand side-draper.

[0003] While operating a harvester the operator must continuously observe the condition of the crop in front of the machine, paying special attention to recognize any special conditions that may require an adjustment to the harvester, or header attached to the front of the harvester. One common crop condition encountered during harvesting operations is referred to as "lodged crop," which describes crop that has buckled and fallen down either partially or entirely. Lodging can occur for many reasons including wind, heavy precipitation, hail or snow, over fertilization, wildlife damage, etc. It is important to recognize this condition before the harvester reaches the lodged crop because most often the header and harvester will need to be adjusted to avoid missing crop.SUMMARY OF THE INVENTION

[0004] According to one embodiment, there is provided a method for detecting crop conditions in front of a header on a crop harvesting machine. The header includes a ground engaging device mounted on a bottom of the header. The ground engaging device has an adjustable height. The method comprises the steps of obtaining an image of a field in front of the header, using the image to identify lodged crop in the field, identifying a location of the lodged crop in the field, determining when the header will reach the location of the lodged crop inthe field, and when it is determined that the header will reach the location of the lodged crop in the field, retracting the ground engaging device.

[0005] According to another embodiment, there is provided a system for detecting crop conditions in front of a header on a crop harvesting machine. The header includes a ground engaging device mounted on a bottom of the header. The ground engaging device has an adjustable height. The system comprises a sensor and an image processor. The sensor is configured to obtain an image of a field in front of the header. The image processor is configured to use the image to identify lodged crop in the field, identify a location of the lodged crop in the field, determine when the header will reach the location of the lodged crop in the field, and when the image processor determines that the header will reach the location of the lodged crop in the field, sending a signal to a machine controller to retract the ground engaging device.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Advantages of the present invention will be readily appreciated as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings wherein:

[0007] Figure 1 is a top view of a crop harvesting machine harvesting crop in accordance with one embodiment of the present invention;

[0008] Figure 2 is a side view of the crop harvesting machine of Figure 1;

[0009] Figure 3 is an enlarged fragmentary side view of the components for adjusting the cut height and angle on the crop harvesting machine of Figure 1;

[0010] Figure 4 is a schematic view of a system for detecting crop conditions in front of the crop harvesting machine in accordance with one embodiment of the present invention;

[0011] Figure 5 is a flow diagram illustrating one embodiment for detecting crop conditions in front of the harvesting machine in accordance with the present invention;

[0012] Figure 6 is an exemplary image frame obtained using the process for detecting crop conditions in accordance with the present invention;

[0013] Figure 7 reflects the image frame of Figure 6 after application of header segmentation;

[0014] Figure 8 reflects the image frame of Figure 7 after application of lodged crop segmentation;

[0015] Figure 9 reflects exemplary classifications of the lodge direction classifier;

[0016] Figure 10 reflects the image frame of Figure 6 after application of a field gridder; and

[0017] Figure 11 is a flow diagram illustrating one embodiment for adjusting the crop harvesting machine in accordance with the present invention.DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0018] Referring to the Figures, where like numerals indicate like or corresponding parts throughout the several views, a crop harvesting machine for harvesting agricultural crops is shown generally at 10. Referring to Figures 1-3, the crop harvesting machine 10 includes a header 12 mounted on a feederhouse 14 on a combine harvester 16 for movement in a forward working direction D. The harvester 16 includes a cab 18 mounted on the front of the harvester 16.

[0019] The header 10 includes a front portion 20, a rear portion 22 and a bottom portion 24 extending laterally between opposite ends 26. Crop pick-up reels 28 are positioned generally above the front portion 20 of the header 12 for engaging the crops 38 to be harvested. A cutter bar assembly 30 operatively extends across the front portion 20 of the header 12 between the opposite ends 26 for cutting the crops 38 to be harvested. A draper belt assembly 32 extends behind the cutter bar assembly 30 and includes a feed draper belt 34 and a pair of side draper belts 36 disposed on opposites sides of the feed draper belt 34. The side draper belts 36 transport harvested agricultural crops to the feed draper belt 34, which then directs the harvested crops to the feederhouse 14 of the combineharvester 16 for processing. Figure 1 illustrates the unharvested crop 38 in front of the crop harvesting machine 10 as well as the cut stubble 40 after the crop harvesting machine 10 has harvested the crop. The unharvested crop 38 includes normal standing crop 42 as well as lodged crop 44.

[0020] Referring to Figures 2 and 3, gauge wheels 46 are pivotably coupled to the bottom portion 24 of the header 12. Although depicted as gauge wheels 46, the header 12 may include any suitable ground engaging device without varying the scope of the invention. Referring to Figure 3, the gauge wheel position 48 can be adjusted to adjust the cut height 50 of the header 12. In the retracted position, the gauge wheels 46 lower the header 12 resulting in a shorter cut height, and in the extended position, the gauge wheels 46 raise the header 12 resulting in a taller cut height.

[0021] The header 12 includes a header tilt actuator 52 that controls the pitch angle of the header 12 relative to the feederhouse 14, thereby adjusting the cut angle of the cutter bar assembly 30 relative to ground 54. The header also includes a header suspension 56 on each end 26 of the header 12 that supports the majority of the weight of the header 12, thereby leaving a relatively small proportion of the weight to rest on the gauge wheels 46 along the ground 54. Independent flexing of each header suspension 56 adjusts the suspension position 58 so that each end 26 of the header 12 can float upwardly and downwardly as it follows the ground 54. The crop harvesting machine 10 also includes a feederhouse actuator 60 to adjust the rotational position of the feederhouse 14.

[0022] Referring to Figure 4, the crop harvesting machine 10 includes a system 62 for detecting crop conditions in front of the harvesting machine 10. The system 62 includes an optical sensor 64, an image processor 66 and a machine controller 68. Although depicted in Figures 1 and 2 with a single optical sensor 64, the system 62 may include any suitable number of optical sensors 64 without varying the scope of the invention. The optical sensor 64 is preferably positioned so that the header 12 and a portion of the crop 38 in front of the header 12 are in the sensor field of view 70. The optical sensor 64 may be mounted anywhere on the crop harvesting machine 10. For example, the optical sensor 64 may be mounted on the top center portion on the front of the cab 18.

[0023] Figure 5 illustrates an exemplary process 72 for detecting crop conditions in front of the harvesting machine 10. At the start of the process (step 74), the operator selects the crop type being harvested via a user interface device, typically a screen in the harvester cab 18, which is sometimes referred to as a virtual, or universal terminal (VT) (step 76). Alternatively, the crop type may be selected automatically by the system 62 based on the crop characteristics it recognizes using the optical sensor 64 and image processor 66. The system 62 then loads the trained segmentation or classification models corresponding to the type of crop being harvested (step 78). The monitoring software pipeline then begins.

[0024] The system 62 obtains images 94 from the optical sensor 64 (step 80) and transmits these images 94 to the image processor 66 for analysis. Figure 6 reflects an exemplary image 94 obtained from the optical sensor 64. The image 94 includes the reel 28 and the draper belts 34, 36 on the header 12, and reflects the crop flow 96 on the draper area 34, 36. The image 94 also includes an image of the field 98, which includes the unharvested crop 38 inside 102 the forward path 100 of the header 12 and both unharvested crop 38 and cut crop 40 outside 104 the forward path 100 of the header 12. The crop 38 inside 102 the forward path 100 of the header 12 includes normal standing crop 42 and lodged crop 44.

[0025] Returning to Figure 5, after obtaining the images 94 from the optical sensor 64 at step 80, the system performs a "header segmentation" function (step 82) where each image frame 94 is analyzed to distinguish the header 12 from the field 98. As reflected in Figure 7, the system 62 identifies three segmentation masks 106a, 106b and 106c on the image frame 94. The first segmentation mask 106a identifies the pixels identified as part of the field 98. The second segmentation mask 106b identifies the pixels identified as part of the reel 28, and the third segmentation mask 106c identifies the pixels identified as part of the draper area 34, 36.

[0026] Returning to Figure 5, after header segmentation, the system 62 sends the pixels identified as part of the field 98 to a lodged crop segmentation function (step 84). Referring to Figure 8, the lodged crop segmentation classifiesthe crop pixels into three categories: lodged crop 44, normal standing crop 42 and cut stubble 40. Thus, the lodged crop segmentation function identifies the location of the lodged crop 44, normal standing crop 42 and cut stubble 40 in the image frame 94. The lodged crop segmentation recognizes common patterns that occur in images including lodged crop. There is no specifically known feature that indicates that the crop is lodged; rather, it is the inference of a trained neural network that outputs the probability that the pixels, or pixel regions are either lodged, standing, or cut.

[0027] Returning to Figure 5, after the crop pixels have been classified, the system 62 sends the crop pixels to a lodge identifier (step 86). The lodge identifier isolates the lodged segments from the segmentation output and discards the segments that are not significantly lodged by comparing their size to a threshold limit value. The remaining lodged crop segment data is then passed to a lodge direction classifier (step 88), which classifies the direction of the lodging. For example, referring to Figure 9, the lodge direction classifier may classify the direction of lodging into as parallel and in 108 the direction of travel D, parallel and opposite 110 the direction of travel D and perpendicular 112 to the direction of travel D. Although depicted in three different directions, the lodge direction classifier may classify the direction of the lodging in any number of directions without varying the scope of the invention.

[0028] Returning to Figure 5, after classifying the direction of the lodge, the system 62 sends the information regarding the field 98 to the field gridder (step 90). Referring to Figure 10., the field gridder creates a grid pattern 114 over the region of interest on the field 98 that represents the forward path 100 of the header 12 and overlays the lodged crop grid locations 116 as well as the direction classification for the lodged crop 44 onto the grid. The system 62 determines when the header 12 will reach the lodged crop 44, and if it determines that the header 12 will imminently reach the lodged crop 44, the system 62 will set a lodging_detected flag as true. Returning to Figure 5, the resulting image data is further processed into a single grid status message that is then passed to the machine controller 68 (step 92). The process then returns to step 80 to process the next image frame 94.

[0029] Figure 11 reflects the process 118 performed by the machine controller 68 when it receives the grid status message from the crop condition processor (step 120). The machine controller 68 determines whether the lodging_detected flag is true (step 122). If the machine controller 68 determines that the lodging_detected flag is true, it sets the prev_lodged_status flag to true (step 124) and sends a message to the user interface to display a warning to the operator (step 126). The machine controller 68 then raises the gauge wheels 46 to the lodged crop position (step 128). The machine controller 68 may also initiate other adjustments of header parameters required to sufficiently gather all crop. For example, the machine controller 68 may lower the reel 28 and move it forward. The adjustment targets can be a memorized position set by the operator, or from known positions stored in a lookup table. The process 118 then returns to step 120 until it receives another grid status message from the crop condition processor.

[0030] If at step 122, the machine controller 68 determines that the lodging_detected flag is false, it determines whether the prev_lodged_status flag is set to true (step 130). If the machine controller 68 determines that the prev_lodged_status flag is set to false, it returns to step 120 until it receives another grid status message from the crop condition processor. Otherwise, the machine controller sets the prev_lodged_status flag to false (step 132) and enables normal operating mode on the crop harvesting machine 10, which includes lowering the gauge wheels 46 to the normal position. The process 118 then returns to step 120 until it receives another grid status message from the crop condition processor.

[0031] The invention has been described in an illustrative manner, and it is to be understood that the terminology, which has been used, is intended to be in the nature of words of description rather than of limitation. Many modifications and variations of the present invention are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims, the invention may be practiced other than as specifically described.

Claims

CLAIMS1. A method for an image processor to detect crop conditions in front of a header on a crop harvesting machine, wherein the header includes a ground engaging device mounted on a bottom of the header, wherein the ground engaging device has an adjustable height, the method comprising the steps of:obtaining, by an optical sensor, an image of a field in front of the header; using the image from the optical sensor to identify lodged crop in the field;identifying a location of the lodged crop in the field;determining when the header will reach the location of the lodged crop in the field; andwhen it is determined that the header will reach the location of the lodged crop in the field, sending a signal to a machine controller to retract the ground engaging device.

2. The method of claim 1 further comprising the steps of:identifying a path of the header;creating a grid pattern on the image over the path of the header; and identifying the lodged crop on the grid pattern.

3. The method of claim 2, further comprising the step of using the image to determine a direction for the lodged crop.

4. The method of claim 3, further comprising the step of identifying the direction of the lodged crop on the grid pattern.

5. The method of any one of claims 1-4 further comprising the step of sending a notification regarding the lodged crop to a user interface.

6. A system for detecting crop conditions in front of a header on a crop harvesting machine, wherein the header includes a ground engaging device mounted on a bottom of the header, wherein the ground engaging device has an adjustable height, the system comprising:an optical sensor configured to obtain an image of a field in front of the header;an image processor configured to:use the image to identify lodged crop in the field;identify a location of the lodged crop in the field;determine when the header will reach the location of the lodged crop in the field; andwhen the image processor determines that the header will reach the location of the lodged crop in the field, sending a signal to a machine controller to retract the ground engaging device.

7. The system of claim 6 wherein the image processor is configured to: identify a path of the header;create a grid pattern on the image over the path of the header; and identify the lodged crop on the grid pattern.

8. The system of claim 7, wherein the image processor is configured to use the image to determine a direction for the lodged crop.

9. The system of claim 8 wherein the image processor is configured to identify the direction of the lodged crop on the grid pattern.

10. The system of any one of claims 6-9 wherein the image processor is configured to send a notification regarding the lodged crop to a user interface.

Images