METHOD FOR DETECTING DISTURBANCES IN A HARVESTING ORDER

Description

[0001] The invention relates to a method for detecting malfunctions in a harvesting arrangement according to the preamble of claim 1.

[0002] State-of-the-art agricultural harvesting machines, such as combine harvesters, feature crop handling systems designed to collect harvested crops and convey them for further processing. These systems typically comprise a multitude of individual components that can be operated with varying parameters. For example, the cutting height, reel position and speed, and the movement of the transverse auger and inclined conveyor can be adjusted in the crop handling systems of combine harvesters. Similarly, the harvester's travel speed determines the amount of crop picked up by the handling system.

[0003] Depending on the settings of the harvesting system, as well as the type, quantity, and characteristics of the harvested crop, various types of malfunctions can occur. A well-known example is a crop jam in the header of a combine harvester. Once such a jam has fully developed, the harvesting process must be stopped, and a time-consuming clearing operation is required. The time this takes seriously impacts the combine's efficiency. Typically, such a jam—or other malfunctions in harvesting systems—does not occur suddenly, but rather develops gradually over a longer period until it becomes completely blocked, necessitating a complete shutdown of operations.One problem is that only when a complete blockage is readily and clearly recognizable is it too late to correct the operation of the harvesting system, which would have functioned without interruption. Therefore, efforts are being made to detect such an impending blockage or other disruption early on, so that appropriate countermeasures, such as temporarily reducing the current harvest throughput, can prevent the blockage without interrupting harvesting operations.

[0004] The German patent application DE 10 2008 032 191 A1, which is considered the closest available patent application, describes a self-propelled harvesting machine with a sensor unit for monitoring the crop flow within the harvesting header of the machine. Specifically, a camera is provided that creates images of the crop flow. These images are subjected to an image comparison process in which the images of the crop flow are compared with reference images. Such a macroscopic comparison of the current image with the reference image allows for the determination of whether there is a deviation of the current crop flow from the acceptable crop flow as defined by the reference image.This takes advantage of the fact that with an increased amount of harvested crop in the current image, differences will already be apparent upon macroscopic observation, such as the proportion and distribution of lighter versus darker brightness areas in the two images, etc.

[0005] A disadvantage of this approach is that it focuses solely on the fundamental existence of a difference between the reference image and the current image at the macroscopic level. On the one hand, this is too unspecific, as such a difference can occur in various disturbance scenarios. While all of these can, in principle, be detected by the presence of a perceptible difference, each requires a different, and potentially contradictory, corrective action to avoid or resolve them. Therefore, the intended response to the detected difference may not solve the problem but could even exacerbate it. Another disadvantage of this prior art approach is that there are also disturbances that cannot be detected, or at least not early enough, using such a macroscopic image comparison.Furthermore, EP 2 143 316 discloses a detection device that uses image analysis to detect crop buildup problems in a harvesting header and then initiates measures to counteract these detected buildup effects. EP 2 143 316 proposes to eliminate this crop buildup effect by increasing the crop flow in the harvesting header and thus the crop throughput. However, such a solution has the disadvantage that the crop buildup effect would be further exacerbated by high crop throughputs.

[0006] Against this background, the object of the present invention is to further develop and improve the prior art method for detecting malfunctions in a crop recovery arrangement with regard to accuracy, reliability and timeliness.

[0007] The above problem is solved in a method for detecting disturbances of a harvesting arrangement with the features of the preamble of claim 1 by the features of the characterizing part of claim 1.

[0008] Essential to the invention is the realization that information about the movement of individual objects within the crop flow can provide a reliable indicator for the early detection of malfunctions in the crop harvesting system. Instead of a macroscopic comparison of images as a whole, the analysis is performed at a microscopic level, namely on a small part or section of the image relative to the entire captured area. This realization, in turn, is based on the fact that, to a first approximation, the movement of different objects within the crop flow depends more on the macroscopic properties of the crop flow and, if applicable, the object's position within the crop flow than on whether the object belongs to the grain or is merely another component of the crop.Therefore, the focus is less on identifying a specific type of object within the harvested crop, but rather on detecting any object within the crop that is visually distinct enough to allow its movement within the crop stream to be tracked. At least for the other components of the crop stream located in the vicinity of this object, it can then be concluded that their movement is strongly correlated with that of the detected object.

[0009] Furthermore, according to the invention, the movement of the object with regard to the magnitude of the speed on the one hand and the direction on the other hand, as well as any respective changes in these quantities, are taken into account in the assessment.

[0010] The preferred embodiment of subclaim 3 relies on an optical identification feature of the object to determine its movement in the crop flow, wherein, according to subclaim 4, the determination of the object is logically subordinate to the detection of the optical identification feature. According to subclaim 5, this can involve an examination of the entire crop flow or parts thereof.

[0011] To determine the movement of the object, dependent claim 6 proposes a particularly efficient determination of the displacement of a single pixel of the object or of the recognition feature with regard to the required computing power.

[0012] The preferred embodiments of dependent claims 7 to 9 relate to the consideration of a multitude of objects in the crop flow. Such an extended analysis allows both the frequency with which distinctive identifying features occur and their respective movements to be taken into account and related to one another.

[0013] Subclaim 10 further provides for the determination of an optical flow for a part of the detected crop flow or for the detected crop flow as a whole.

[0014] Finally, dependent claims 11 and 12 relate to preferred embodiments of the camera arrangement for recording the harvested crop flow.

[0015] Further details, features, objectives and advantages of the present invention are explained in more detail below with reference to the drawing of a preferred embodiment. The drawing shows Fig. 1 a schematic front view of a crop recovery arrangement of a combine harvester for carrying out the proposed method, Fig. 2a-c three time-delayed photographs of the crop flow at the crop recovery arrangement of the Fig. 1 During the execution of the proposed method and Fig. 3, a course of an average velocity of the harvested crop flow at the harvested crop recovery arrangement of the Fig. 1 , which was determined during the execution of the proposed procedure.

[0016] The proposed method serves to detect malfunctions in a crop retrieval system 1. The crop retrieval system 1 belongs to an agricultural harvesting machine 2, which in the illustrated embodiment is a combine harvester 3. Specifically, the crop retrieval system 1 – which here is a cutting unit 1a – comprises, in a manner known per se, a housing 4, a reel 5 adjustable on support arms, which reel 5 assists in feeding the crop flow 6 into the cutting unit 1a, and a stem divider 7 arranged at each of the outer front edges of the cutting unit 1a, which separates the individual stems from one another within the crop flow 6, thus creating a harvesting lane.The cutting unit 1a further comprises a cutter bar 8 equipped with oscillating knife blades and a transverse conveyor screw 9, which gathers the harvested crop towards the center of the cutting unit 1a and transfers it to an inclined conveyor 10, whereby the inclined conveyor 10 itself no longer belongs to the cutting unit 1a in the narrower sense.

[0017] In the proposed method, the crop flow 6 is optically detected at a crop recovery device 1 of the agricultural harvesting machine 2. This crop flow 6 need not be the entire crop flow 6 processed by the crop recovery device 1; rather, it will regularly and preferably be a portion of the entire crop flow 6 processed by the crop recovery device 1, and specifically that portion of the crop flow 6 which moves through a detection area 6a. The proposed method is characterized in that the movement of an object 11a, b within the optically detected crop flow 6 is determined, and that a disturbance of the crop recovery device 1 is detected based on the determined movement of the object 11a, b. Fig. 2a-c In this context, each example shows the recording of the harvested crop flow 6 at the location in the Fig. 1 marked detection area 6a of the harvesting arrangement 1. Two such objects 11a, b are shown as examples in the Fig. 2a-c depicted in their movement.

[0018] Preferably, the determination of the motion of object 11a, b includes a determination of the velocity of object 11a, b. This can, on the one hand, include a determination of a velocity magnitude 12a-d of the motion of object 11a, b. In the Fig. 2b, und 2c This velocity value is entered as the positional distance of the respective object 11a, b from the respective previous recording. Alternatively or additionally, the determination of the movement of object 11a, b can include a determination of a direction of movement 13a-d of object 11a, b, which is also exemplified in the Fig. 2b und 2c is registered.

[0019] It is also preferably provided that the determination of the motion of object 11a, d includes a determination of a change in the velocity of object 11a, d over time. This can likewise include a determination of the change in the magnitude of the velocity 12a-d over time and, alternatively or additionally, a determination of a change in the direction of motion 13a-d over time. As can be seen from the Fig. 2b und 2c As can be seen, both of these quantities change in the illustrated embodiment for both objects 11a, b.

[0020] Here, it is preferred that, to determine the movement of object 11a, b, a large number of images 14a-c of the crop flow 6 are generated successively over time. Specifically, the Fig. 2a a first such recording 14a, which Fig. 2b a second such recording 14b and the Fig. 2c a third such recording 14c.

[0021] According to a preferred embodiment, an optical identifier 15a, b of the object 11a, b is determined to ascertain its movement. Such an optical identifier 15a, b can be any preferably optically distinctive substructure of the object 11a, b. It is preferably provided that the optical identifier has dimensions of 1.5 cm by 1.5 cm in the viewing perspective. It can also be the object 11a, b as a whole, as is exemplified here. Preferably, a specific position of the identifier 15a, b is determined in the plurality of images 14a-c. This position is then used to determine the position of the object 15a, b. Fig. 2a-c not entered separately for the sake of clarity, but can be identified by the entered speed values ​​12a-d and the directions of movement 13a-d of the objects 11a, b.

[0022] Preferably, the object 11a, b in the harvested material stream 6 is defined based on a detected optical identifier 15a, b. Thus, according to this preferred variant, an object 11a, b is not first determined and then a corresponding optical identifier 15a, b. Rather, such an optical identifier 15a, b is first detected or recognized in the captured harvested material stream 6, and then the object 11a, b is defined as that component of the harvested material stream 6 to which the optical identifier 15a, b belongs. In such a case, the complete object 11a, b as such need not be fully known or optically detected. It is sufficient that the optical detection only concerns the identifier 15a, b, and the rest of the object 11a, b remains "invisible" in this sense. Since object 11a, b necessarily moves along with the identifying feature 15a, b, this does not restrict the determination of the movement of object 11a, b.

[0023] Here, it is preferred that, to define the object 11a, b, the recorded crop flow 6 is examined, in particular, essentially completely, using a criterion for finding optical recognition features. Thus, the entire recorded crop flow 6 can be examined for optical recognition features, whereby the criterion may refer to a particularly high-contrast contour or an unusual characteristic of this contour. These are both properties that facilitate the "tracking" and "re-identification" of the recognition feature 15a, b in the successive recordings 14a-c. The objects 11a, b are then selected as those components of the crop flow 6 whose respective movement can be determined particularly well based on their recognition features 15a, b.

[0024] Another preferred embodiment provides that the determination of the movement of object 11a, b is based on a distance determination of the position determined in each of the multiple images 14a-c. This relationship has already been presented above in the description and illustration of the velocity magnitude 12-d and the direction of movement 13a-d. Based on the rate at which the images 14a-c are created—i.e., the frame rate—and based on the geometry and dimensions of the area in which the crop flow 6 is detected—i.e., the detection area 6a—as well as depending on the corresponding device for detecting the crop flow 6, a corresponding movement or velocity magnitude 12a-d can be assigned to a distance between the respective positions of the recognition feature 15a, b in the images 14a-c.Preferably, for distance determination, a displacement of a pixel associated with object 11a, b, and in particular with the recognition feature 15a, b, is calculated between the multiple images 14a-c. By calculating only the displacement of a single pixel of the recognition feature 15a, b, the computational effort for this calculation is significantly reduced compared to considering the recognition feature 15a, b or even the object 11a, b as a whole. The resulting savings in computational effort advantageously allow for an increase in the frame rate or the processing of images 14a-c with a higher resolution.

[0025] It is preferred that a large number of objects 11a, b, and in particular optical detection features 15a, b, are identified in the detected crop flow 6. Such a large number of detection features 15a, b can be determined using the above criterion for locating optical detection features. It is conceivable that the disturbance is also detected based on a number of the identified detection features 15a, b. The point is that, when applying such a criterion, a larger or smaller number of such detection features 15a, b are identified depending on the state of the crop flow 6. On the one hand, a larger number of such features can allow the movement of a larger number of objects 11a, b to be determined, thus providing a broader basis for detecting a disturbance in the crop recovery system 1.On the other hand, the mere fact that a larger or smaller number of the identifying features 15a, b have been determined can also constitute an independent factor in the detection of such a disturbance.

[0026] It is also possible that the disturbance is detected based on a change in the number of the specific recognition features 15a, b over time.

[0027] Such a temporal change can also contribute to the detection of a disturbance in another way. Specifically, a movement over time can be determined for the multitude of recognition features 15a, b. This can also be expressed more precisely by determining a movement over time for the multitude of objects 11a, b, to which the multitude of recognition features 15a, b each belong. Preferably, in this case, the disturbance is also detected based on the movement determined for each of the multitude of recognition features 15a, b. Furthermore, the disturbance can also be detected based on a variance of the movement determined for each of the multitude of recognition features 15a, b. Here, the term "variance" is not to be understood in the narrow stochastic sense, but rather in the sense of a general difference.Therefore, one criterion for detecting a malfunction of the harvesting arrangement 1 can be how different the specific movements of the various detection features 15a, b are from each other.

[0028] It is also preferred that the determination of the motion of object 11a, b includes at least partially determining an optical flux of the detected crop flow 6. According to expert understanding, such an optical flux is the vector field of the velocity projected onto the image plane—here corresponding to a shift from image to image—of the visible points in object space. Such an optical flux thus describes the motion of a whole multitude of objects—preferably even essentially all visible objects—in the detected crop flow 6.

[0029] A preferred embodiment of the proposed method provides that a velocity property of the crop flow 6 is determined based on the specific movement of object 11a, b. This velocity property can, for example, be an average velocity 16 of the crop flow 6, whereby this does not refer to a time-based average, but rather an average of the velocities of different parts of the crop flow 6, e.g., of different objects 11a, b, at the same time. Such an average velocity 16 over a time period 17 is—together with an upper limit 18 and a lower limit 19—in the Fig. 3 depicted. Here, it is further preferred that, based on the determined velocity characteristic of the crop flow 6, a crop jam is recognized as a disturbance. In the scenario of Fig. 3 A crop jam can therefore be detected as a malfunction the moment the average speed 16 exceeds the upper limit 18. A preferred embodiment is characterized in that, upon detection of a crop jam, a reduction in the throughput of the harvesting machine 2 is triggered. In this way, the impending complete crop jam can be resolved before a permanent blockage occurs in the crop recovery arrangement 1. A different malfunction can also be detected when the lower limit 19 is undershot, which is then addressed, for example, by increasing the throughput of the harvesting machine 2. Such a malfunction could be, for example, a so-called "clumping" or "piling." This is regularly accompanied by a large number of specific detection characteristics 15a, b, and by the fact that the movement determined for each of the numerous detection characteristics 15a, b exhibits low variance.

[0030] A low or insufficient flow of harvested material can be identified as a disturbance by determining a low number of detection characteristics 15a, b. In this case, the movement determined for each of these detection characteristics 15a, b also regularly exhibits low variance.

[0031] Regarding the implementation of the proposed method, it is preferred that a crop camera arrangement 20 is arranged on the harvesting machine 2 to record the crop flow 6, as exemplified in the Fig. 1 is shown. Likewise in the Fig. 1 The preferred embodiment is shown, according to which the crop camera arrangement 6 is arranged for detection in the area between a cutting arrangement – ​​here the cutter bar 8 – and a transverse conveyor arrangement – ​​in this example the transverse auger 9 – of the crop recovery arrangement 1. This detection area is the most informative with regard to potentially occurring malfunctions of the crop recovery arrangement 1. In principle, the crop camera arrangement 20 can consist of a single camera, but this would then have to capture a comparatively large viewing angle. It is therefore preferred that – as also in the Fig. 1 As shown, the crop camera arrangement 20 comprises a plurality of camera devices 21a, b for the substantially non-overlapping acquisition of the crop flow 6. In this way, both the distortion in the acquisition of the crop flow 6 and the requirements, in particular for the resolution of the camera devices 21a, b, are reduced.

[0032] A preferred embodiment provides that the crop camera arrangement 20 is configured for monochrome acquisition of the crop stream 6. In other words, the crop camera arrangement 20 does not distinguish between different colors, but only records brightness differences for each pixel. Thus, each pixel is assigned only a brightness value of, in principle, any depth. The advantage of this is that, for the analysis relevant here, essentially only the brightness differences are relevant; therefore, color information can be dispensed with, and the data volume—relative to the resolution—is reduced. This allows for higher resolutions compared to a color image. Furthermore, it is preferred that the crop camera arrangement 20 is configured for monochrome acquisition in the infrared range. Here, the infrared range is understood to be the wavelength range from 800 nm upwards.Specifically, the crop camera arrangement 20 can be configured for detection in the range between 800 nm and 900 nm, and furthermore, in particular, in the range between 825 nm and 875 nm. Such a range limitation can also be achieved by a bandpass filter in the crop camera arrangement 20.

[0033] It is also preferred that illumination is provided by an LED light source in the infrared range or in the wavelength range intended for detection. This illumination is advantageously provided from the direction of the crop camera arrangement 20. The illumination can preferably be spatially uniform over the illumination area or over the detection area 6a and can preferably be dynamically adjusted based on the ambient light intensity.

[0034] Finally, it is preferred that the crop camera arrangement 20 has a frame rate between 20 and 40 frames per second. The crop camera arrangement 20 can also have a frame rate between 25 and 35 frames per second. It has been found that—with reference to the expected speed range of the crop flow and the dimensions of the crop retrieval arrangement 1—at this frame rate, a shift between two images in a medium range of pixels—e.g., between 25 and 15 pixels—is to be expected, which is particularly suitable for the processing described here. Bezugszeichenliste

[0035] 1. Crop recovery arrangement 1a. Cutting unit 2. Harvesting machine 3. Combine harvester 4. Housing 5. Reel 6. Crop flow 6a. Detection range 7. Stem divider 8. Cutter bar 9. Cross auger 10. Inclined conveyor 11a,b. Objects 12a-d. Speed ​​13a-d. Direction of movement 14a-c. Recordings 15a,b. Identifying features 16. Average speed 17. Time profile 18. Upper limit 19. Lower limit 20. Crop camera arrangement 21a,b. Camera devices

Claims

1. Method for detecting disturbances of a crop collecting assembly (1), wherein a crop flow (6) at the crop collecting assembly (1) of an agricultural harvesting machine (2) is optically recorded, wherein the movement over time of an object (11a, b) in the crop flow (6) is determined in the optically recorded crop flow (6) and, on the basis of the determined movement of the object (11a, b), a disturbance of the crop collecting assembly (1) is detected, wherein, on the basis of the determined movement of the object, a speed property, preferably an average speed (16), of the crop flow (6) is ascertained, and wherein, on the basis of the ascertained speed property of the crop flow (6), a crop blockage is detected as a disturbance, wherein the determination of the movement of the object (11a, b) comprises a determination of a speed of the object (11a, b), preferably a determination of an absolute speed (12a-d) of the movement of the object (11a, b) and / or a determination of a direction of movement (13a-d) of the movement of the object (11a, b), characterized in that the determination of the movement of the object (11a, b) comprises a determination of a change in the speed of the object (11a, b) over time, preferably a determination of the change in the absolute speed (12a-d) over time and / or a determination of a change in the direction of movement (13a-d) over time, and in that, when a crop blockage is detected, a reduction of a throughput of the harvesting machine (2) is initiated.

2. Method according to Claim 1, characterized in that, for the determination of the movement of the object (11a, b), a multiplicity of photos (14a-c) of the crop flow (6) are generated one after the other in time.

3. Method according to either of Claims 1 or 2, characterized in that, for the determination of the movement of the object (11a, b), an optical identifying feature (15a, b) of the object (11a, b) is determined, preferably in that a respective position of the identifying feature (15a, b) is determined in the multiplicity of photos (14a-c).

4. Method according to Claim 3, characterized in that the object (11a, b) is found in the crop flow (6) on the basis of a located optical identifying feature (15a, b).

5. Method according to Claim 4, characterized in that, for establishing the object (11a, b), the recorded crop flow (6) is investigated, in particular substantially completely, on the basis of a criterion for locating optical identifying features.

6. Method according to one of Claims 2 to 5, characterized in that the determination of the movement of the object (11a, b) is based on a distance determination of the position respectively determined in the multiplicity of photos (14a-c), preferably in that, for the distance determination, a displacement of an image point assigned to the object (11a, b), in particular the identifying feature (15a, b), between the multiplicity of photos (14a-c) is determined.

7. Method according to one of Claims 3 to 6, characterized in that a multiplicity of objects (11a, b), in particular optical identifying features (15a, b), in the recorded crop flow (6) is determined, preferably in that the multiplicity of identifying features (15a, b) is determined on the basis of the criterion for locating optical identifying features, in particular in that the disturbance is also detected on the basis of a number of determined identifying features (15a, b).

8. Method according to Claim 7, characterized in that the disturbance is also detected on the basis of a change in the number of determined identifying features (15a, b) over time.

9. Method according to Claim 7 or 8, characterized in that, for the multiplicity of identifying features (15a, b), a respective movement over time is determined, preferably in that the disturbance is also detected on the basis of the movement respectively determined for the multiplicity of identifying features (15a, b), in particular in that the disturbance is also detected on the basis of a variance in the movement determined respectively for the multiplicity of identifying features (15a, b).

10. Method according to one of Claims 1 to 9, characterized in that the determination of the movement of the object (11a, b) includes the at least partial ascertainment of an optical flow of the recorded crop flow (6).

11. Method according to one of Claims 1 to 10, characterized in that, for the recording of the crop flow (6), a crop camera assembly (20) is arranged on the harvesting machine (2), in particular in that the crop camera assembly (20) is arranged for recording in the area between a cutting assembly and a cross-conveying assembly of the crop collecting assembly (1), preferably in that the crop camera assembly (20) has a multiplicity of camera devices (21a, b) for the substantially nonoverlapping recording of the crop flow (6).

12. Method according to Claim 11, characterized in that the crop camera assembly (20) is designed for monochromatic recording of the crop flow (6), preferably for monochromatic recording in the infrared range, in particular in that the crop camera assembly (20) has an image refresh rate of between 20 and 40 images per second, more particularly between 25 and 35 images per second.