PROCEDURE FOR OPERATING A SELF-PROPELLED AGRICULTURAL WORKING MACHINE

Description

[0001] The invention relates to a method for operating a self-propelled agricultural working machine according to the preamble of claim 1 and to such a self-propelled agricultural working machine according to claim 12.

[0002] The term "self-propelled agricultural machine" is to be understood broadly in this context. This includes not only harvesting machines such as combine harvesters and forage harvesters, but also tractors and similar vehicles.

[0003] When operating a self-propelled agricultural machine, sensor-based monitoring of the machine's surroundings is becoming increasingly important. Because the machine's surroundings are not available in a standardized, easily detectable form, but rather have a non-deterministic structure within certain limits, sensor-based detection of predetermined properties of the machine's surroundings presents a challenge. This applies particularly to the area in front of the agricultural machine, i.e., the area of ​​the field to be cultivated.

[0004] The known method (US Pat. No. 6,389,785), from which the invention is based, provides, in one variant, a sensor arrangement comprising a laser-based sensor system. The laser-based sensor system detects properties of the field, also referred to as field information, based on the sensor information from the sensor system. A driver assistance system of the agricultural machine then generates control actions based on the field information, which include, for example, executing adjustment movements of a front attachment for swath pickup (pick-up).

[0005] However, user influence on the control actions is limited or at least complex, as essential information is not readily available to the user, or at least not with the best possible accuracy. For example, it is not readily possible for the user to identify the position of the crop edge perpendicular to the direction of travel in the area immediately in front of the front attachment.

[0006] From US 2014 / 324272 A1 a method according to the preamble of claim 1 is known.

[0007] The invention is based on the problem of designing and developing the known method in such a way that properties of the area in front of the agricultural work machine can be made available to the user in the most user-friendly way possible.

[0008] The above problem is solved in a method according to the preamble of claim 1 by the features of the characterising part of claim 1.

[0009] The fundamental idea is to process the sensor information from the laser-based sensor system and graphically present it to the user in such a way that they can derive essential field information for the respective agricultural work order as directly as possible. To this end, the proposed method generates a display image or a display image sequence from the sensor information, representing a three-dimensional view of an area of ​​the field suitable for obtaining the said information. Such a display allows the user, from a comfortable position, to perceive essential field information early on and, if necessary, initiate corrective measures.

[0010] In particular, it is proposed that the driver assistance system has an image processing system for processing the sensor information and that the image processing system generates a display image or a display image sequence in the form of a three-dimensional visualization of the predetermined, relevant apron area of ​​the work machine on a display of the driver assistance system based on the sensor information of the laser-based sensor system.

[0011] A "display image sequence" is understood here to be a sequence of multiple display images corresponding to consecutive sensor information. "Three-dimensional visualization" means that the respective display image graphically represents the apron area of ​​the work machine detected or scanned by the sensor system in the three spatial dimensions of "length," "width," and "height." Thus, a three-dimensional image of the apron area is generated using graphic means on a two-dimensional surface, namely the display. The respective display image is supplemented, in particular, by a graphic representation of coordinate axes for the spatial dimensions of "length," "width," and / or "height."

[0012] According to the preferred embodiment according to claim 2, the sensor information of the laser-based sensor system comprises or is height information related to the field level. "Height information" refers to the data corresponding to the geometric height of the sensor-detected objects (plants, animals, persons, or the like). The respective height information can be provided or displayed as a linear unit, particularly in meters or centimeters, or as a percentage.

[0013] Claims 3 and 4 relate to particularly preferred possibilities for displaying different height information or heights in the display image or in the display image sequence. This is preferably achieved by color-differentiating different height information. In particular, for different height information, a difference in at least one color property such as hue, brightness, or saturation is provided. A specific hue, brightness, and / or saturation value or value range can indicate the external shape, in particular geometric height, of a specific sub-area of ​​the apron area and distinguish this from a sub-area with a different shape, in particular geometric height.For example, standing crops can be distinguished from harvested areas of the field by means of corresponding color differences, since these sub-areas of the apron area have different geometric heights and thus a different height contour is recorded by sensors for these sub-areas of the apron area.

[0014] According to the invention, crop edge detection is carried out based on the sensor information, i.e., a crop edge and its position are detected. The position of the detected crop edge is then graphically highlighted according to this embodiment. If the crop edge is a lateral crop edge, i.e., one that runs along the direction of travel, the position of the crop edge relative to the direction transverse to the direction of travel, i.e., the position in the width direction of the work machine or lane, is graphically highlighted. In principle, however, it is also conceivable to detect crop edges running transverse to the direction of travel using crop edge detection and, if necessary, also to highlight them graphically.Crop edge detection is carried out, for example, by comparing the sensor information of adjacent sub-areas and / or by comparing parts of the display image and / or the display image sequence that are assigned to adjacent sub-areas, wherein, for example, one sub-area is defined by standing crops and the adjacent sub-area is defined by a harvested field area.

[0015] Claim 5 defines different types of stand edges that can be determined by stand edge detection. In principle, multiple stand edges can be detected in the described manner and, in particular, highlighted graphically.

[0016] Claims 6 and 7 relate to the possibility of graphically highlighting the respectively detected stand edge by means of a line, in particular a straight line. Such a line represents the course of the stand edge. The user then preferably has the possibility of shifting the displayed line, i.e., the graphically highlighted position of the stand edge, in the display image or in the display image sequence, for example, as part of a corrective measure. In particular, the respective line can be shifted in a direction transverse to its course, i.e., a parallel offset can be performed between the previous position and the subsequent position of the line.

[0017] In order to be able to move such a line, the line in the display forms, in particular, a virtual slider (claim 8). Particularly preferably, the display is a touch display, i.e., a display that allows user input as well as moving the line (claim 9).

[0018] According to the further preferred embodiment according to claim 10, a section display can also be graphically shown on the display together with the display image or the display image sequence.

[0019] Claim 11 describes the possibility of generating at least one control action depending on the position of one or more crop edges graphically highlighted in the display image or in the display image sequence. The driver assistance system can preferably perform edge guidance. Such edge guidance can be carried out in particular by a lane planner of the driver assistance system, which aligns the course of the lane or lane with the detected crop edge as the agricultural machine travels along the field. By adjusting the line shown in the display, which is assigned to the respective crop edge, the lane or lane is then aligned with the adjusted line when the edge guidance is carried out, in particular by the lane planner.This adjustment of the line can then apply exclusively to the current lane or lane, or if necessary also to one or more other, preferably all other, lanes or lanes.

[0020] According to the invention, the maximum height and / or average height of the standing stock in the area in front of the work machine is also graphically highlighted in the display image or in the display image sequence. This can also be done using a corresponding line, in particular a straight line. The line then runs along the height contour of the standing stock visualized in the display image or in the display image sequence, i.e., the upper edge or surface formed by the stock in relation to the vertical direction.

[0021] According to a further teaching according to claim 12, which has independent significance, a self-propelled agricultural working machine, in particular a harvesting machine, is claimed as such for carrying out a method according to the proposal. Reference may be made to all embodiments suitable for describing the working machine as such.

[0022] In the following, the invention is explained in more detail with reference to a drawing which merely represents an exemplary embodiment. In the drawing, Fig. 1 a proposed self-propelled agricultural working machine in a side view, Fig. 2 the agricultural working machine according to Fig. 1 in a view from above and Fig. 3 the display of a driver assistance system of the agricultural machine according to Fig. 1 in a very schematic representation.

[0023] The proposed solution can be applied to a wide range of self-propelled agricultural machines. These include harvesting machines such as combine harvesters, forage harvesters, or the like, as well as tractors, in particular. In the illustrated and thus preferred embodiment, the agricultural machine 1 is a combine harvester, which, as shown in the Figu-ren 1 and 2 is in a harvesting process.

[0024] The proposed agricultural work machine 1 is equipped with at least one working element 2-8. The combine harvester here and preferably has a front attachment 2 with a cutting unit 3, a drive 4, a threshing unit 5, a separating device 6, a cleaning device 7, and a distribution device 8. The working elements 2-8 each serve to carry out or support agricultural work in a field.

[0025] The proposed agricultural work machine 1 is further equipped with a driver assistance system 9 for generating control actions within the work machine 1. The control actions can, on the one hand, relate to the control and parameterization of the working elements 2-8 and, for example, include steering movements of the agricultural work machine and / or height or transverse adjustments of working elements of the agricultural work machine, and, on the other hand, relate to the display of information for the user.

[0026] Furthermore, the driver assistance system 9 has a sensor arrangement 10 for generating apron information, wherein the driver assistance system 9 generates the control actions based on the apron information. The sensor arrangement 10 has a laser-based sensor system 11, here, for example, a sensor system based on a LIDAR sensor, and optionally an optional camera-based sensor system 12, both of which are arranged on the work machine 1, preferably in a position elevated relative to the field plane E. In any case, the laser-based sensor system 11, and optionally also the optional camera-based sensor system 12, each generates sensor information about a predetermined, relevant apron area 13 of the work machine 1.Preferably, the sensor arrangement 10 generates the apron information based on the sensor information of at least the laser-based sensor system 11, or the sensor information of at least the laser-based sensor system 11 forms at least part of the apron information. In the illustrated and thus preferred embodiment, the relevant apron area 13 is defined as an area of ​​predetermined shape and size that lies in front of the work machine 1 in the direction of travel F. In . Fig. 1 the apron area 13 is shown as a rectangular area in front of the work machine 1.

[0027] It is now essential that the driver assistance system 9 has an image processing system 14 for processing the sensor information and that the image processing system 14, based on the sensor information of the laser-based sensor system 11, generates a display image A or a display image sequence in the form of a three-dimensional visualization of the predetermined, relevant apron area 13 of the work machine 1 on a display 15 of the driver assistance system 9.

[0028] The sensor information of the laser-based sensor system 11 is thus processed by the image processing system 14, which is provided here by a computing device 16 of the driver assistance system 9, in such a way that a spatially effective image of the apron area 13 is generated on the display 15 using graphic means. The respective display image A, for example, also each display image A of the display image sequence, thus graphically represents the apron area 13 detected by the sensor system 11 in the three spatial dimensions "length", "width" and "height", as shown in Fig. 3 is shown schematically. In addition to the respective display image A, i.e. the three-dimensional visualization of the apron area 13, coordinate axes x, y, z for the spatial dimensions "length", "width", and "height" are shown on the display 15. The coordinate axis x is assigned to the spatial dimension "length". The coordinate axis y is assigned to the spatial dimension "width", and the coordinate axis z is assigned to the spatial dimension "height". The coordinate axes x, y, z, within which the display image A is shown and / or to which the display image A is aligned, have a measurement division in meters, for example.

[0029] The sensor information of the laser-based sensor system 11 here and preferably includes height information related to the field plane E or is height information related to the field plane E. The field plane E, i.e., the plane along which the agricultural work machine travels, is the reference surface to which the geometric height of the sensor-detected objects is related. The data corresponding to the geometric height is referred to as height information.

[0030] In order to be able to easily distinguish different heights of the objects, in particular plants or parts of plants, detected by the laser-based sensor system 11 in the apron area 13 in the display image A or the display image sequence and in particular to be able to determine the boundaries between adjacent sub-areas of the apron area 13 with different heights, it is here and preferably further the case that in the display image A or in the display image sequence different height information related to the field plane E, which is generated by the laser-based sensor system 11, is displayed in different colors by the image processing system 14.

[0031] Different colors mean that at least one color property such as hue, brightness, and saturation is different for different heights. In particular, a hue, brightness, and / or saturation value or value range is defined in the display image A or in the display image sequence by the existence of a specific external shape of a sub-area of ​​the apron area 13, since the external shape of the sub-area of ​​the apron area 13 depends on the geometric heights of the objects, in particular the plants or plant parts, in this sub-area. Thus, standing crop 17 fundamentally differs in its geometric height from a harvested area 18 of the field, which may have plant parts in the form of stubble.As a result, in the display image A or in the display image sequence, the three-dimensional visualization of the standing crop 17 on the display 15 has, for example, a different color tone, a different brightness and / or a different saturation than the three-dimensional visualization of the harvested area 18 of the field.

[0032] In particular, it is conceivable that in the display image A or in the display image sequence a hue, brightness and / or saturation value or value range is defined by the existence of a standing crop 17 and / or a hue, brightness and / or saturation value or value range is defined by the existence of a harvested area 18 of the field and / or a hue, brightness and / or saturation value or value range is defined by the existence of a lying crop and / or a hue, brightness and / or saturation value or value range is defined by the existence of a driving track 19 and / or a hue, brightness and / or saturation value or value range is defined by the existence of a spray track and / or a hue, brightness and / or saturation value or value range is defined by the existence of a headland and / or a hue, brightness and / or saturation value or value range is defined by the existence of an area 20 adjacent to the field and / or a color,Brightness and / or saturation value or range of values ​​is defined by the existence of an obstacle.

[0033] How Fig. 3 shows, here the image processing system 14 further performs a stand edge detection based on the sensor information of the laser-based sensor system 11 and / or based on the display image A or the display image sequence. In this case, for example, the sensor information assigned to adjacent sub-areas of the apron area 13 is compared with one another. Additionally or alternatively, parts of the display image A and / or the display image sequence that are assigned to different sub-areas of the apron area 13 can also be compared with one another. In particular, in this context, hue, brightness and / or saturation values ​​or value ranges assigned to the corresponding sub-areas can be compared with one another. The existence of a stand edge 21 can then be inferred from the differences determined during this comparison.In the display image A or in the display image sequence, the position of a stand edge 21 detected during the stand edge detection is then graphically highlighted, as shown in . Fig. 3 is shown. Since the present embodiment involves a lateral crop edge 21, i.e. one running along the direction of travel F, the position relative to the direction transverse to the direction of travel F is graphically highlighted in the display image A or in the display image sequence.

[0034] As part of the crop edge detection, the image processing system 14 also graphically highlights the position of different crop edges 21, particularly in relation to the direction transverse to the direction of travel F. Various crop edges 21 are shown as examples in Fig. 2 For example, the boundary between the standing crop 17 and a harvested area 18 of the field can be recognized and, in particular, graphically highlighted as the crop edge 21. This is also conceivable for the boundary between the standing crop 17 and the area 20 adjacent to the field, with the crop edge 21 here being the field boundary. This is also conceivable for the boundary between the standing crop and a headland, with the crop edge 21 then being the headland edge. This is also conceivable for the boundary between the standing crop 17 and a lying crop and / or for the boundary between the standing crop 17 and a spray track, i.e., a driving track in the standing crop 17. The respective boundary therefore defines the crop edge 21.

[0035] How Fig. 3 shows, in the exemplary embodiment shown here and preferred in this respect, the position of the respective crop edge 21 in relation to the direction transverse to the direction of travel is graphically highlighted by the image processing system 14 by a line 22, in particular a straight line 22, which runs along the crop edge 21 visualized in the display image A or in the display image sequence. The crop edge 21 here is a lateral crop edge 21, specifically between the standing crop 17 and a harvested area 18 of the field. In principle, however, any other of the aforementioned crop edges 21 can also be graphically highlighted by a line 22, in particular a straight line 22, in the manner described.

[0036] In a particularly preferred embodiment, the respective line 22, in particular the line 22 associated with the lateral edge 21, is movable by the user in the display image A or in the display image sequence. The line 22 here and preferably forms a virtual slider, which can be moved by the user (in Fig. 3 (shown in dashed lines) in the display image A or in the display image sequence. For this purpose, the display 15 is preferably designed as a touch display 15.

[0037] By moving line 22, which is Fig. 3 The driver assistance system 9 and the control actions to be generated by it can be individually adapted by means of the arrows shown in the figure. For example, if during a harvesting operation the working width of the front attachment 2 is to extend slightly beyond the standing crop 17 and slightly overlap with the harvested area 18 of the field, the line 22 can be diverted from the Fig. 3 shown position, which is located on the existing edge 21, should be shifted slightly to the left. The driver assistance system 9 will then orient itself to the new position of line 22 during further processing.

[0038] Here, and preferably, a section display 25 can also be graphically displayed on the display 15 together with the display image A or the display image sequence. The graphically displayed section display 25 can overlap with the display image A or the display image sequence. The section display 25 provides the user with another option for checking the work result.

[0039] The driver assistance system 9 here and preferably generates at least one control action depending on the position of one or more crop edges 21 graphically highlighted in the display image A or in the display image sequence, in particular depending on the position of the respective line 22 in the display image A or in the display image sequence. Particularly preferably, the driver assistance system 9 is capable of performing edge guidance, in which the driver assistance system 9 orients itself to the graphically highlighted crop edge(s) 21, in particular to the position of the respective line 22, to generate the control action. For example, the control action can comprise the execution of steering movements of the agricultural working machine 1 and / or the transverse adjustment of a working element, in particular front attachment 2, of the agricultural working machine 1. This is advantageous, for example, for the described case in which during a harvesting process, as described in the Figuren 2 and 3 shown, the working width of the front attachment 2 should extend laterally beyond the standing crop 17. For this purpose, the line 22 can be extended as described from the Fig. 3 shown position slightly to the left, whereupon the driver assistance system 9 then orientates itself to the new position of line 22.

[0040] Furthermore, it is provided that the maximum height and / or the average height of the standing stock 17 in the predetermined, relevant apron area 13 is graphically highlighted in the display image A or in the display image sequence. This is done here and preferably by a further line 23, 24, in particular a straight line 23, 24, which runs along the height contour of the standing stock 17 visualized in the display image A or in the display image sequence. Line 23 indicates the maximum height, and line 24 indicates the average height.

[0041] Fig. 3 Finally, it also shows that the display 15 can represent a plurality of display areas 26, 27, wherein here and preferably in a first display area 26 the display image A or the display image sequence and in particular the coordinate axes x, y, z are displayed, and wherein in a second display area 27 virtual operating elements 28 for making settings of the driver assistance system 9, in particular the sensor arrangement 10, the image processing system 14, the display 15 and / or the computing device 16, are displayed. Furthermore, a scale 29 is provided in the first display area 26, in which a value for the geometric height is assigned to certain color tones that are or can be used in the respective display image A. The scale 29 here has, for example, a graduation in centimeters.

[0042] According to a further teaching of independent significance, the self-propelled agricultural machine 1 is claimed as such for carrying out a method according to the proposal described above. Reference is made to all relevant statements. List of reference symbols

[0043] 1Working machine 2-8Working elements 9Driver assistance system 10Sensor arrangement 11Laser-based sensor system 12Camera-based sensor system 13Relevant apron area 14Image processing system 15Display 16Calculating device 17Standing crop 18Harvested area of ​​the field 19Track 20Area adjacent to the field 21Crop edge 22-24Lines 25Section display 26, 27Display areas 28Virtual control elements 29Scale ADisplay image EField level FDirection of travel x, y, zCoordinate axes

Claims

1. A method for operating a self-propelled agricultural working machine (1), in particular a harvesting machine, wherein the agricultural working machine (1) has at least one working unit (2-8) for carrying out or supporting agricultural work in a field, wherein the agricultural working machine (1) has a driver assistance system (9) for generating control actions within the working machine (1), wherein the driver assistance system (9) has a sensor assembly (10) for generating frontal field information, wherein the driver assistance system (9) generates the control actions on the basis of the frontal field information, wherein the sensor assembly (10) has a laser-based sensor system (11) which generates sensor information in respect of a predetermined, relevant frontal field region (13) of the working machine (1), wherein the driver assistance system (9) has an image processing system (14) for processing the sensor information, wherein, on the basis of the sensor information from the laser-based sensor system (11), a displayed image (A) or a displayed image sequence is generated by the image processing system (14) on a display (15) of the driver assistance system (9) in the form of a three-dimensional visualisation of the predetermined, relevant frontal field region (13) of the working machine (1), characterized in that a field crop edge detection is carried out by the image processing system (14) on the basis of the sensor information from the laser-based sensor system (11) and / or on the basis of the displayed image (A) or the displayed image sequence, and in that the position of a field crop edge (21) detected in the context of the field crop edge detection (21) is graphically accentuated in the displayed image (A) or in the displayed image sequence, in particular with respect to the direction transverse to the direction of travel (F), and in that the maximum height and / or the mean height of the standing field crop (17) in the predetermined, relevant frontal field region (13) of the working machine (1) is graphically accentuated in the displayed image (A) or in the displayed image sequence, preferably in that the mean height and / or the maximum height of the standing field crop (17) is graphically accentuated by a line (23, 24), in particular a straight line (23, 24), which extends along the height contour of the standing field crop (17) visualised in the displayed image (A) or in the displayed image sequence.

2. The method according to claim 1, characterized in that the sensor information from the laser-based sensor system comprises, or is, height information with respect to the plane of the field (E).

3. The method according to claim 1 or claim 2, characterized in that different height information with respect to the plane of the field (E) which is generated by the laser-based sensor system (11) is depicted in different colours in the displayed image (A) or in the displayed image sequence.

4. The method according to one of the preceding claims, characterized in that in the displayed image (A) or in the displayed image sequence, - a colour value, brightness value and / or saturation value or range of values is defined by the existence of standing field crop (17), and / or - a colour value, brightness value and / or saturation value or range of values is defined by the existence of a harvested region (18) of the field, and / or - a colour value, brightness value and / or saturation value or range of values is defined by the existence of lying field crop, and / or - a colour value, brightness value and / or saturation value or range of values is defined by the existence of a drive track (19), and / or - a colour value, brightness value and / or saturation value or range of values is defined by the existence of a spray track, and / or - a colour value, brightness value and / or saturation value or range of values is defined by the existence of a headland, and / or - a colour value, brightness value and / or saturation value or range of values is defined by the existence of a region (20) bordering the field, and / or - a colour value, brightness value and / or saturation value or range of values is defined by the existence of an obstacle.

5. The method according to one of the preceding claims, characterized in that in the context of the field crop edge detection, the position with respect to the direction transverse to the direction of travel (F): - of the boundary between the standing field crop (17) and a harvested region (18) of the field, and / or - of the boundary between the standing field crop (17) and the region (20) adjoining the field, and / or - of the boundary between the standing field crop (17) and the headland, and / or - of the boundary between the standing field crop (17) and a lying field crop, and / or - of the boundary between the standing field crop (17) and a spray track, is detected and in particular graphically accentuated by the image processing system (14).

6. The method according to one of the preceding claims, characterized in that, with respect to the direction transverse to the direction of travel (F), the position of the respective field crop edge (21) is graphically accentuated by the image processing system (14) by a line (22), in particular a straight line (22), which extends along the field crop edge (21) visualised in the displayed image (A) or in the displayed image sequence.

7. The method according to claim 6, characterized in that the respective line (22) can be displaced in the displayed image (A) or in the displayed image sequence by the user.

8. The method according to claim 6 or claim 7, characterized in that the line (22) forms a virtual slider.

9. The method according to one of the preceding claims, characterized in that the display (15) is a touch display (15).

10. The method according to one of the preceding claims, characterized in that a part-width indicator (25) can be, or is, graphically depicted on the display (15) together with the displayed image (A) or the displayed image sequence, preferably in that in the display (15), the graphically depicted part-width indicator (25) overlaps the displayed image (A) or the displayed image sequence.

11. The method according to one of the preceding claims, characterized in that the driver assistance system (9) generates at least one control action as a function of the position of one or more field crop edges (21) graphically accentuated in the displayed image (A) or in the displayed image sequence, in particular as a function of the position of the respective line (22) in the displayed image (A) or in the displayed image sequence, preferably in that the driver assistance system (9) executes an edge guidance, and / or in that the control action comprises carrying out steering movements of the agricultural working machine (1) and / or the transverse adjustment of a working unit, in particular a front attachment (2), of the agricultural working machine (1).

12. A self-propelled agricultural working machine, wherein the agricultural working machine (1) has at least one working unit (2-8) for carrying out or supporting agricultural work in a field, wherein the agricultural working machine (1) has a driver assistance system (9) for generating control actions within the working machine (1), wherein the driver assistance system (9) has a sensor assembly (10) for generating frontal field information, wherein the driver assistance system (9) generates the control actions on the basis of the frontal field information, wherein the sensor assembly (10) has a laser-based sensor system (11) which generates sensor information in respect of a predetermined, relevant frontal field region (13) of the working machine (1), wherein the agricultural working machine (1) is provided and configured for carrying out a method according to claim 1.