System and method for material transport with one or more imaging devices on the transferring vehicle and on the receiving vehicle for controlling the material distribution in the loading space of the receiving vehicle

Abstract

A method for enabling the transport of material from a transferring vehicle (91) having a discharge chute (89) with a material conveying arm or an end (87) of a discharge chute (89) into a receiving vehicle (79) having a container (81) for storing transported material, the method comprising the following steps: a. capturing image data from at least one imaging device (10) directed at the container (81); b. identifying and locating the container (81) from the acquired image data using a chute locator (22) of an image processing module (18); c. identifying the discharge chute (89) from the acquired image data using the discharge chute locator (22) of the image processing module (18); d. detecting a representation of the fill level or volume distribution of material in the container (81); e. determining a position of the discharge chute (89) or the end (87) of the discharge chute (89) relative to the container (81) based on the identified location of the container (81) and the identified discharge chute (89); f. aligning the material conveying arm or the end (87) of the discharge chute (89) over a current target area of ​​the material-receiving container (81) based on the determined position of the discharge chute (89) or the chute end (87) relative to the container (81); g. Determining further target areas of the container (81) for receiving material, based on the representation of the fill level or the volume distribution of material in the container (81); h. Transport of material from the transferring vehicle (91) to the current destination area of ​​the container (81) in the receiving vehicle (79); i. Detecting the time at which the current target area of ​​the container (81) is filled with material; j. Repeating steps f to i until the subsequent target areas of the container (81) are filled; and k. Completion of the transport of material from the transferring vehicle (91) to the receiving vehicle (79).

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is an international application claiming priority to U.S. Provisional Application 61 / 597,346, filed on February 10, 2012, U.S. Provisional Application 61 / 597,374, filed on February 10, 2012, and U.S. Provisional Application 61 / 597,380, filed on February 10, 2012. All of the foregoing are incorporated herein by reference. JOINT RESEARCH AGREEMENT

[0002] This application arises out of activities conducted under or in connection with a joint research agreement between Carnegie Mellon University and Deere & Company, entitled "Development Agreement between Deere & Company and Carnegie Mellon University," dated January 1, 2008, and accordingly claims the benefits available under 35 USC § 103(c). FIELD OF THE INVENTION

[0003] The present invention relates to a method and a stereo-optical system for enabling the unloading of material from a vehicle. BACKGROUND

[0004] Certain prior art systems may attempt to use GPS receivers for the purpose of maintaining a suitable distance between two vehicles during unloading or during the transfer of agricultural or other material, e.g., coal or other raw materials, from one vehicle to the other. However, these prior art systems are subject to errors or gaps in the positioning provided by the GPS receivers. For example, one or more of the GPS receivers may be subject to misdetermination of their position due to, among other things, electromagnetic interference, multipath propagation of the received satellite signals, intermittent reception of the satellite signals, or low received signal strength of the satellite signals. If the vehicles utilize cameras or other imaging devices in a remote work area, such asIn an agricultural field, the imaging devices may be temporarily exposed to sunlight, shadows, dust, flare, or other lighting conditions that may temporarily disrupt the proper operation of the imaging devices, potentially leading to errors in determining the distance to objects monitored by the imaging devices. Therefore, an improved system for implementing the unloading of agricultural material from a vehicle is needed to compensate for or appropriately handle errors in estimating the position of the vehicles or in the orientation of the vehicles relative to each other.US 2009 / 0 044 505 A1 discloses an agricultural work machine, in particular a forage harvester, with at least one discharge spout for conveying collected and processed crop material to a transport vehicle. An electro-optical device is provided for controlling the direction of the discharge spout, at least during the conveying process to the transport vehicle. The electro-optical device detects characteristic parameters of the discharge spout as well as characteristic parameters of the transport vehicle and / or the agricultural work machine. EP 2 311 307 A1 relates to a fill level measuring device for measuring and displaying a residual fill potential of a target area, for example a container, which is filled with goods through a target, for example an open top, by means of a goods carrier, for example a discharge pipe, of an agricultural vehicle, for example a harvester.The level measuring device comprises a 3D sensor for observing at least a portion of the target area, which includes at least a portion of the target, a data control system, and a visual display unit. The data control system serves to provide a visual display on the optical display unit indicating the residual fill potential by displaying the current one of at least three different levels of the residual fill potential. DE 10 2007 009 666 A1 relates to an arrangement for filling a container with bulk material, comprising a filling device with an outflow opening for the bulk material and a container with a filling opening through which the bulk material enters the container.US 2011 / 0 307 149 A1 discloses a grain transfer control system for automatically controlling the relative positions of a discharge nozzle of a work machine's discharge system and a receiving bin based on a real-time model of a fill level profile for regions of the receiving bin, wherein the profile is modeled using known or estimated grain flow rates and positions of the nozzle relative to regions of the receiving bin as a function of time, and the system adjusts the relative positions of the nozzle and the receiving bin to achieve a generally uniform filling of the receiving bin. US 2010 / 0 042 248 A1 relates to a system for on-vehicle monitoring and control of the loading of goods such as grain or loose or bulk materials.A method for transferring liquid materials from a storage container into a transport container using a loading system comprises a control device for the loading system and a user interface device capable of mutual wireless communication. WO 2011 / 101 458 A1 relates to a method for steering a movable unloading device of an agricultural harvesting machine to a container driven alongside the harvesting machine, which container has a near and a far upper edge, as seen from the harvesting machine. A 3D camera captures an image of the container in the form of a pixel array, wherein the camera also generates a value for the distance between the camera and the object in the image for each pixel.For image analysis, a series of vertical stripes are selected in the image, the filtered distance values ​​for each vertical position along the stripes are determined, and the positions of the near and far upper limits are determined from these distance values. Based on this data, the unloading device is moved toward a predetermined position relative to the container, wherein the predetermined relative position is a position in which the unloading device directs the crop to a position above the near upper limit or below the far upper limit. US 2010 / 0332 051 A1 relates to a control arrangement for controlling the transfer of agricultural crop from a harvesting machine to a transport vehicle, wherein the transport vehicle comprises a loading container.The control device detects the fill level and / or the outer contour of the loading container based on signals from a sensor device and automatically controls the position of the outlet end of a discharge device in relation to the harvesting machine and / or the ejection direction of the discharge device and / or the position of the transport vehicle with the loading container in relation to the harvesting machine in such a way that the loading container is successively filled with crop material.US 2011 / 0 307 149 A1 relates to a grain transfer control system for automatically controlling the relative positions of a discharge nozzle of a work machine discharge system and a receiving bin based on a real-time model of a fill level profile for regions of the receiving bin, wherein the profile is modeled using known or estimated flow rates of the grain and positions of the nozzle relative to regions of the receiving bin as a function of time, and the system adjusts the relative positions of the nozzle and the receiving bin to achieve a generally uniform filling of the receiving bin.US 8 380 401 B2 relates to a grain transfer control system for automatically controlling the relative positions of a discharge nozzle of a work machine discharge system and a receiving hopper based on a real-time model of a fill level profile for portions of the receiving hopper, wherein the profile is modeled using known or estimated grain flow rates and positions of the nozzle relative to portions of the receiving hopper as a function of time, and the system adjusts the relative positions of the nozzle and receiving hopper to achieve generally uniform filling of the receiving hopper. US 8 060 283 B2 discloses a lead vehicle location receiver that determines the location of the lead vehicle. SUMMARY OF THE INVENTION

[0005] The system and method enables the transfer of agricultural material from a transferring vehicle (e.g., a harvester) to a receiving vehicle (e.g., a grain trailer). The system and method includes a receiving vehicle, which includes a drive unit for driving the receiving vehicle and a cargo space for storing agricultural material, as well as a transferring vehicle for depositing harvested agricultural material in the cargo space of the receiving vehicle.

[0006] Two embodiments of the present invention include one or two imaging devices mounted exclusively on the transferring vehicle, either a combine harvester or a self-propelled forage harvester. In a first embodiment, one or two primary imaging devices are mounted on the combine harvester (a transferring vehicle), and no imaging device is mounted on the receiving vehicle. In a second embodiment, one or two imaging devices are mounted on a self-propelled forage harvester—also a transferring vehicle—and no imaging device is mounted on the receiving vehicle.

[0007] Embodiments of the present invention include a first imaging device mounted at a first location on the receiving vehicle, facing the cargo area thereof. The first imaging device acquires a first set of image data. A second imaging device is connected to a second location (e.g., mounted or movably attached there) on the transferring vehicle and faces the cargo area of ​​the receiving vehicle. The second imaging device acquires a second set of image data.

[0008] The transferring vehicle according to one of the above-mentioned embodiments may comprise, as an image processing module, a container identification module that can identify a container or the perimeter of a container of the cargo space in at least one of the first acquired image data sets and the second acquired image data sets (if a second image processing device has been included in the system configuration). The image processing may also comprise a discharge chute location device that serves to identify a discharge chute of the transferring vehicle in the acquired image data (the first acquired image data sets, the second acquired image data sets, or both).The image processing module may include an image data evaluation function that determines whether to use the first image data, the second image data, or both (if a second image processing device has been included in the system configuration) based on the evaluation of significant changes in pixel data intensity or significant changes in ambient light conditions during a given sampling interval. In a system with only a single image processing device, the image data evaluation function is either disabled, not included in the system, or operates with logic that passes the single captured image to the next function.The image processing module may also include an alignment module for determining the relative position of the chute to the container perimeter and generating command data for the transferring vehicle's control controller to control the transferring vehicle in concert with the receiving vehicle such that the chute is aligned within a central region (or other target region) of the container perimeter.

[0009] In operation, this is a method for enabling the transfer of material from a transferring vehicle having a chute with a container for storing the transferred material to a receiving vehicle having a container for storing the transferred material, the method comprising the following steps: a. acquiring image data from at least one imaging device directed at the container; b. Identifying and locating the container from the acquired image data using a chute locator of an image processing module; c. Identifying the chute from the acquired image data using the image processing module's chute locator; d. Detection of a representation of the fill level or volume distribution of the material in the container; e. Determining a position of the discharge chute or the end of the discharge chute relative to the container based on the identified location of the container and the identified discharge chute; f. Aligning the material conveying arm or the end of the discharge chute over a current target area of ​​the material receiving container based on the determined position of the discharge chute or the end of the chute relative to the container; g. Determination of further target areas of the material-receiving container based on the representation of the fill level or volume distribution of the material in the container h. Transport of the material from the transferring vehicle to the current destination area of ​​the container in the receiving vehicle; i. Detection of the time at which the current target area of ​​the container is filled with the material: j. Repeating steps f to i until the subsequent target areas of the container are filled; and k. Completion of the transport of the material from the transferring vehicle to the receiving vehicle. BRIEF DESCRIPTION OF THE DRAWINGS At Fig. 1 is a block diagram of an embodiment of a guidance system for a transferring vehicle, namely a combine harvester, enhanced with machine vision to enable the unloading of agricultural material from the transferring vehicle (e.g. a combine harvester); At Fig. Figure 2 is a block diagram of another embodiment of a guidance system enhanced with machine vision for a transferring vehicle, namely a self-propelled forage harvester, to enable the unloading of agricultural material from the transferring vehicle; At Fig. Figure 3 is a plan view of an imaging device mounted on a transferring vehicle and directed towards the receiving vehicle; At Fig. 4A is a plan view of imaging equipment (e.g., a monocular or stereo-optical system) mounted on a transferring vehicle and directed toward a cargo area of ​​the receiving vehicle; At Fig. 4B is a plan view of imaging devices (e.g., a monocular or stereo-optical system) mounted on a receiving vehicle and a transferring vehicle, facing a cargo area of ​​a receiving vehicle; At Fig. 4C is a view along a horizontal plane, corresponding to the reference line 4C-4C in Fig. 4B; Fig. 4D illustrates with a two-dimensional representation various possible distributions of material inside a container or a cargo space, corresponding to a cross-section along the reference line 4D-4D in Fig. 4B; At Fig. 4E is a plan view of a transferring vehicle and a receiving vehicle, with the transferring vehicle oriented according to a matrix of possible offset positions; Fig. 5A shows a block diagram of a process for identifying containers using error-corrected image data; Fig. Figure 5B shows a block diagram of a container identification process capable of using error-corrected image data as well as disparity images; Fig. 6A shows a block diagram of a process for locating a discharge chute using corrected image data and chute position data; Fig. 6B shows a block diagram of a process for locating a discharge chute using corrected image data, disparity images, and chute position data; Fig. 7 shows a flow diagram of a method for operating a machine vision-enhanced guidance system for facilitating the unloading of agricultural material from a transferring vehicle; and Fig. Figure 8 is a schematic diagram illustrating the data flow and processing by the image processing module, from the raw images to the vehicle commands. DESCRIPTION OF THE PREFERRED EMBODIMENT

[0010] According to an embodiment of the present invention requiring imaging devices located in the transferring vehicle, the Fig. 1 and Fig. 2 guidance systems 11, 111 enhanced by machine vision for a transferring vehicle 91 ( Fig. 3) for the technical implementation of the unloading of agricultural material (e.g. grain) from the transferring vehicle 91 ( Fig. 1 - combine harvester; Fig. 2 - self-propelled forage harvester) into a receiving vehicle 79 (e.g., grain trailer or bunker wagon). The transferring vehicle 91 may include other vehicles, such as a combine harvester or other heavy equipment for picking up material and depositing it into the receiving vehicle. For example, a stereo image reproduction system may function as an extension of a satellite navigation receiver or location-determining receiver 42 for the route guidance of the transferring vehicle 91. If one now returns Fig. 3, the first imaging device 10 has a first field of view 77, recognizable by the dashed lines. The second imaging device 12 has a second field of view 177, recognizable by the dashed lines. The boundaries of the fields of view 77, 177 are shown for illustrative purposes only and vary in practice. The systems 11, 111 may include a first imaging device 10 and a second imaging device 12 coupled to an image processing module 18 ( Fig. 1 and Fig. 2). Embodiments of a first imaging device 10 may optionally include a primary stereo camera or a monocular camera, while the second imaging device 12 may include a secondary stereo camera or a monocular camera. In one configuration, the second imaging device 12 is a stereo camera, which may be optional and provides redundancy for the first imaging device 10 in the event of a failure or malfunction, or in the absence of image data from the first imaging device 10, when the first field of view 77 of the first imaging device 10 is sufficient to view the container 85. In one configuration, the second imaging device is monocular and is required for a stereo image of the container 85 when used in conjunction with an image from a monocular first imaging device 10, wherein the first field of view 77 of the first imaging device 10 is sufficient to view the container 85. Fig. 3 is a top view of a transferring vehicle 91 and a receiving vehicle 79. As Fig. 3, the transferring vehicle 91 is embodied as a combine harvester with a harvesting header 185, while the receiving vehicle 79 is depicted as a tractor with a grain trailer. More generally, the receiving vehicle 79 comprises the combination of a drive unit and a storage unit 93 (e.g., a trailer-type storage unit attached to a drive unit). Each imaging device 10, 12 includes an image cleaner 101 to convert the raw image into a degraded image. Although the transported material disclosed in this specification is agricultural material, the invention is not to be understood as being limited to agricultural material; it may also be used with other materials such as coal and other raw materials.

[0011] Fig. 3 shows a first imaging device 10 located in the transferring vehicle 91 (e.g., a combine harvester) and a second imaging device 12 on a discharge chute 89 of the transferring vehicle 91. The second imaging device 12 may be optional if the first imaging device 10 is a stereo camera and the first field of view 77 of the first imaging device 10 is sufficient to view the container 85. The discharge chute 89 may also be referred to as a discharge auger. The end of the discharge chute 89 may be referred to as a discharge tube. Fig. 3, the discharge chute 89 or the end of the discharge chute 87 is generally aligned over a central area 83, a central region or a target area which defines the grid pattern 82 ( Fig. 4A and Fig. 4B) of the hopper 85 of the receiving vehicle 79 for unloading material from the transferring vehicle 91 into the receiving vehicle 79. Similarly, the transferring vehicle 91 and the receiving vehicle 79 are aligned with each other as shown, regardless of whether the vehicles are traveling forward together in a typical manner during harvesting (e.g., with coordinated or track-based travel) or are stationary. During unloading, the master controller 59 ( Fig. 1 and Fig. 2) maintaining an overall uniform spatial offset (e.g., a predominantly static offset that varies between vehicles 91, 79 only within specified target tolerances), possibly with the necessary incremental change in the offset to uniformly fill the container 85. The master controller 59 supports maintaining a uniform front / rear offset (Φ or φ) as well as a lateral offset (Δ).

[0012] Now to the subject of Fig. 1, Fig. 2, and Fig. 3: The transferring vehicle 91 may be equipped with a chute rotation sensor 116, which serves to measure the rotation angle of the chute 89. For an imaging device 12 mounted on the chute, the rotation angle of the chute 89 may be used to enable the merging of image data from the first imaging device 10 and the second imaging device 12, or to generate stereo image data, with the first imaging device 10 and the second imaging device 12 each providing monocular image data for the same scene or object.

[0013] In any arrangement of imaging devices 10, 12 disclosed in this document, in which the fields of view 77, 177 are superimposed, merging image data from a first imaging device 10 and a second imaging device 12 enables the image processing module 18 to generate a virtual profile of the material distribution level ( Fig. 4D) within the loading space 85, even if the entire surface of the agricultural material is not visible to one of the two imaging devices 10, 12. Even if the second imaging device 12 is not mounted on the discharge chute 89 in certain arrangements, the discharge chute rotation sensor 116 can enable the use of the end of the discharge chute 87 as a reference point in acquired image data (e.g., for the purpose of data merging, virtual concatenation, or synchronization of image data from different imaging devices). The virtual profile of the entire surface of the agricultural material in the loading space 93 enables the systems 11, 111 or the image processing module 18 to intelligently implement a filling strategy for the loading space 93 of the receiving vehicle 79.

[0014] The first imaging device 10 and the second imaging device 12 may output a digital data format as stereo video image data or as a series of stereo still images at regular or periodic intervals or other sampling intervals. Each stereo image (e.g., the first image data or the second image data) consists of two sub-images of the same scene or a portion of the same scene. For example, the first imaging device 10 has a first field of view 77 of the cargo area 93 of the receiving vehicle 79 at the location, the first field of view 77 at least partially overlapping with a second field of view 177 of the second imaging device 12 (if present). In one embodiment, the first imaging device 10 or the second imaging device 12, or both, may comprise a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS) array, or other suitable device for detecting or acquiring image data.

[0015] In one arrangement, an optical sensor 110, 112 comprises a light meter, a photosensor, a photoresistor, a photosensitive device, or a cadmium sulfide cell. A first optical sensor 110 may be connected to the first imaging device 10; a second optical sensor 112 may be connected to the second imaging device 12. The first optical sensor 110 and the second optical sensor 112 may each be coupled to the image processing module 18. The optical sensor 110, 112 provides a reading or level representative of the ambient light intensity in the field of view of the respective associated imaging devices 10, 12.

[0016] The image processing module 18 can be connected either directly or indirectly to lighting means 14 ( Fig. 1) in a transferring vehicle 91 for illuminating a storage container 85 and / or a discharge chute 89. The image processing module 18 may, for example, contain a light controller 50 ( Fig. 2), which includes control drivers, relays, or switches, which in turn control the switching on or off of lights 14 on the transferring vehicle 91. The image processing module 18 can also switch on the lights 14, 52 on the transferring vehicle to control the storage container 85 ( Fig. 3A) and the discharge chute 89, or both, when an optical sensor 110, 112 or light meter indicates that the ambient brightness falls below a certain minimum limit. In one arrangement, the optical sensors 110, 112 are aligned in the same direction as the lens or aperture of the imaging devices 10, 12.

[0017] In the combine harvester version ( Fig. 1) the vehicle controller 46 controls the discharge chute 89, which has a rotation sensor 116 for detecting a rotation angle of the discharge chute (a) in Fig. 5A and (β) in Fig. 5C, the discharge chute 89 comprises a rotational axis relative to one or more axes of rotation, as well as a rotary actuator 216 for moving the discharge chute 89 to change the angle of rotation of the discharge chute. Thus, the position of the discharge chute 89 relative to the receiving vehicle 79 or the associated storage container 85 is changed. The rotary actuator 216 may comprise a motor, a linear motor, an electro-hydraulic device, a ratchet-based or cable-operated mechanical device, or another device for moving the discharge chute 89 or the end of the discharge chute 87. The angle of rotation of the discharge chute may comprise a simple, cascading, or multi-dimensional angle, measured with respect to a reference axis parallel to the direction of travel of the transferring vehicle.

[0018] If the rotary actuator 216 is an electro-hydraulic device, the use of proportional control valves in the hydraulic cylinder of the electro-hydraulic device that rotates the discharge chute (or changes the angle of rotation of the discharge chute) allows for greater fine adjustment of the angle of the discharge chute (e.g., "a") than would be possible under other circumstances. Accordingly, proportional control valves of the electro-hydraulic device assist the rotary actuator 216 in ensuring a uniform profile or distribution of the discharged agricultural material within the loading space 93 or the container 85.Many commercially available combine harvesters are typically equipped with non-proportional control valves for controlling the angle of the discharge chute or the movement of the discharge chute 89; electro-hydraulic devices with non-proportional control valves may fill the hopper by means of an inefficient multimodal or hill-shaped distribution (e.g. 508) of the agricultural material, with local peaks and local troughs, as for example in . Fig. 4D shown.

[0019] A vehicle controller 46 may be coupled to the data bus of the vehicle 60 to provide a data message indicating when the drive of the auger 47 for discharging agricultural material from the transferring vehicle is turned on or off. The drive of the auger 47 may include an auger, an electric motor for driving the auger, and a rotation sensor for detecting the rotational movement or speed of the auger or associated axle. In one embodiment, a container for storing agricultural material from a transferring vehicle 91 (e.g., a grain tank) is associated with the auger (not shown). If the vehicle controller 46 (e.g.,If the auger control unit indicates that the auger of the transferring vehicle 91 is rotating or operating, the image processing module 18 activates the chute locating device 22 and the container identification module 20. The vehicle controller 46 can thus conserve data processing resources or keep energy consumption low by inactivating (or placing in standby mode) the container identification module 20 and the chute identification module 22 while the transferring vehicle 91 harvests the agricultural material but does not deposit it into the receiving vehicle 79.

[0020] In Fig. 1, the image processing module 18 or other control device may include a controller, a microcomputer, a microprocessor, a microcontroller, an application-specific integrated circuit, a programmable logic array, a logic device, an arithmetic logic unit, a digital signal processor, or other data processor, as well as associated electronic hardware and software. In one embodiment, the image processing module 18 may include a disparity generator 103, a container identification module 20, a chute locator 22, an alignment module 24, a material profile module 27, and a vehicle module 1000.

[0021] The image processing module 18 can be connected to a data storage device and can include, for example, an electronic memory, a non-volatile random access memory, a magnetic disk drive, an optical drive, a magnetic storage device, or an optical storage device. If the container identification module 20, the chute locator 22, the alignment module 24, the material profile module 27, and the vehicle module 1000 are software modules, they are stored in the data storage device.

[0022] The container identification module 20 detects a set of two- or three-dimensional points (e.g., in the form of Cartesian coordinates or polar coordinates) in the acquired image data or in the actual environment, these points defining at least a portion of the perimeter of the container 81 of the cargo space 85 ( Fig. 3). The set of two- or three-dimensional points corresponds to pixel positions in the images collected by the first imaging device 10, the second imaging device 12, or both imaging devices. The container identification module 20 can operate on or retrieve container reference data.

[0023] The container reference data includes one or more of the following components: the reference dimensions (e.g., length, width, and height), volume, reference shape, drawings, models, layout, and arrangement of the container 85; the perimeter of the container 81; the edges of the container 181; the reference dimensions, reference shape, drawings, models, layout, and arrangement of the entire load space 93 of the receiving vehicle; the wheelbase of the axles under the load space, the turning circle of the axles under the load space, the arrangement of the towing device under the load space 93 of the receiving vehicle; and the distance between the pivot point of the towing device and the wheelbase of the axles under the load space. The container reference data can be stored and retrieved from the data storage device (e.g., non-volatile electronic memory).The container reference data can, for example, be stored by a corresponding recognition device for the receiving vehicle located on the data storage device of the transferring vehicle's systems 11, 111, be retrievable by this device, or be indexed. For each such recognition device for the receiving vehicle, a corresponding unique reference data set for the container can be stored in the data storage device.

[0024] In one arrangement, the container identification module 18 identifies the position of the container 85 as follows: if the linear orientation of a set of pixels in the acquired image data corresponds to one or more edges 181 of the perimeter 81 of the container 85, as specified by the container's reference data, the position of the container is thereby identified. A target area, a central region, or a central area of ​​the opening 83 of the container 85 may be identified, among other ways, by dividing the distance (e.g., the shortest distance or the typical distance along the surface) between opposite sides of the container (by two), or by identifying corners of the container and the points where the diagonals intersecting the corners intersect. In one arrangement, the central area may be defined as an opening in the container (e.g.,A circular, elliptical, or rectangular opening having an opening surface area that is at least two times larger than or equal to the cross-sectional area of ​​the end of the discharge chute. However, other surfaces also fall within the scope of the patent claims.

[0025] The chute locator 22 identifies one or more of the following: (1) the pixels reproducing at least a portion of the chute 89; or (2) the chute end reproducing pixels associated with the end 87 of the chute 89. The chute identification module 22 may utilize color discrimination, intensity discrimination, or texture discrimination techniques to identify background pixels from one or more selected chute pixels using associated chute pixel patterns or corresponding properties (such as color or color pattern - (e.g., red / green / blue (RGB) pixel values), pixel intensity pattern, texture pattern, luminance, brightness, hue, or reflectivity) as applied to the chute 89 or the end of the chute 87 for identification purposes.

[0026] To maintain the alignment of the discharge chute 56 over the central area, central region, or target area of ​​the agricultural material unloading container 85, the alignment module 24, the master controller 59, or both periodically estimate or determine movement commands. The alignment module 24, the master controller 59, or both may send commands or requests to the transferring vehicle 91 regarding the speed or heading of the transferring vehicle to maintain the alignment of the position of the transferring vehicle 91 relative to the receiving vehicle. For example, the alignment module 24 may transmit a change request to the master controller 59 regarding the spatial offset between the vehicles 79, 91.The master controller 59 or the coordination module 57 then sends a control or heading command to the steering controller 32, a braking or deceleration command to a braking system 34, and a drive, acceleration, or torque command to a drive controller 40 to establish the spatial offset of the target objects or to change the spatial offset.

[0027] In another configuration, during loading of the receiving vehicle's hopper 85, the alignment module 24 may regularly or periodically shift, adjust, or rotate the target area or the central area to ensure uniform filling, uniform height, or uniform distribution of the agricultural material throughout the hopper 85, wherein the image processing module 18 has determined the fill level of the agricultural material in the image data from the material profile module 27.

[0028] The image processing module 18 may include a material profile module 27 or a fill level sensor for detecting a one-dimensional, two-dimensional or three-dimensional representation of the fill level or volume distribution of the agricultural material in the container 85 or in the loading space 93. Fig. For example, Figure 4D shows various illustrative two-dimensional representations of the fill level of the container 85 or the distribution of agricultural material in the container 85, where Fig. 4D is described in detail below.

[0029] In one arrangement, the coordination module 57 or the control controller 32 adjusts the relative position (or offset) of the transferring vehicle 91 to the receiving vehicle 79. The alignment module 24, the coordination module 57, and the auger rotation system 116 can control the relative position of the discharge chute 89 or the end of the discharge chute 87 to the periphery of the bin 81 to achieve uniform filling to the desired fill level. For example, the combine harvester's rotary actuator 216 can adjust the angle of the discharge chute (e.g.,adjust a first chute angle (α), a second chute angle (β), or a composite angle (α and β) formed by the chute 89 with respect to a reference axis or a reference coordinate system associated with the transferring vehicle 91 or a generally perpendicular plane associated with the direction of travel of the transferring vehicle 91, where the chute 89 meets the vehicle and rotates relative to the vehicle. With respect to the self-propelled forage harvester, the chute angle is controlled by a chute controller 54 in conjunction with a rotation sensor 116, a tilt sensor 118, a deflection sensor 120, a rotation actuator 122, a tilt actuator 124, and a deflection actuator 126.

[0030] The end of the discharge chute 87 can be adjusted for the purpose of depositing agricultural material by changing the angle of the discharge chute or the position of the discharge chute within the circumference of the container 81 plus a tolerance safety distance from the circumference of the container 81 within the container 85. The end of the discharge chute 87 can be adjusted by various methods, which can be applied alternatively or cumulatively. In a first method, the alignment module 24 adjusts the end of the discharge chute 87 for discharging agricultural material in such a way that the angle of the discharge chute (e.g., a first angle of the discharge chute (α), a second angle of the discharge chute (β), or both (i.e., (α and β)) is / are changed.In a second method, the alignment module 24 requests (or commands) the coordination module 57 to perform the front-to-rear offset adjustment (Φ or φ), the lateral adjustment (Δ), or both, wherein the coordination module 57 manages or choreographs the relative front-to-rear offset and the lateral offset between the transferring vehicle 91 and the receiving vehicle 79. In a third method, the alignment module 24 primarily adjusts the end of the discharge chute 87 for discharging agricultural material by changing the angle of the discharge chute, and secondarily and at regular intervals (e.g., periodically), the coordination module 57 changes the front-to-rear offset and the lateral offset by adjusting the respective front-to-rear offset (Φ or φ) orthe lateral adjustment (Δ) in order to achieve a uniform fill level or a uniform loading of the container with the agricultural material. The end of the discharge chute 87 can accordingly be regularly readjusted for the purpose of depositing agricultural material (e.g. according to a matrix of one or more rows or columns of fixed offset positions) by changing the spatial relationship between the transferring vehicle and the receiving vehicle by a front / rear offset or a lateral offset in order to achieve a target orientation or a desired uniform distribution when filling the container 85 or the loading space 93 with agricultural material (. Fig. 4E), whereby the adjustment of the angle of the discharge chute serves to achieve a fine adjustment of the distribution of the agricultural material within the container (e.g. from any position within the matrix).

[0031] In the image processing module 18, the image data evaluation device 25 may include an image evaluation unit, a judgment module, a Boolean logic-based circuit, an electronic module, and a software module or software instructions to make the decision as to whether to use the first image data, the second image data, or both to establish a relative position of the discharge chute to the perimeter of the container (or the orientation of the spatial offset between the vehicles) based on an evaluation of the significant change in the intensity of pixel data or significant change under ambient light conditions during a sampling time interval.

[0032] In the combine harvester, the master controller 59 is coupled to the vehicle data bus (e.g., 60). In the self-propelled forage harvester, on the other hand, the master controller 59 is coupled to the implement database 58, which is connected to the vehicle data bus 60 via a gateway 29. In one embodiment, the master controller 59 comprises an auto-guidance module 55 for route guidance and a coordination module 57. The auto-guidance module 55 for route guidance or the master controller 59 can control the transferring vehicle 91 based on location data from the first location receiver 42, as well as according to a path map or a desired route of the vehicle (e.g., as stored in the data memory).The auto-guidance module 55 for route guidance or the master controller 59 transmits command data to the steering controller 32, the brake switch 36 and the drive controller 40 in order to automatically control the route of the transferring vehicle 91 when automatically following a route map or to follow a course manually specified by the operator via the user interface 44 or the control system 30.

[0033] The coordination module 57 may facilitate the alignment of movements (e.g., choreography) between the transferring vehicle 91 and the receiving vehicle 79 during the depositing or transport of agricultural material between the vehicles. For example, the coordination module 57 may enable the maintenance of a uniform lateral offset (Δ) and a uniform front / rear offset (Φ or φ) between the vehicles 91, 79 during the unloading of agricultural material, subject to any adjustments to achieve a uniform distribution of the material in the container 85. The uniform lateral offset and the uniform front / rear offset may collectively be referred to as a uniform spatial offset. In certain embodiments, the maintenance of the lateral offset and the front / rear offset, respectively, isthe coordination of a change in the lateral offset as well as the front / rear offset (e.g. according to a two-dimensional matrix of predefined positions (X, Y points) for uniform filling of a particular container or loading space) is a necessary or desired prerequisite for the implementation of the adjustment of the angle of the discharge chute 89 or the end of the discharge chute 87 by the alignment module 24.

[0034] In one embodiment, in a leader mode, the transferring vehicle 91 is controlled by the auto-guide module 55 or the master controller 32 according to a path plan, or by a human operator. When the transferring vehicle 91 is operating in an automated mode or a self-steering mode, the master controller 59 provides command data locally to the steering controller 32, the brake switch 36, and the drive motor controller 40 of the transferring vehicle 91. In an automated mode and in a leader-follower mode, the transferring vehicle 91 is automatically controlled and directed during the placement of agricultural material from the transferring vehicle 91 to the receiving vehicle 79.

[0035] The image processing module 18 provides image data (rectified data, disparity data, or both) to a user interface processing module 26, which directly or indirectly provides status message data and performance message data for a user interface 44.

[0036] In one embodiment, a location receiver 42, a first wireless communication device 48, a vehicle controller 46, a steering controller 32, a brake controller 36, and a propulsion controller 40 may communicate via the vehicle data bus 60. The steering controller 32 is, in turn, coupled to a control system of the transferring vehicle 91; the brake controller 36 is coupled to the braking system 34 of the transferring vehicle 91; and the propulsion controller 40 is coupled to the propulsion system 38 of the transferring vehicle 91.

[0037] The control system 30 may include an electrically driven control system, an electro-hydraulic control system, a gear-driven control system, a rack-and-pinion control system, or another control system that changes the course of the transferring vehicle 91 or the orientation of one or more wheels of the transferring vehicle. The braking system 34 may include a regenerative braking system, an electro-hydraulic braking system, a mechanical braking system, or another braking system capable of decelerating the vehicle using hydraulic or mechanical forces, frictional forces, or electrical forces.The drive system 38 may include one or more of the following devices: (1) a combination of an electric motor and an electrical control device; (2) an internal combustion engine controlled by an electronic fuel injection system or other fuel metering device controllable by electrical signals; or (3) a hybrid vehicle in which an internal combustion engine drives an electric generator coupled to one or more electric motors.

[0038] The systems 11, 111 facilitate the transport of agricultural material from the transferring vehicle 91 to the receiving vehicle 79 and include a drive section 75 for driving the receiving vehicle and a cargo space 93 for storing agricultural material. A stereo imaging device, such as the first imaging device 10, is aligned with the end of the receiving vehicle 79 facing the cargo space 93. As shown in the Fig. 1 and Fig. 2, the first imaging device 10 and the optional second imaging device 12 are respectively Fig. 3 mounted on the transferring vehicle 91.

[0039] In summary, one or more imaging devices 10, 12 are arranged to capture image data. A container identification module 20 detects a perimeter of a container 81 of the cargo space 93 in the captured image data. The cargo space 93 has an opening inward from the perimeter of the container, which is intended to receive the agricultural material. A discharge chute locator 22 is configured to identify a discharge chute 89 of the transferring vehicle 91 in the captured image data. An alignment module 24 is operable to sense the relative position of the discharge chute 89 and the perimeter of the container 81 and to generate command data that is sent to the transferring vehicle 91 for the purpose of controlling the transferring vehicle 91 in concert with the receiving vehicle 79 such that the discharge chute 89 is aligned within a central region 83 or the opening of the grid pattern 82 of the perimeter of the container.For the purpose of controlling the transferring vehicle 91 in accordance with the receiving vehicle 79, a control controller 32 is connected to a control system 30 of the transferring vehicle 91.

[0040] In order to control an optional mast 573 for mounting and adjustable positioning of the first imaging device 10, the second imaging device 12, or both, in one embodiment, an optional mast control unit 674, recognizable by the dashed lines, is connected to the data bus of the vehicle 60 ( Fig. 1) or the tool data bus 58 ( Fig. 2). The mast controller 674 is used to change the orientation or height above ground of the first imaging device 10, the second imaging device 12, or both, where the orientation can be expressed in one of the following ways: tilt angle, wide angle, pitch angle, low angle, or roll angle.

[0041] In an illustrative embodiment of a machine vision-enhanced control system 11, 111 having an adjustable mast 573, at least one imaging device 10, 12 is mounted on the cargo area 93 of the receiving vehicle 79 and acquires image data. The adjustable mast 573 is capable, for example, using data from the mast controller 674, of adjusting a height of the imaging device 10, 12 within a height range and, in doing so, adjusting the tilt angle of the imaging device 10, 12 within a downward tilt angle range, as well as a rotation angle or wide angle within a wide angle range. The image processing module 18 is provided or programmed (e.g.,The method may be used to determine (e.g., via software instructions or code) whether to adjust the height of the imaging device 10, 12, or to decrease or increase the tilt angle of the imaging device 10, 12 based on an evaluation of a significant change in pixel data intensity or significant change in ambient light conditions (e.g., by the optical sensor 110, 112) during a sampling time interval. Under certain operating conditions, such as ambient light conditions outdoors, decreasing or increasing the tilt angle may increase the quality level of the acquired image data or reduce the change in image data intensity to a level below a threshold variation.Reduced fluctuations in the intensity of the image data or a reduced accumulation of dust or dirt particles on a lens of the imaging device are some of the advantages that can be achieved, for example, by increasing or adjusting the tilt angle of the imaging device 10, 12. As already mentioned, a container identification module 20 is capable of detecting a perimeter of a container 81 of the cargo space 93 in the acquired image data. Likewise, a drop chute locator 22 can detect a drop chute of the transferring vehicle 91 in the acquired image data.An alignment module 24 detects the relative position of the discharge chute 89 and the perimeter of the container 81 and generates command data for the control controller 32 to control the transferring vehicle 91 in conjunction with the receiving vehicle 79 such that the discharge chute 89 or the end of the discharge chute 87 is aligned within a target zone of the grid pattern 82 or the central region 83 of the perimeter of the container 81.

[0042] In an illustrative embodiment of a machine vision-enhanced control system with an adjustable mast 573, the image processing module 18 transmits a data message to a mast controller 674 (or the adjustable mast 573) to increase the tilt angle in the event of a significant change in pixel data intensity or a significant change in ambient light conditions that exceeds a threshold of variation during a sampling time interval. For example, the image processing module 18 transmits a data message to a mast controller 674 to adjust the tilt angle in increments (e.g., one-degree increases or decreases) within an angular range of approximately minus 10 degrees to approximately minus 25 degrees, and from a generally horizontal plane.

[0043] In one arrangement, a user interface 44 is arranged so that container reference data or dimensional parameters related to the receiving vehicle can be entered. The container reference data or dimensional parameters include, for example, a distance between a hitch or pivot point (which couples the drive unit 75 and the cargo space 93) and the axle of the front wheel under the cargo space 93 of the receiving vehicle 79.

[0044] In an alternative embodiment, the first imaging device 10 comprises a monocular imaging device, and the second imaging device comprises a monocular imaging device 12, each providing first monocular image data and second monocular image data. The image processing module 18 can create a stereo image from the first monocular image data (e.g., right-side image data) and the second monocular image data (e.g., left-side image data) with respect to the relative position and orientation of the first imaging device 10 and the second imaging device 12. The image processing module 18 determines the following: (1) at least two points on a common line of sight that half-separates the lenses of both the first imaging device 10 and the second imaging device 12, and (2) a linear spatial separation between the first imaging device 10 and the second imaging device 12, wherein the first field of view 477 ( Fig. 4A) of the first imaging device 10 and the second field of view 277 ( Fig. 4A) of the second imaging device 12 overlap, at least partially, to capture the discharge chute 89, the end of the discharge chute 87, and the periphery of the container 81 in the captured image data.

[0045] In an alternative embodiment, the Fig. 1 and Fig. 2 further includes an optional distance sensor 440 and an optional inertial sensor 442, indicated by the dashed lines. The distance sensor 440 may consist of a magnetic rotation sensor, a gear-driven sensor, or a contactless sensor for measuring the rotational speed of one or more wheels of the transferring vehicle in order to estimate a distance traveled by the transferring vehicle during a measurement period or a ground speed. The distance sensor 440 may be coupled to the data bus of the vehicle 60 or a tool data bus 58. The inertial sensor 442 may consist of one or more accelerometers, gyroscopes, or other inertial-measuring devices coupled to the data bus of the vehicle 60 or a tool data bus 58.The optional distance sensor 440 and the optional inertial sensor 442 can augment or supplement position data or movement data provided by the first location receiver 42.

[0046] The control system 111, which has been enhanced with machine vision, Fig. 2 is similar to System 11 from Fig. 1, except that the system 111 from Fig. 2 further comprises a tool data bus 58, a gateway 29, a light controller 50, and a discharge controller 54 coupled to the vehicle data bus 60 for the headlights 14 and the discharge chute 89, respectively. The light controller 50 controls the headlights 14; the discharge controller 54 controls the discharge chute 89 by means of a servo motor, an electric motor, or an electro-hydraulic mechanism for changing or adjusting the orientation or angle of the discharge chute 89 or the end of the discharge chute 87. In one arrangement, the tool data bus 58 may comprise a controller area network (CAN) data bus. Likewise, the vehicle data bus 60 may comprise a controller area network (CAN) data bus. In an alternative embodiment, the tool data bus, the vehicle 60 data bus, or both, may be an ISO data bus (where “ISO” stands for “International Organization for Standardization”) orISOBUS, Ethernet or another data protocol or communication standard.

[0047] The self-propelled forage harvester includes a gateway 29 to ensure secure or controlled communication between the implement data bus 58 and the vehicle data bus 60. The gateway 29 comprises a firewall (e.g., hardware or software), a network router, or other security device that restricts or prevents a network component or device connected to the implement data bus 58 from communicating with the vehicle data bus 60 or a network component or device on the implement data bus 31 (e.g., in the event of unauthorized communication), unless the network component or device connected to the implement data bus 58 follows a specific security protocol or uses a handshake, a password and key, or another security measure.The gateway 29 can further, in one embodiment, encrypt communications with the data bus of the vehicle 60 and, upon entry of a correct encryption key or if other security measures are met, decrypt communications from the data bus of the vehicle 60. The gateway 29 can allow network devices connected to the implement data bus 58 to communicate via an open standard or via third-party hardware and software, whereas the network devices connected to the data bus of the vehicle 60 are provided exclusively by the manufacturer of the transferring vehicle (e.g., a self-propelled forage harvester) or by companies approved by the manufacturer.

[0048] Fig. Figure 2 illustrates how a first location receiver 42, a user interface 44, a user interface processing module 26, and the gateway 29 are connected to the implement data bus 58, although in other embodiments, these components or network devices may be connected to the data bus of the vehicle 60. The light controller 50 and the discharge controller control unit 54 are coupled to the vehicle data bus 60. The light controller 50 and the discharge controller 54 are in turn coupled, either directly or indirectly, to the headlights 14 of the transferring vehicle 91 and to the discharge chute 89 of the transferring vehicle 91 (e.g., a self-propelled forage harvester). Although the system according to Fig. 2 is particularly suitable for use or installation on a self-propelled forage harvester (SPFH), the system can be Fig. 2 also apply to harvesting machines or other heavy equipment.

[0049] At Fig. 4A is a plan view of a transferring vehicle 91 and a receiving vehicle 79. The same reference numerals refer to the same components in Fig. 4A and Fig. 3. Fig. 4A shows a first imaging device 10 located on the body of the transferring vehicle 91. The first imaging device 10 has a first field of view 477, recognizable by the dashed lines. According to Fig. 4A, the discharge chute 89 or the end of the discharge chute 87 is generally aligned over a central area 83, a central region or target area, or a grid pattern 82 of the storage unit 93 or container 85 for discharging material from the transferring vehicle 91 into the receiving vehicle 79. Similarly, the transferring vehicle 91 and the receiving vehicle 79 are aligned with each other in the manner shown, even when the vehicles 79, 91 are moving in a coordinated direction of travel or generally in a parallel direction of travel and are otherwise moving relative to each other at a very low speed or not at all.

[0050] According to Fig. 5A, an optional second imaging device 12 may be mounted on the drop chute 87 of the transferring vehicle 91 with a second field of view 277 that may be slightly offset, overlapped, or aligned with the first field of view 477 to provide redundancy in the event of failure, miscontrol, unavailability, or unreliability of the imaging device 10 or the provision of low-quality image data. For example, the first imaging device 10 may not function reliably if dust, fog, salt, or air pollutants accumulate on it, or if it is exposed to insufficient ambient light or excessive glare from sunlight or light reflections. According to Fig. 4A, the image processing module 18 may estimate the distance or range from the first imaging device 10, the second imaging device 12, or both to an object in the image, e.g., the discharge chute 89, the end of the discharge chute 87, the perimeter of the container 81, or the fill level or profile of agricultural material in the container 85 (e.g., at various positions or coordinates within the container 85).

[0051] At Fig. 4B is a plan view of a transferring vehicle 91 and a receiving vehicle 79. The same reference numerals refer to the same components in Fig. 3, Fig. 4A and Fig. 4B. Fig. 4B shows a first imaging device 10 exclusively on the body of the transferring vehicle 91. The first imaging device 10 has a first field of view 477, recognizable by the dashed lines. In Fig. 4B, the discharge chute 89 or the end of the discharge chute 87 is generally aligned over a central area 83, a central region or target area or grid pattern 82 of the storage unit 93 or the container 85 for discharging material from the transferring vehicle 91 into the receiving vehicle 79. Similarly, the transferring vehicle 91 and the receiving vehicle 79 are aligned with each other as shown, even when the vehicles 79, 91 are moving in a coordinated direction of travel or at a constant offset (e.g., Φ or φ; Δ).

[0052] At Fig. 4C is a horizontal view, with the view of the reference line 4C-4C in Fig. 4B follows. In one embodiment, the first imaging device 10 is mounted on the transferring vehicle 91 on a support structure 573 (e.g., a tubular mast that allows tilting or pivoting) to provide a downward field of view 677 or a downwardly inclined field of view.

[0053] In an alternative embodiment, the support structure 573 comprises an adjustable mast or a telescopic mast controlled by a mast controller 674 to remotely adjust the height, tilt angle, pitch angle, rotation angle, or wide angle to provide reliable image data for processing by the image processing module 18.

[0054] If the first imaging device 10 is sufficiently elevated or mounted on the transferring vehicle 91 with respect to the loading space 93, the first imaging device 10 has a view or a second field of view 677 downward into the loading space 93 or container 85 that is sufficient to monitor and profile the surface (or the height (z) compared to the respective x, y coordinates in the container) of the agricultural material (e.g., grain) during the deposition of the agricultural material in the loading space 93. The first imaging device 10 can be mounted on the roof of the transferring vehicle 91 so that it directly faces the side thereof, together with the discharge chute 89, during the unloading of agricultural material, or faces away from the side of the transferring vehicle 91.

[0055] In an illustrative arrangement, corresponding to the downward field of view 677, the viewing axis perpendicular to the respective lens of the first imaging device 10 is tilted downward from the generally horizontal plane at an inclination angle (ε) (e.g., approximately 10 to 25 degrees downward). Tilting a field of view or viewing axis of the imaging device 10 downward from the horizontal provides several advantages. First, a smaller portion of the sky is visible in the field of view of the imaging device 10, so that the acquired image data exhibits a more uniform image intensity profile. The tilted arrangement of the viewing axis(es) (perpendicular to the lens of the imaging device 10) is well-suited to addressing potential dynamic range issues, such as those caused by bright sunlight or changing cloud cover.Second, this makes the lower portion of the cargo area 93 more visible in the image data, allowing for the capture of image data related to one or more wheels below the cargo area 93. The wheel is a feature beneath the cargo area 93 that is highly amenable to monitoring using image processing techniques. Third, tilting the stereo camera downward may result in less accumulation of dust or other debris on the lens or external window of the imaging device 10, 12.

[0056] At Fig. 4D is a two-dimensional representation for illustrative purposes of various possible distributions of material inside a container 85 as viewed along the reference line 4D in Fig. 4B. In one arrangement, the y-axis is congruent with the longitudinal axis or direction of movement of the container, the z-axis is congruent with the height of the material in the container, and the x-axis is perpendicular to the direction of movement of the container, with the x-, y-, and z-axes generally orthogonal to each other.

[0057] In the diagram in Fig. In Figure 5D, the vertical axis corresponds to the mean height (Z) 500 of the material in bin 85, and the horizontal axis corresponds to the longitudinal axis (y) 502 of bin 85. The maximum capacity 504, or bin capacity, is indicated by the dashed line on the vertical axis. The front 512 of bin 85 is located at the origin, while the rear 514 of bin 85 is located on the vertical axis.

[0058] Fig. Figure 4D shows three illustrative distributions of material within the container 85. The first distribution is a two-peaked profile 508 with two main peaks in the distribution of material within the container 85. The two-peaked profile 508 is shown as a dotted line. The two-peaked profile 508 may occur at a location where the control of the angle of the discharge chute is determined by an electro-hydraulic system with non-proportional valves.

[0059] The second distribution is the front-sloping, single-peak profile 510, with a single peak of material sloping toward the front of the container 85. The front-sloping, single-peak profile 510 is depicted as alternating long and short dashed lines. The second distribution may occur when the volume or length (y) of the container 85 is above a minimum threshold and the relative orientation between the end of the discharge chute 87 and the container 85 is largely stationary during a significant portion of the discharge.

[0060] The third distribution is the target profile 508, which, as set forth in this document, can be achieved by following a suitable filling strategy. For example, the angle of the discharge chute can be adjusted during unloading to promote a uniform distribution of the agricultural material in the container. Furthermore, the lateral offset (Δ) or the front-to-rear offset (Φ or φ) between the vehicles 79, 91 can be adjusted according to a matrix (e.g., an XY coordinate matrix of equidistant locations of the transferring vehicle relative to an equidistant position of the receiving vehicle) of relative unloading positions, particularly for longer or wider containers that cannot be filled uniformly from a single relative unloading position between the vehicles 79, 91.

[0061] At Fig. 4E is a plan view of a transferring and receiving vehicle 79, with the transferring vehicle 91 aligned according to a matrix 500 of possible offset positions 502, 504 between the transferring vehicle 91 and the receiving vehicle 79. Each offset position 502, 504 can be expressed as a combination of a unique lateral offset (Δ) and a unique front-to-rear offset (Φ or φ) between the vehicles 79, 91. As shown, the matrix 500 is a two-dimensional 2 x 3 matrix (2 columns and 3 rows) of possible offset positions 502, 504. Although six possible offset positions 502, 504 are shown here, in alternative embodiments, the matrix 500 may include any number of possible offset positions greater than or equal to two.Here, the transferring vehicle 91 occupies the offset position 504 in the first column at the second row of the matrix 500, while the remaining possible offset positions 502 are not occupied by the transferring vehicle. Upon instruction from each of the systems 11, 111, the image processing module 18 or the master controller 59 of the transferring vehicle 91 can switch to any free or other possible offset position 502 within the matrix 500 to promote or facilitate an even distribution of the agricultural material within the container 85 or the cargo space of the receiving vehicle 79.The spatial offset between the transferring vehicle 91 and the receiving vehicle 79 can be adjusted according to the matrix 500 or another matrix of predetermined spatial offset positions so as to promote an even distribution of the agricultural material in the loading space of the receiving vehicle 79, each such matrix being associated with a unique lateral offset (Δ) and a unique front-to-rear offset (Φ or φ) between the vehicles 79, 91.

[0062] In an embodiment according to Fig. 4E, both the transferring vehicle 91 and the receiving vehicle 79 may move forward at approximately the same speed and direction (e.g., within a tolerance or error range of the controller during harvesting), with the relative position of the receiving vehicle 79 generally being fixed or constant with respect to any position 502, 504 of the matrix 500 that the transferring vehicle 91 may occupy.

[0063] In an alternative embodiment, the receiving vehicle 79 may be represented as occupying a two-dimensional matrix (e.g., a 3 x 3 matrix with three columns and three rows) of possible offset positions, while the position of the transferring vehicle 91 is generally fixed or constant with respect to each possible matrix position of the receiving vehicle 79. At the direction of each of the systems 11, 111, the image processing module 18 or the master controller 59 of the transferring vehicle 91 may change to any vacant or other possible offset position within the matrix to promote or facilitate even distribution of the agricultural material within the container 85 or cargo space 93 of the receiving vehicle 79.

[0064] According to the Fig. 5A, Fig. 5B, Fig. 6A and Fig. 6B, each of the blocks or modules may represent a software module, an electronic module, or a combination of both. Software modules may contain program instructions, subroutines, object-oriented program code, or other software content. The arrows between the blocks and modules in Fig. 6 indicate the flow of data and information between the blocks. The arrows can represent physical or virtual communication paths, or a combination of the two. Physical communication paths refer to transmission lines or one or more data buses for sending, receiving, or transmitting data. Virtual communication paths refer to the transfer of data, programs, or data messages between modules.

[0065] As in the Fig. 1 and Fig. 2, the first imaging device 10, the second imaging device 12, or both devices together provide stereo camera raw images (or raw image data) to the image cleaning module 101. Fig. Figure 5A is a block diagram showing raw camera images (monocular or stereoscopic) processed by an image cleaner 101 to generate error-corrected images as input to the container identification module 20. Optional input to the container identification module 20 is the data from the drop chute locator 22, which is described in more detail below. Fig. 5B is a block diagram showing raw (monocular or stereoscopic) camera images processed by an image adjuster 101 to produce a defect-corrected image. The corrected image is processed by the disparity generator 103 to generate regions in the form of disparity data. The corrected images and the disparity data are then processed together with the chute position data 1002 by the chute locator 22. The output data from the chute locator 22 is used as input to the bin identification module 20. In an alternative embodiment, the data from the chute locator is used as input to the bin identification module 22 to improve the material distribution in the bin 85. Like reference numerals in the Fig. 1, Fig. 2, Fig. 5A, Fig. 6A and Fig. 6B denote identical elements.

[0066] Fig. Figure 6A is a block diagram showing raw camera images (monocular or stereoscopic) processed by an image cleaner 101 to produce degraded images used as input to the chute locator 1001 for further processing along with the chute position data 1002 provided by the vehicle module 1000. The output data from the chute locator 22 can be used as input to the container identification module 20.

[0067] Fig. Figure 6B is a block diagram showing raw camera images (monocular or stereoscopic) processed by an image adjuster 101 to generate de-corrected images. The corrected images are processed by a disparity generator 103 to generate regions in the form of disparity data. The corrected images and disparity data, along with the chute position data 1002, are then processed by the chute locator 22. The output data from the chute locator 22 may be further processed by the container identification module 20. The image adjuster module 101 processes the acquired image data or raw stereo images to reduce or eliminate radial distortion and provides the necessary image alignment for stereo correspondence.The radial distortion is related to the radial lenses of the first imaging device 10, the second imaging device 12, or both. The image cleanup module 101 receives the raw stereo image data as input and returns error-corrected stereo image data.

[0068] In one illustrative embodiment, the image cleaner 101 eliminates or reduces any vertical or differential offsets between a pair of stereo images of the same scene. Furthermore, the image cleaner module 101 may align the horizontal component (or horizontal pixel rows of the stereo images) to be parallel to the scan lines or common reference axis of each imaging device (e.g., the left and right imaging devices) within the first and second imaging devices 10, 12. For example, the image cleaner 101 may reorder pixels from their original coordinates to cleaned coordinates for the left, right, or both images to enable acquisition of the images or the rectified left and right images of the stereo image.The error-corrected image enables efficient processing and easier identification of corresponding pixels or objects within the left and right images of a common scene for subsequent image processing.

[0069] In one arrangement, disparity generator 103 applies a stereo matching or disparity algorithm to acquired stereo image data, such as the de-corrected stereo image data provided by image adjuster 101. The stereo matching or disparity algorithm may comprise a sum of absolute differences algorithm, a sum of squared differences algorithm, a consensus algorithm, or another algorithm for calculating the difference or disparity of each pair of corresponding pixels of the right and left images (e.g., along the horizontal axis of the two images or a line parallel to it).

[0070] In an illustrative method using sums of absolute differences, the right and left images (or blocks or rows of image data) can be translated to align corresponding pixels of the right and left images. The stereo matching algorithm, or disparity algorithm, calculates a disparity value between corresponding pixels in the right- and left-side image data. For example, to estimate the disparity value, each intensity value of a subject pixel and the sum of the intensity values ​​of the surrounding pixels (e.g., in a block or matrix of pixels) are compared with each intensity value of the corresponding second subject pixel and the sum of the intensity values ​​of the surrounding pixels (e.g., in a block or matrix of pixels). The disparity values ​​can be used to create a disparity map ora disparity image for the corresponding right and left side image data.

[0071] A bin locator estimates a distance or range of distances from the first imaging device 10, the second imaging device 12, or both devices to the pixels on the perimeter of the bin 81, the discharge chute 89, the end of the chute 87, or any other straight edge, curve, ellipse, circle, or object identified by the edge detector, the linear Hough transformer, or both. For example, the image processing module 18 may use the disparity map or disparity image to estimate a distance or range of distances from the first imaging device 10, the second imaging device 12, or both devices to the pixels on the perimeter of the bin 81, the bin edges 181, the bin opening 83, in the vicinity of the aforementioned objects, or elsewhere.

[0072] In one embodiment, the container identification module 22 includes the following components: (1) an edge detector for measuring the line strength or reliability of one or more edges 181 or points on the container perimeter 81 in the image data; (2) a linear Hough transformer for identifying the angle and offset of appropriate linear segments in the image data relative to a reference point or optical axis.Reference axis of one or more imaging devices 10, 12; (3) a container locator that uses spatial and angular constraints to eliminate suitable linear segments that, for logical or other reasons, cannot be among the detected linear segments of the container perimeter 81 or among the points on the container perimeter 81; and (4) a container locator that converts the detected and non-eliminated linear segments or points into two- or three-dimensional coordinates relative to a reference point or frame of reference of the receiving vehicle and the harvesting vehicle.

[0073] The edge detector may apply an edge detection algorithm to the rectified image data provided by image adjuster 101. The edge detector may use any number of suitable edge detection algorithms. Edge detection refers to the process of identifying and locating discontinuities between pixels of an image or acquired image data. These discontinuities may, for example, represent material changes, evident as changes in the color or intensity of pixels, thereby defining object boundaries in an image. A gradient edge detection method may be implemented by filtering the image data so that regions with larger discontinuities or gradients return different pixel values ​​than regions with smaller discontinuities or gradients.For example, the gradient method detects the edges of objects by estimating the first derivative of the pixel intensities of the image data. The Laplace method detects the edges of objects in an image by searching for zero crossings of the second derivative of the pixel intensities of an image. Other examples of suitable edge detection algorithms are, above all, the Sobel operator and the Roberts and Canny algorithms, which are among the best-known methods in this field. The edge detector can return a numerical value, a signal, or a symbol as a measure of the strength or reliability of the edges 181 of an image. For example, the edge detector can provide a numerical value or a measure of the edge strength within a range or scale, or a relative strength or credibility to the linear Hough transformer.

[0074] The linear Hough transformer receives edge data (e.g., a measure of edge strength) for the receiving vehicle and determines the estimated angle and offset of the strong line segments, curve segments, or generally straight edges (e.g., of the container 85, the discharge chute 89, and the opening 83) in the image data. The estimated angle is related to the angle or the composite angle (e.g., a multi-dimensional angle) of a linear axis intersecting the lenses of the first imaging device 10, the second imaging device 12, or both. The linear Hough transformer includes a feature extractor for identifying line segments of objects with specific shapes from the image data.For example, the linear Hough transformer uses the edge data output by the edge detector to identify the parameters of straight-line or ellipse equations of objects in the image data, or classifies the edge data as line segments, circles, or ellipses. This enables the detection of containers or chutes with common straight, rectangular, circular, or elliptical features.

[0075] In one embodiment, the data manager supports the entry or selection of container reference data via the user interface 44. The data manager supports the entry, storage, and retrieval of container reference data, such as hanger dimensions, by the image processing module 18 to provide the container locator with spatial constraints on the line segments or data points that are potential edges 181 of the container opening.

[0076] In one embodiment, the angle estimator comprises a Kalman filter or an extended Kalman filter. The angle estimator estimates the angle of the storage unit 93 (e.g., the trailer) of the receiving vehicle 79 relative to the travel axis of the drive portion 75 (e.g., the tractor) of the receiving vehicle 79. The angle estimator (e.g., a Kalman filter) provides the bin locator with the angle constraints regarding the lines or data points representing potential edges 181 of the bin opening. In one embodiment, the chute position estimator or Kalman filter is coupled to the bin locator 22. The angle estimator's filter returns (or can return) the obtained estimated angle of the storage unit 93 relative to the travel axis of the drive portion 75 of the vehicle.

[0077] The container locator is adapted to receive the dimensions of the perimeter of the container 81 or the storage unit 93 as input to facilitate the identification of suitable linear segments to be considered as linear segments of the perimeter of the container 81. In one embodiment, the container locator is adapted to receive an estimated angle of the storage unit 93 relative to the drive portion 75 of the vehicle as input to facilitate the identification of suitable linear segments to be considered as linear segments of the perimeter of the container 81.The container locator uses spatial and angular constraints to eliminate lines in the image data that, for logical or other reasons, cannot be part of the container's opening 83 or edges 181, and then selects preferred lines (or data points on the edge of the container 81) as the most likely candidates for the container's opening 83 (containing material) or edges 181. The container locator describes (or transforms) these preferred lines in three-dimensional coordinates relative to the vehicle or other reference frame so that they represent the perimeter of the container 85.

[0078] In one embodiment, the chute locator 22 includes a chute classifier configured to identify appropriate pixels representing a portion of the chute 89 or the end of the chute 87 based on at least one reflectance, intensity, color, or texture characteristic of the image data (or pixels), the cleaned image data, or the raw image data. The chute locator 22 is configured to estimate a relative position of the chute 89 to the imaging device based on the pixels detected and classified as part of the chute 89. The discharge chute locator 22 receives the estimated chute position or chute angle (α) of the combine relative to the imaging device mounting location or the optical axis or reference axis of one or more imaging devices based on previous measurements to provide boundary data for the possible location of the discharge chute.

[0079] The chute classifier contains software instructions for an algorithm that uses expected color and texture features in the processed image data or the raw image data to identify appropriate pixels likely to belong to the discharge chute 89 or the end of the chute 87. For example, in one arrangement, the end of the chute 87 may be painted, coated, labeled, or marked with a coating, paint, or texture with a higher optical or infrared reflectivity, or higher intensity or brightness, than the rest of the discharge chute 89 or the transferring vehicle.The higher brightness, intensity or reflectivity of the end of the chute 87 (or the corresponding chute pixels of the image data compared to the background pixels) can be achieved by painting or coating the end of the chute 87 in the colors white, yellow, chrome or in a lighter color compared to the remaining part of the chute 89 or parts of the transferring vehicle within the field of view of the imaging devices 10, 12.

[0080] In one embodiment, the chute position estimator comprises a Kalman filter or an extended Kalman filter that receives data from previous measurements and bin reference data as input and outputs an estimate of the chute position, chute angle, and associated errors. The chute position estimator provides an estimate of the position or angle of the combine harvester's discharge chute, or its error, with respect to one or more of the following parameters: (1) the mounting position or pivot point of the transferring vehicle's discharge chute, or (2) the optical axis or other reference axis or reference point of the first imaging device 10, the second imaging device 12, or both, or (3) the axis of forward travel of the transferring vehicle.The Kalman filter outputs constraints on the possible locations of the discharge chute 89 or the end of the chute 87, an estimated chute position, a range for the chute location, or an estimated range for the chute position. In one embodiment, the chute position estimator or Kalman filter is coupled to the discharge chute locator 22.

[0081] The discharge chute locator 22 uses pixels classified as belonging to the auger of the discharge chute of the combine harvester 89 and uses a disparity image generated by the disparity generator 103 to estimate the relative location of the discharge chute to the first imaging device 10, the second imaging device 12, or both, a reference axis, or a coordinate system associated with the vehicle.

[0082] Fig. Figure 7 shows a flow chart of a method for unloading agricultural material from a vehicle or for transferring it between a transferring vehicle 91 and a receiving vehicle 79. In the Fig. 7, one or more embodiments of the systems 11, 111 described hereinbefore may be used.

[0083] In step S902, the first imaging device 10 points toward the cargo area of ​​the receiving vehicle 79 (e.g., a grain trailer) and acquires first image data (e.g., first stereo image data, first monocular image data, or the right image of a stereo image). For example, the first imaging device may be mounted on the body of the transferring vehicle and face the receiving vehicle and the container. In one embodiment, the first imaging device 10 has a first field of view 77 ( Fig. 3) or the field of view 477 ( Fig. 4A and Fig. 4B) of the loading space of the receiving vehicle 79.

[0084] In an alternative embodiment, the first imaging device 10 comprises a monocular imaging device that provides the first image portion (e.g., the left image) of the stereo image data of a scene or object.

[0085] In step S904, the optional second imaging device 12 points toward the cargo area 93 of the receiving vehicle 79 (e.g., a grain trailer) and acquires second image data (e.g., second stereo image data, second monocular image data, or the left image of a stereo image). For example, the second imaging device 12 may be mounted on the discharge chute 89 of the transferring vehicle 91 and facing the receiving vehicle 79 ( Fig. 3 and Fig. 4A). In one embodiment, the second image processing device 12 has a second field of view (177, 277) of the cargo space of the receiving vehicle, wherein the first field of view (77, 477) at least partially overlaps with the second field of view (177 or 277).

[0086] In an alternative embodiment, the second imaging device 12 comprises a monocular imaging device that provides the second image portion (e.g., the right image) of the stereo image data of a scene or object, wherein the image processing module 18 supports creating a stereo image from a combination of the first image portion (of the first monocular imaging device) and the second image portion with respect to the relative position and orientation of the first imaging device 10 and the second imaging device 12.

[0087] In step S906, an image processing module 18 or a container identification module 20 identifies the perimeter of a container 81 in the cargo space 93 in the acquired image data (e.g., in the data of the first image, the second image, or both), the cargo space 93 having an opening 83 inside the container 81 for receiving the agricultural material. Step S906 may be performed using various methods, which may be applied alternatively or cumulatively. As part of a first method, the image processing module 18 or the container identification module 20 may use one of the following methods or substeps: (1) measuring the line thickness of one or more edges 181 in the image data; (2) identifying the angle and offset of appropriate linear segments in the image data relative to an optical or reference axis (e.g.,the direction of travel of the transferring vehicle), or to a reference point associated with one or more imaging devices 10, 12; and (3) the use of spatial and angular constraints to eliminate suitable linear segments that, for logical or other reasons, cannot be among the detected linear segments of the container perimeter, wherein the container locator 20 converts the detected and non-eliminated linear segments or points into two- or three-dimensional coordinates relative to a reference point or frame of reference of the receiving vehicle and the harvesting vehicle.

[0088] In a second method, the image processing module 18 or the container identification module is configured to receive the dimensions of the circumference of the container 81 or the storage unit 93 as input to facilitate the identification of suitable linear segments to be considered as linear segments of the circumference of the container 81.

[0089] In the third method, the image processing module 18 or the container identification module 20 may obtain an estimated angle of the loading space 93 with respect to the drive part 75 of the vehicle to enable the identification of any linear segments under consideration, which in that case are then considered as linear segments of the circumference of the container 81.

[0090] In a fourth method, the image processing module 18 or the container identification module 20 provides the obtained estimated angle of the cargo space 93 with respect to the drive part 75 of the vehicle.

[0091] In step S908, the image processing module 18 or a chute locator 22 identifies a chute 89 (or the end of the chute 87) of the transferring vehicle 91 in the acquired image data. The image processing module 18 or the chute locator 22 may use various methods, which may be applied alternately or cumulatively. In a first method, based on the expected color and the expected surface structure features of the image data, the image processing module 18 or the chute locating device 22 identifies any pixels in the image data (e.g., error-corrected data or raw image data) that may be considered, wherein the pixels that may be considered represent a part of the chute 89 (e.g., the chute of the auger conveyor of the combine harvester) or the end of the chute 87.

[0092] In a second method, based on the classified, identified, candidate pixels of a portion of the chute 89, the image processing module 18 or the chute identification module 22 estimates a relative position or angle of the chute 89 or the end of the chute 87 relative to the imaging device.

[0093] In a third method, based on previous measurements used to provide boundary data about the possible positions of the discharge chute 89, the image processing module 18 or the discharge chute identification module 22 obtains an estimated position of the discharge chute or the angle of the discharge chute of the combine harvester with respect to the mounting location, the optical axis, the reference axis, or the reference point of the imaging device 10, 12.

[0094] In a fourth method, the image processing module 18 or the discharge chute locating device 22 provides the estimated position of the combine harvester's discharge chute or the estimated angle of the discharge chute to the container identification module 20.

[0095] In step S910, the image data evaluation device 25 or the image processing module 18 determines whether to use the first image data, the second image data, or both based on an assessment of the intensity of pixel data or ambient light conditions. Step S910 can be performed using various methods, which can be applied alternately or cumulatively.

[0096] In a first method, in which a first optical sensor 110 is connected to the respective first imaging device 10, the image data evaluation device 25 or the image processing module 18 makes a decision to use the first image data if the variation in the ambient light intensity during a sampling time interval (e.g., corresponding to a sampling rate of 1 to 120 samples per second) is less than or equal to the maximum ambient light variation, as measured by the first optical sensor 110. In this case, according to the first method, the first image data are acquired exclusively by the first imaging device 10. A baseline, average, or mode level of the ambient light variation in the image data, a pixel block in the first image data, or an object in the image data (e.g.,The chute, the end of the chute, the perimeter of the container, or the containers) may be detected or tracked during operation or normal operation of the systems 11, 111. In one embodiment, the maximum ambient light intensity is set to be greater than the baseline, average, or mode level. For example, the maximum ambient light intensity (e.g., within the visible light spectrum, the near-infrared spectrum, or the infrared spectrum) is set to be statistically greater (e.g., about one to two standard deviations above the baseline level) than the average or mode level, or a signal level difference between the maximum ambient light intensity and the average level is equal to or greater than a threshold (e.g., in a range of approximately 3 dB to 6 dB).

[0097] In a second method, in which a second optical sensor 112 is connected to the second imaging device 12, the image data evaluation device 25 or the image processing module 18 makes a decision to use the second image data if the ambient light variation during a sampling time interval (e.g., corresponding to a sampling rate of 1 to 120 samples per second) is less than or equal to the maximum ambient light variation as measured by the second optical sensor 112. In this case, according to the second method, the second image data is acquired exclusively by the second imaging device 12. A baseline, average, or mode level of the ambient light variation in the second image data, a pixel block in the image data, or an object in the image data (e.g.,The chute, the end of the chute, the perimeter of the container or containers may be detected or tracked during operation or normal operation of the systems 11, 111. In one embodiment, the maximum ambient light intensity is set to be greater than the baseline, average, or mode level. For example, the maximum ambient light intensity (e.g., within the visible light spectrum, the near-infrared spectrum, or the infrared spectrum) is set to be statistically greater (e.g., about one to two standard deviations above the baseline level) than the average or mode level, or a signal level difference between the maximum ambient light intensity and the average level is equal to or greater than a threshold (e.g., in a range of approximately 3 dB to 6 dB).

[0098] In a third method, the image processing module 18 or the image data evaluation device 25 makes the decision to use the first image data of the first imaging device 10 if the variation in pixel intensity of a chute, an end of a chute, or a container in the first image during a sampling time interval is less than or equal to the maximum variation in pixel intensity as detected by the image processing module 18.

[0099] In a fourth method, the image processing module 18 or the image data evaluation device 25 makes the decision to use the second image data of the second imaging device 12 if the change in the pixel intensity of a discharge chute or an end of a discharge chute in the second image during a sampling time interval is less than or equal to the maximum variation in the pixel intensity as detected by the image processing module 18.

[0100] In a fifth method based on the pixel intensity of rejected image data falling outside a desired range or a variation of the pixel intensity during the sampling time interval, wherein the image processing module 18 or the image data evaluation device 25 is set such that the processing orthe use of the rejected image data comprising a portion of the acquired first image data or second image data, and which would otherwise be corrupted by one or more of the following conditions, is deliberately deactivated, the image processing module 18 and the image data evaluation device 25 are suitable for examining the question of whether the first image data, the second image data, or both should be used to identify the perimeter of the container as well as to identify the discharge chute (or the end of the discharge chute): (1) excessive transient sunlight at sunrise or sunset or excessive light from other sources (e.g., headlights of other vehicles), (2) transient sunlight or changing cloud cover, (3) fog, precipitation, or humidity (4) shading (e.g.,by vegetation, trees, buildings or equipment covers), (5) airborne dust or dirt particles, (6) light reflections (e.g. from polished, shiny or reflective surfaces of other machinery or vehicles) or other lighting conditions that may temporarily interrupt or hinder the proper operation of the imaging devices 10, 12.

[0101] In step S912, the image processing module 18 or the alignment module 24 determines the relative position of the discharge chute 89 or the end of the discharge chute 87 and the perimeter of the container 81 to generate command data to modulate the ground speed of the transferring vehicle 91 or reposition the discharge chute 89, or both, in concert with the receiving vehicle, such that the discharge chute 89 (or the end of the discharge chute 87) is aligned with a central region 83 of the perimeter of the container 81. The image processing module 18 may use, query, or retrieve previously stored data, such as dimensional parameters related to the receiving vehicle or dimensional parameters including a distance between a hitch and the rotational axis of the front wheel under the cargo space 93.Such dimensional parameters can be entered, for example, via a user interface 44 connected to the data bus of the vehicle 60 or via the image processing module 18.

[0102] To perform step S912, the image processing module 18 may use first location data from the first location receiver 42 disposed on the transferring vehicle 91 to detect the relative position between the discharge chute and the perimeter of the container and generate command data to modulate the ground speed of the transferring vehicle 91 or to reposition the discharge chute 89, or both in concert, such that the discharge chute 89 is aligned within a central region of the perimeter of the container 181 or a portion of the grid pattern 82.

[0103] In step S914, in a first arrangement, the master controller 59 or the propulsion controller 40 modulates the ground speed of the transferring vehicle 91. In a second arrangement, the vehicle controller 46 or the discharge controller 54 repositions the discharge chute 89. The rotary actuator 122 (e.g., a servo motor, an electric motor, a linear motor with a linear / rotary gear drive, or an electro-hydraulic device) controls the angle of the discharge chute 89 or the end of the discharge chute 87 relative to the direction of travel or another reference axis of the transferring vehicle, in response to the orientation module 24 or the image processing module 18 (e.g., an intelligent material placement controller). In a third arrangement, both the speed of the transferring vehicle and the discharge chute 89 are repositioned.

[0104] Fig.Figure 8 illustrates the data flow and the conversion of raw images into vehicle commands by the image processing module 18. The components and modules have been discussed in more detail above. The dashed lines represent optional steps and / or modules. Raw images are acquired by the imaging device 10, 12 (e.g., a camera that can be either stereo or monocular). Some embodiments of the present invention require only one imaging device. Raw images are processed by the image adjuster 101 to obtain defect-corrected images. Defect-corrected images are processed by the image data evaluation device 25 to determine, based on an image quality score for the corrected image, whether the image should be further used by the alignment module 24. Defect-corrected images are also processed by the container identification module 20 and the material profile module 27.Defect-corrected images can also be used in conjunction with disparity images by the chute tracking device 22 if a disparity image generator 103 is present. Otherwise, the chute tracking device 22 uses exclusively the data stored in the vehicle module 1000, in particular data about the transferring vehicle 91, the dimensions of the chute 89, and the kinematic model of the chute. Furthermore, the chute tracking device 22 requires data about the vehicle status information, in particular about the travel speed of the transferring vehicle, the angle(s) of the chute, the on / off status of the drive of the auger 47, and the relative GPS position of the receiving vehicle 79 if machine synchronization exists. The output data of the discharge chute locating device 22 are transmitted to the container identification module 20 and, in conjunction with error-corrected images and, if applicable,also disparity images from the container identification module 20 are processed to determine the location and dimensions of the container. Defect-corrected images and (if available) also disparity images are processed by the material profile module 27 in conjunction with container location and dimension data received from the container identification module 20 to generate a fill profile of the container 85. The alignment module 24 processes data generated by the container identification module 20 or material profile module 27 in conjunction with the vehicle status information to generate vehicle commands, such asfor the travel speed / steering of the transferring vehicle 91, for the position of the discharge chute, for the on / off status of the auger drive, and the travel speed / steering of the receiving vehicle 79 when machine synchronization exists in order to reposition the end of the discharge chute 87 over the appropriate open area of ​​the hopper 85 and thus establish an even, uniform distribution of the agricultural material in the hopper 85.

[0105] While the disclosure has been described in detail and reference has been made to specific embodiments of the disclosure, it will be apparent to one of ordinary skill in the art that various changes may be made to this disclosure without departing from the spirit and scope of the embodiments. Thus, it is intended that the present disclosure cover all changes and modifications to this disclosure, as long as they come within the scope of the appended claims and their equivalents.

Claims

[1] A method for enabling the transport of material from a transferring vehicle (91) having a discharge chute (89) with a material conveying arm or an end (87) of a discharge chute (89) into a receiving vehicle (79) having a container (81) for storing transported material, the method comprising the following steps: a. capturing image data from at least one imaging device (10) directed at the container (81); b. identifying and locating the container (81) from the acquired image data using a chute locator (22) of an image processing module (18); c. identifying the discharge chute (89) from the acquired image data using the discharge chute locator (22) of the image processing module (18); d. detecting a representation of the fill level or volume distribution of material in the container (81); e. determining a position of the discharge chute (89) or the end (87) of the discharge chute (89) relative to the container (81) based on the identified location of the container (81) and the identified discharge chute (89); f. aligning the material conveying arm or the end (87) of the discharge chute (89) over a current target area of ​​the material-receiving container (81) based on the determined position of the discharge chute (89) or the chute end (87) relative to the container (81); g. Determining further target areas of the container (81) for receiving material, based on the representation of the fill level or the volume distribution of material in the container (81); h. Transport of material from the transferring vehicle (91) to the current destination area of ​​the container (81) in the receiving vehicle (79); i. Detecting the time at which the current target area of ​​the container (81) is filled with material; j. Repeating steps f to i until the subsequent target areas of the container (81) are filled; and k. Completion of the transport of material from the transferring vehicle (91) to the receiving vehicle (79). [2] The method according to claim 1, characterized by that the representation of the fill level or volume distribution of material in the container (81) is one-dimensional. [3] The method according to claim 1, characterized by that the representation of the fill level or volume distribution of material in the container (81) is two-dimensional. [4] The method according to claim 1, characterized by that the representation of the fill level or volume distribution of material in the container (81) is three-dimensional. [5] The method according to any one of the preceding claims, characterized bythat the step of detecting the representation of the fill level or volume distribution of material in the container (81) further comprises the following steps: Receiving data from a distributed level sensor; and Generating a representation of the fill level or volume distribution of material in the container (81) based on the messages from the distributed fill level sensor. [6] The method according to any one of the preceding claims, characterized by that the step of detecting the representation of the fill level or volume distribution of material in the container (81) further comprises the following steps; Receiving error-corrected image data from the at least one image processing device; Generating disparity image data based on the error-corrected image data; and Generating a representation of the fill level or volume distribution of material in the container (81) using range data based on disparity image data. [7] The method according to claim 1, characterized by that the step of determining further target areas of the container (81) which are to receive material according to the representation of the fill level or the volume distribution of the material in the container (81) comprises the following steps; Development of a target area matrix for the container (81), characterized by that each target area of ​​the matrix is ​​identified by a predefined group of coordinates related to the container (81); Identification of filled or unfilled target areas of the matrix based on the representation of the fill level or volume distribution of material in the container (81); and Determining the predefined set of coordinates of a target area to be subsequently filled and the predefined set of coordinates of the current target area over which the material conveying arm or the end (87) of the discharge chute (89) is located. [8] The method according to claim 7, characterized by in that the step of aligning the material conveying arm or the end (87) of the discharge chute (89) over a current target area of ​​the material receiving container (81) comprises the step of actuating a control mechanism to position the material conveying arm or the end (87) of the discharge chute (89) over the predefined set of coordinates of a target area to be subsequently filled. [9] The method according to claim 8, characterized bythat the step of actuating a control mechanism for positioning the material conveying arm or the end (87) of the discharge chute (89) over the predefined set of coordinates of a further target area to be filled comprises the following steps: Determining a current lateral discharge angle of the material conveying arm or the end (87) of the discharge chute (89); Converting the predefined set of coordinates between the current target area and the further target area to be filled into a change in the current lateral launch angle, thereby forming a new current lateral launch angle; Transmission of the new lateral discharge angle to a chute control of the transferring vehicle (91); and Operating the chute control to move from the current lateral discharge angle to the new lateral discharge angle; whereby the material conveying arm or the end (87) of the discharge chute (89) is aligned over the further target area to be filled. [10] The method according to claim 7, characterized by in that the step of aligning the material conveying arm or the end (87) of the discharge chute (89) over a current target area of ​​the material receiving container (81) comprises the step of changing the lateral offset between the receiving vehicle (79) and the transferring vehicle (91) with the aim of moving the material conveying arm or the end (87) of the discharge chute (89) from its position over the current target area to the position over the target area to be subsequently filled. [11] The method according to claim 10, characterized by that the step of changing a lateral offset between the receiving vehicle (79) and the transferring vehicle (91) comprises the following step: Determining a current lateral offset between the transferring vehicle (91) and the receiving vehicle (79); Converting the predefined set of coordinates between the current target area and the further target area to be filled into a change in the current lateral offset, resulting in a new lateral offset; transmitting the new lateral offset to a control controller of the transferring vehicle (91); and Controlling the transferring vehicle (91) to establish a new lateral offset between the receiving vehicle (79) and the transferring vehicle (91); whereby the material conveying arm or the end (87) of the discharge chute (89) is aligned over the further target area to be filled. [12] The method according to claim 7, characterized byin that the step of aligning the material conveying arm or the end (87) of the discharge chute (89) over a current target area of ​​the material receiving container (81) includes the step of changing the longitudinal offset between the receiving vehicle (79) and the transferring vehicle (91) with the aim of moving the material conveying arm or the end (87) of the discharge chute (89) from its position over the current target area to the position over the further target area to be filled. [13] The method according to claim 12, characterized by that the step of changing a longitudinal offset between the receiving vehicle (79) and the transferring vehicle (91) comprises the following steps: Determining an initial longitudinal offset between the transferring vehicle (91) and the receiving vehicle (79); Converting the predefined set of coordinates between the current target area and the further target area to be filled into a change in the current longitudinal offset, thereby forming a new longitudinal offset; transmitting the new longitudinal offset to a drive control of the transferring vehicle (91) or a brake switch of the transferring vehicle (91); and Accelerating or decelerating the transferring vehicle (91) to establish the new longitudinal offset between the receiving vehicle (79) and the transferring vehicle (91); whereby the material conveying arm or the end (87) of the discharge chute (89) is aligned over the target area to be subsequently filled. [14] The method according to any one of the preceding claims, characterized bythat the step of transporting material from the transferring vehicle (91) to the current target area of ​​the container (81) of the receiving vehicle (79) comprises the step of actuating an auger. [15] The method according to any one of the preceding claims, characterized by that the step of transporting material from the transferring vehicle (91) to the current target area of ​​the container (81) of the receiving vehicle (79) comprises the step of opening the discharge chute (89). [16] The method according to any one of the preceding claims, characterized by that the step of determining the time at which the current target area of ​​the container (81) is filled with the material comprises the step of detecting the representation of the fill level or the volume distribution of material in the current target area of ​​the container (81). [17] The method according to claim 16, characterized bythat the step of detecting a representation of the fill level or volume distribution of material in the current target area of ​​the container (81) further comprises the following steps; Receiving data from a distributed level sensor; and Generating a representation of the fill level or volume distribution of the material in the current target area in the container (81) based on the data from the distributed fill level sensor. [18] The method according to claim 16, characterized by that the step of detecting a representation of the fill level or volume distribution of material in the current target area of ​​the container (81) further comprises the following steps; Receiving error-corrected image data from the at least one image processing device; Generating disparity image data based on the error-corrected image data; and Generating a representation of the fill level or volume distribution of material in the container (81) using range data based on disparity image data. [19] The method according to claim 14, characterized by that the step of terminating the transport of material from the transferring vehicle (91) to the receiving vehicle (79) at the time when the current target area of ​​the container (81) is filled with the material comprises the step of deactivating the auger. [20] The method according to claim 15, characterized by that the step of terminating the transport of material from the transferring vehicle (91) to the receiving vehicle (79) comprises the step of closing the discharge chute (89). [21] The method according to claim 1, characterized by that the material is of agricultural or mineral origin. [22] The method according to any one of the preceding claims, characterized bythat the step of aligning the material conveying arm or the end (87) of the discharge chute (89) over a current target area of ​​the container (81) to be filled with material comprises: Generating command data used by at least one vehicle controller or chute controller, Transmitting the command data to the transferring vehicle (91) for the purpose of controlling the transferring vehicle (91) in accordance with the receiving vehicle (79) such that the discharge chute (89) is aligned within a central region (83) or the opening of the grid pattern (82) of the periphery of the container (81); wherein the command data is based on the position of the discharge chute (89) relative to the container (81). [23] The method according to claim 6 or any of the claims dependent on claim 6, characterized bythat the step of receiving error-corrected image data from the at least one image processing device further comprises receiving error-corrected image data from an image processing device in the transferring vehicle (91), wherein the receiving vehicle (79) is not equipped with image processing devices. [24] The method according to claim 6 or any of the claims dependent on claim 6, characterized by that the step of receiving error-corrected image data from the at least one image processing device further comprises: Receiving a first set of error-corrected image data from a first image processing device in the transferring vehicle (91); and Receiving a second set of error-corrected image data from a second image processing device in the transferring vehicle (91), wherein the receiving vehicle (79) is not equipped with image processing devices. [25] The method according to claim 18 or any of the claims dependent on claim 18, characterized by that the step of receiving error-corrected image data from the at least one image processing device further comprises receiving error-corrected image data from the at least one image processing device in the transferring vehicle (91), wherein the receiving vehicle (79) is not equipped with image processing devices. [26] The method according to claim 18 or any of the claims dependent on claim 18, characterized by that the step of receiving error-corrected image data from the at least one image processing device further comprises: Receiving a first set of error-corrected image data from a first image processing device in the transferring vehicle (91); and Receiving a second set of error-corrected image data from a second image processing device in the transferring vehicle (91), wherein the receiving vehicle (79) is not equipped with image processing devices. [27] The method according to any one of the preceding claims, characterized by that the identification of the discharge chute (89) from the image data using the discharge chute locating device (22) of the image processing module (18) further comprises determining an orientation of the discharge chute (89), and for this purpose comprises estimating or determining movement commands based on the acquired image data at regular intervals.

Images