System and method for improving the robustness of an automated unloading system

Abstract

System (11, 111, 211) for facilitating the transfer of agricultural product from a transferring vehicle (2) to a receiving vehicle (6), wherein the system (11, 111, 211) comprises: a discharge pipe (47), comprising: a length, a rotation end which is rotatably connected to the transfer vehicle and a discharge end (13B), an actuator for rotation to rotate the unloading pipe (47) about a vertical axis of rotation from a rest position to a maximum position along a maximum arc, a vertical actuator to raise and lower the discharge end (13B) vertically between a rest position height and a maximum position height, one or more sensors to determine a current rotation angle φ of the unloading pipe (47) and a current elevation angle θ of the unloading pipe (47), and a screw conveyor drive to transfer agricultural product from the transferring vehicle (2) through the unloading pipe (47) into the receiving vehicle (6); wherein the receiving vehicle (6) comprises a driven part for driving the receiving vehicle (6) and a storage part (4) for storing agricultural product, wherein the storage part (4) comprises a front, a back and two longitudinal sides joined at corners, each side having a top edge; a first imaging device (10) connected to the unloading pipe (47) and pointing forward towards the storage part (4) of the receiving vehicle (6), wherein the first imaging device (10) collects initial image data; a second imaging device (12) connected to a chassis of the transferring vehicle (2) and pointing towards the storage part (4) of the receiving vehicle (6), wherein the second imaging device (12) collects second image data; and an image processing module (18) associated with the transferring vehicle (2), wherein the image processing module (18) comprises a processor for executing software to position the discharge end (13B) of the unloading pipe (47) in relation to the front or rear or top edge of the storage part (4) of the receiving vehicle based on a filling strategy which uses the first image data and the second image data.

Description

CROSS-REFERENCE TO RELATED REGISTRATIONS BACKGROUND OF THE INVENTION

[0001] During unloading, the driver of a tractor pulling a grain trailer will generally try to match the speed of the combine harvester as it harvests to optimize efficiency. This can sometimes prove difficult, especially for inexperienced tractor drivers. Sometimes the tractor driver will maintain a constant speed (below the optimal harvesting speed) during unloading, and the harvester driver (combine or forage harvester) will adjust the relative position between the vehicles by increasing or decreasing the combine's speed. The unloading process requires much more precision as the grain trailer begins to fill. The goal is for the tractor and harvester drivers to choreograph their movements to achieve a smooth, even filling of the grain trailer.This invention automates this interaction by, for example, calculating the position of the screw conveyor, the relative position of the trailer and the harvesting vehicle, and the point of entry of the harvested crop into the trailer based on scanning, where the grain is unloaded into the grain trailer, scanning and profiling the fill level of the grain trailer, and executing a strategy to fill the trailer evenly to a desired fill level.

[0002] Sometimes circumstances arise in which it is difficult or impossible for the system to determine the relative position between the harvesting vehicle and the grain trailer, and therefore where the grain is unloaded into the grain trailer. Several techniques can be used to mitigate the negative impact on the system when such circumstances occur. DE10 2010 004 648 A1 discloses a harvesting machine, in particular a forage harvester, with a device for transferring harvested crop into a loading container, wherein the transfer device can be moved into different positions by means of at least one actuator in order to align a stream of harvested crop exiting the transfer device. DE 603 ​​19 618 T2 relates to a discharge chute control system for controlling and aligning a harvested crop discharge chute of a material collection vehicle with respect to a separate material loader moving along the length of the collection vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] For ease of understanding and simple application of the present invention, the invention is now described for the purposes of illustration and not limitation in connection with the following figures, wherein: Fig. 1A and Fig. 1B Front views of a combine harvester or a forage harvester are shown, illustrating many features of the present invention; Fig. 2 is a perspective view of a combine harvester unloading grain into a trailer; Fig. 3 is a side view of a combine harvester, illustrating the repositioning of a first imaging device and showing a screw conveyor moving up and down; Fig. 4 is a side view of a tractor pulling a trailer; Fig. 5 is a process flow diagram of an embodiment of the present invention for adjusting the relative displacement (offset) between the combine harvester and the grain trailer; Fig. 6 a process flow diagram of an embodiment of the present invention for overcoming conditions that reduce system performance; Fig. 7 is a top view of the trailer, illustrating the filling zones of a filling strategy and rotation limits of the screw conveyor; Fig. 7A is a rear view of a combine harvester along a trailer, illustrating minimum clearances and auger conveyor angle to avoid contact of the auger conveyor with the top edge of the trailer; Fig. 8 a block diagram of an embodiment of a stereoscopic vision system for a harvesting vehicle for managing the unloading of agricultural product from the harvesting vehicle (e.g., combine harvester); Fig. 9 a block diagram of a further embodiment of a stereo vision system for a harvesting vehicle for managing the unloading of agricultural product from the harvesting vehicle (e.g., a self-propelled forage harvester); Fig. 10 a block diagram of an embodiment of a system for a receiving vehicle (without stereo vision) for managing the unloading of agricultural product from a vehicle and Fig. 11 is a process flow diagram of 'Machine Sync' logic for controlling the relative position between the combine harvester and the grain trailer. DETAILED DESCRIPTION OF THE INVENTION

[0004] In accordance with one embodiment, shows Fig. 8 A system 11 for a harvesting vehicle for managing the unloading of agricultural product from the harvesting vehicle (e.g., combine harvester) into a receiving vehicle (e.g., grain trailer or wagon). In one embodiment, the system 11 comprises a first imaging device 10 and a second imaging device 12 coupled to an image processing module 18. The first imaging device 10 may comprise a primary stereo camera, while the second imaging device 12 may comprise a secondary stereo camera. For example, the first imaging device 10 or the second imaging device 12 is mounted at a sufficiently high elevation above ground level to provide some view into the storage container 4 (e.g., grain trailer) or sufficient view of the interior of the storage container 4 and its contents to determine a profile, distribution, or level of agricultural product (e.g., grain) within a volume or part of the volume defined by the container 4.

[0005] The image processing module 18 can be connected, directly or indirectly, to lamps 15 on a vehicle (e.g., harvester) for illuminating a storage container and / or unloading pipe end (e.g., 13A in Fig. 1) or be coupled to illuminate a field of view of the first imaging device 10, the second imaging device 12, or both for capturing raw images (e.g., of sufficient brightness, contrast, and color reproduction). For example, the image processing module 18 can control drivers or switches that, in turn, control the activation or deactivation of the lamps 15 on the harvesting vehicle. The image processing module 18 can control the lamps 15 on the vehicle to illuminate the storage container (e.g., 4 in Fig. 1), of the discharge pipe end (e.g., 13A in Fig. 1) or both activate when a light sensor indicates that the ambient light intensity is below a certain threshold. In one configuration, the light sensor comprises a photocell, a photoresistor, a light-sensitive device, or a cadmium sulfide cell. The unloading pipe also includes a length and a rotating end that is pivotally connected to the transferring vehicle, allowing the unloading pipe to rotate about a vertical axis.

[0006] In one embodiment, the rotation system 16 of the screw conveyor may comprise: (1) a rotation angle sensor for sensing a rotation angle of the discharge pipe (e.g., φ in Fig. 1) or other angles of the unloading pipe / screw conveyor 47 with respect to one or more axes of rotation and (2) an actuator for moving the unloading pipe / screw conveyor 47 to change the rotation angle φ of the unloading pipe or other angles of the unloading pipe; hence, the position of the unloading pipe with respect to the receiving vehicle 6 or its storage container 4. The actuator of the rotation system 16 for the screw conveyor may comprise one or more motors, a linear motor, an electro-hydraulic device, a mechanical device with ratchet or cable actuation, or other device for moving the unloading pipe 89 or the unloading pipe end 87.The angle of the unloading pipe or the rotation angle φ of the unloading pipe may include a simple angle, a compound angle, or multidimensional angles measured with respect to any of the following: a reference axis parallel to the direction of travel of the harvesting vehicle, a generally vertical axis, a generally horizontal axis, or an axis generally orthogonal to at least one of the generally vertical axis and the generally horizontal axis.

[0007] Where the system 11 of FIG. is applied to a combine harvester or a forage harvester, the unloading auger / screw conveyor 47 can be controlled in one or more dimensions (e.g., rotation or movement). In one configuration, the rotation system 16 of the screw conveyor (of the forage harvester or combine harvester) controls a rotation angle φ of the unloading auger / screw conveyor 47 in a generally horizontal plane or about a generally vertical axis. In another configuration, the rotation system 16 of the screw conveyor or the controller of the unloading auger could control one or more of the following angles: (1) the rotation angle φ of the unloading auger in a generally horizontal plane, (2) the tilt angle in a relatively vertical plane, and (3) the flap angle (e.g., forage harvester), where the rotation angle, tilt angle, and flap angle are associated with mutually orthogonal axes.In one configuration, by controlling the rotation angle, the vehicle controller (e.g., 46 of . Fig. 8, Fig. 54 of Fig. 9) automatically extend or retract the unloading pipe / screw conveyor 47 (e.g., unloading arm of the screw conveyor) when appropriate (e.g., when the unloading of the agricultural product is complete).

[0008] The vehicle controller 46 controls the rotation of the screw conveyor 47 for transferring or moving the agricultural product from the harvesting vehicle 2 to the receiving vehicle 6. The vehicle controller 46 can provide a data message indicating whether the screw conveyor 47 is active or inactive for unloading agricultural product from the harvesting vehicle. The screw conveyor 47 can comprise a screw conveyor, an electric motor for driving the screw conveyor, and a rotation sensor for sensing the screw conveyor or its associated shaft. In one embodiment, the screw conveyor 47 is associated with a container for storing agricultural product (e.g., a grain tank) of a harvesting vehicle 2 (e.g., a combine harvester).

[0009] If the image processing module 18 or another sensor determines that the container 4 has reached a target fill level (e.g., full or any percentage or fraction of capacity), the image processing module 18, the vehicle controller 46, or the screw conveyor rotation system 16 can automatically shut off the unloading screw conveyor.

[0010] The image processing module 18 can include a controller, a microcomputer, a microprocessor, a microcontroller, an application-specific integrated circuit, a programmable logic array (PLA), a logic device, an arithmetic logic unit, a digital signal processor or other data processor, and supporting electronic hardware and software.

[0011] In one embodiment, the image processing module 18 comprises a container identification module 20 and an alignment module 24.

[0012] The image processing module 18 can be associated with a data storage device 19. The data storage device 19 can comprise, for example, electronic storage, non-volatile random-access memory, a magnetic disk drive, an image disk drive, a magnetic storage device, or an optical storage device. If the container identification module 20 and the alignment module 24 are software modules, they are stored within the data storage device 19.

[0013] The container identification module 20 identifies a set of two-dimensional or three-dimensional points (e.g., in Cartesian or polar coordinates) in the real world that represent at least one section of the container perimeter (e.g., front edge or rear edge) of the storage section (e.g., trailer 4 in Fig. 1) Define. The set of two-dimensional or three-dimensional points corresponds to pixel positions in images collected by the first imaging device 10, the second imaging device 12, or both. The container identification module 20 can use or retrieve container reference data.

[0014] The container reference data includes one or more of the following: reference dimensions, reference outline, drawings, models, layout, and configuration of the container 4, such as the container perimeter and edges; reference dimensions, reference outline, drawings, models, layout, and configuration of the entire storage unit 4 of the receiving vehicle 6; storage unit wheelbase, storage unit turning circle, and configuration of the trailer hitch of the storage unit 4 of the receiving vehicle 6. The container reference data can be stored and retrieved by the data storage device 19 (e.g., non-volatile electronic storage). For example, the container reference data can be stored, retrieved, or indexed by a corresponding identifier of the receiving vehicle in the data storage device 19 of the harvesting vehicle system 11.For each identifier of the receiving vehicle, corresponding unique container reference data can thus be stored in the data storage device 19.

[0015] In one embodiment, the harvesting vehicle 2 receives a data message from the receiving vehicle 6, in which a vehicle identifier of the receiving vehicle is transmitted regularly (e.g., periodically). In another embodiment, the harvesting vehicle 2 queries the receiving vehicle 6 for its vehicle identifier or establishes a communication channel between the harvesting vehicle 2 and the receiving vehicle 6 via the wireless communication devices (48, 148) in preparation for the unloading process. In yet another embodiment, the receiving vehicle 6 sends its vehicle identifier to the harvesting vehicle 2 when the receiving vehicle 6 approaches the harvesting vehicle 2 within a certain radial distance. In yet another embodiment, only one known configuration of the receiving vehicle with a corresponding harvesting vehicle is used, and the container reference data is stored or saved in the data storage device 20.In the latter embodiment, the harvesting vehicle 2 is programmed, at least temporarily, solely for receiving vehicles with identical containers that are identical in dimensions, capacity, proportion and shape.

[0016] If the linear orientation of a set of pixels in the collected image data corresponds to one or more edges of the perimeter of the container 4, as prescribed by the container reference data, the container's position has been identified. A central area or zone of the container opening of container 4 can be identified by dividing the distance (e.g., shortest distance or surface normal distance) between opposite sides of the container, or by identifying the corners of the container and where diagonal lines sweeping across the corners intersect, among other methods.

[0017] The alignment module 24 estimates movement commands at regular intervals to maintain the alignment of the unloading pipe end 13A above the target of the container 4 for unloading agricultural product. The alignment module 24 can send commands to the harvesting vehicle 2 regarding its speed, velocity, or heading to maintain the alignment of the harvesting vehicle's position relative to the receiving vehicle. For example, the alignment module 24 can send a steering or heading command to the steering controller 32, a braking or deceleration command to a braking system 34, and a drive, acceleration, or torque command to a drive controller 40. Furthermore, similar command data can be sent via the wireless communication devices (48, 148) to the receiving vehicle 6 for observation purposes or to control the receiving vehicle via its steering system controller 32 or its brake controller 36.

[0018] The drive controller 40 can enable a uniform distribution of the agricultural product in the container 4 by accelerating or decelerating the receiving vehicle 6 or its drive unit 75, in order to redistribute or shift the agricultural product evenly within the container 4. In one example, a drive controller 40 temporarily increases its ground speed or, alternatively, the relative speed of the receiving vehicle 6 relative to the harvesting vehicle 2 if an image processing module 18 senses that a front volume of the storage unit is currently filled to a target level (with agricultural product) or until the entire trailer 4 reaches the target fill level.Conversely, the drive controller 40 temporarily reduces its speed over ground or alternately a relative speed of the receiving vehicle relative to the harvesting vehicle if an image processing module 18 senses that a rear volume of the storage part is currently filled to a target level (with agricultural product) or until the entire storage part 93 reaches the target fill level.

[0019] The image processing module 18 provides data to a user interface processing module 26, which, directly or indirectly, provides status message data and performance message data to a user interface 44. As in Fig. As illustrated in Figure 8, the image processing module 18 communicates with a vehicle data bus 31 (e.g., Controller Area Network (CAN) data bus).

[0020] In one embodiment, a destination receiver 42, a first wireless communication device 48, a vehicle controller 46, a steering controller 32, a brake controller 36, and a drive controller 40 are capable of communicating via the vehicle data bus 31. The steering controller 32 is coupled to a steering system 30 of the harvesting vehicle; the brake controller 37 is coupled to the brake system 34 of the harvesting vehicle; and the drive controller 40 is coupled to the drive system 38 of the harvesting vehicle.

[0021] In one configuration, a user interface 44 is provided for entering container reference data or dimensional parameters relating to the receiving vehicle. For example, the container reference data or dimensional parameters include a distance between a trailer hitch (which connects the drive unit 75 and the storage unit 93) and the front wheel pivot axis of the storage unit 93 of the receiving vehicle 6.

[0022] System 11 of Fig. System 8 is well suited for use on a combine harvester or forage harvester as the harvesting vehicle. System 11 of Fig. 8 can be used with a second system (211 of Fig. 10) Communicate and cooperate at receiving vehicle 6 to coordinate the relative alignment of harvesting vehicle 2 and receiving vehicle 6 during unloading or transfer of product from the harvesting vehicle. Same reference numbers in Fig. 8 and Fig. 9 indicate identical elements.

[0023] System 111 of Fig. 9 is the system 11 of Fig. 8 similar; except that the system 111 of Fig. 9 further comprises a device data bus 58, a gateway 29, and vehicle controllers (50, 54) which are coupled to the vehicle data bus 60 for the lamps 52 and the adjuster 56 of the unloading pipe end. The vehicle controller 50 controls the lamps 52; the vehicle controller 54 controls the adjuster 56 of the unloading pipe end to move or adjust the orientation or angle of the unloading pipe or screw conveyor 47, or its unloading pipe end 13B. The adjuster 56 of the unloading pipe can comprise an actuator for moving or adjusting the unloading pipe 89 and one or more sensors for measuring the angle of the unloading pipe, the orientation, or the position of the unloading pipe 89. For example, the adjuster 56 of the unloading pipe or its actuator may include a servo motor, electric motor or an electro-hydraulic mechanism for moving or adjusting the unloading pipe 89.

[0024] Where the system 111 of Fig. When applied to a self-propelled forage harvester, the vehicle controller 54 and the unloading auger adjuster 56 can control or adjust the unloading auger or screw conveyor 47 in multiple dimensions, such as two or three dimensions. For example, the vehicle controller 54 or the unloading auger controller can control one or more of the following angles: (1) rotation angle of the unloading auger in a generally horizontal plane, (2) tilt angle in a relatively vertical plane, and (3) flap angle, where the rotation angle, tilt angle, and flap angle are associated with mutually orthogonal axes. For a forage harvester, the unloading auger 47 (e.g., arm of the unloading screw conveyor) is not usually retracted, and the flow of agricultural product from the unloading auger 89 is generally continuous during the harvesting process.

[0025] If a container 4 of the receiving vehicle 6 is full (or approaching full) with agricultural product (e.g., from a transfer operation), as detected by one or more sensors (e.g., mass or optical sensors) on the receiving vehicle 6 to detect a mass, weight, or volume of agricultural product in the container 4, the imaging system 18 of the harvesting vehicle 2 or the sensors of the receiving vehicle can, via the wireless communication devices 48, 148, display the full state, fill level, or container full of the container 4 to the driver (of the harvesting vehicle 2) on the user interface 44. In response to the full or nearly full state (e.g., approximately 90 percent or more of the full capacity) of the container 4 (e.g., in the context of a forage harvester such as the harvesting vehicle 2), the system 111 of Fig. 9: (1) maintain a final position and orientation of the discharge pipe 47 to continue unloading agricultural product at the same location, or (2) pivot the discharge pipe 47 or the discharge pipe end 13A back and forth in a continuous or stepwise motion to distribute the product evenly in the container 4.

[0026] In one configuration, the device data bus 58 could comprise a Controller Area Network (CAN) device data bus. Similarly, the vehicle data bus 60 could comprise a Controller Area Network (CAN) data bus. In an alternative embodiment, the device data bus 58, the vehicle data bus 60, or both could comprise an ISO (International Organization for Standardization) data bus or ISOBUS, Ethernet, or another data protocol or communication standard.

[0027] The gateway 29 supports secure or controlled communication between the device data bus 58 and the vehicle data bus 60. The gateway 29 includes a firewall or other security device that could restrict or prevent a network element or device on the device data bus 58 from communicating (e.g., unauthorized communication) with the vehicle data bus 60 or a network element or device on the vehicle data bus 31, unless the network element or device on the device data bus 58 follows a certain security protocol, handshake, password and key, or other security measure. Furthermore, in one embodiment, the gateway 29 could encrypt communications to the vehicle data bus 60 and decrypt communications from the vehicle data bus 60 if a suitable encryption key is entered or if other security measures are met.The gateway can allow network devices on the device data bus 58 to communicate via an open standard or third-party hardware and software suppliers, whereas the network devices on the vehicle data bus 60 are provided solely by the manufacturer.

[0028] In Fig. 9. A location receiver 42, a user interface 44, a user interface processing module 26, and the gateway 29 are connected to the device data bus 58. The vehicle controllers 50 and 54 are connected to the vehicle data bus 60. In turn, the vehicle controllers 50 and 54 are connected, directly or indirectly, to lamps 15 on the harvesting vehicle and the unloading auger 89 of the harvesting vehicle (e.g., self-propelled forage harvester). Although the system of Fig. 9 is well suited for use or installation on a self-propelled forage harvester, the system can be of Fig. 9. Also apply to combine harvesters, harvesting machines, or other heavy equipment.

[0029] System 11 of Fig. 8 and the System 111 of Fig. 9 apply to harvest vehicle 2, whereas the system of Fig. 10 applies to the acceptance vehicle 6. Same reference numbers in Fig. 9 and Fig. Figures 10 show identical elements. As already stated, the harvesting vehicle 2 comprises a combine harvester, forage harvester, self-propelled forage harvester, vehicle, or heavy equipment that collects or harvests product for transfer to the receiving vehicle. In one embodiment, the receiving vehicle 6 comprises a powered part (e.g., a tracker in Fig. 1) and a storage section (e.g., 4 in Fig. 1) for storing product transferred from the harvesting vehicle 2. The receiving vehicle 6 may comprise a combination of a tractor and a grain trailer or grain wagon, where the tractor is an illustrative example of the driven part 6 and where the grain trailer is an illustrative example of the storage part 4. In one embodiment, illustrated Fig. 10 a driven part (e.g., tractor) without a first imaging device 10 or a second imaging device 12 on the driven part 75. Same reference numbers in Fig. 9 and Fig. 10 indicate identical elements.

[0030] System 211 of Fig. 10 is part of the system of Fig. 9 similar, except that the system of Fig. 10 the first imaging device 10, the second imaging device 12, the image processing module 18, the user interface 44, the user interface processing module 26, the vehicle controllers 50, 54, the lamps 52 and the unloading pipe 56 of Fig. 9 is deleted. The system 211 of Fig. 10 includes a second wireless communication device 148, for example, with the first communication device 48 of Fig. 8 or Fig. 9 to communicate. The wireless devices 48, 148 can exchange or communicate position data, relative position data, command data or control data for controlling, adjusting or coordinating the position and orientation of the vehicles; in particular, the position and orientation of the unloading pipe 47 or the unloading pipe end 13A above the opening of the container 4. The second wireless communication device 148 is coupled to the vehicle data bus 31. In Fig. 10, the system 211 can be used for an acceptance vehicle 6 in conjunction with the system (11 or 111) of the harvesting vehicle 2 of Fig. 8 or Fig. 9 can be used.

[0031] The image processing module 18 estimates a distance or area from the first imaging device 10, the second imaging device 12, or both, to the pixels or points located on the container circumference or container edge. For example, the image processing module 18 can use the disparity map or disparity image to estimate a distance or area from the first imaging device 10, the second imaging device 12, or both, to the pixels or points located on the container circumference 81, on the container edges 181, the container opening 83, near any of the aforementioned objects, or elsewhere.

[0032] The system 11,111,211, for example, implements a filling strategy for the container by changing the relative speed, velocity, or acceleration of the harvesting vehicle and the receiving vehicle (i.e., via the ISO Class 3 interface) to promote even or uniform filling of the container 4. As an illustrative example of a "front-to-back" filling strategy, the system 11,111,211 causes the receiving vehicle to generally maintain a constant front / back distance relative to the harvesting vehicle (i.e., combine harvester) such that the agricultural product fills the front volume of the trailer and places weight on the drawbar of the connection between the drive unit 6 and the storage unit.When the front volume of container 4 becomes full or reaches a target volume or mass of agricultural product, the system 11, 111, 211 can command the receiving vehicle to temporarily increase its speed or velocity or accelerate relative to the ground (or relative to the harvesting vehicle) via the drive controller 40, so that the agricultural product flowing from the unloading pipe 47 continues to move towards or fall into the rear part of container 4 (i.e., due to the acceleration force of the receiving vehicle).During or in preparation for acceleration, if authorized by the estimated alignment of the unloading pipe end 13A and the container circumference or container edges by the alignment module 24 or the image processing module 18, the vehicle controller 46 may temporarily suspend the rotation of the auger 47 to prevent spillage of agricultural product or missing the container 4, or the alignment module 24 may provide command data via the wireless communication devices 48, 148 such that the drive controller 40 of the harvesting vehicle simultaneously accelerates (i.e., at the same magnitude and direction as the receiving vehicle) to maintain alignment (i.e., substantially the same alignment) between the container circumference or container edge and the unloading pipe end 13A.The system 11, 111, 211, or the drive controller 40 can temporarily increase the relative speed of the receiving vehicle relative to the harvesting vehicle if the imaging module 18 senses that a front volume of the trailer 4 is currently filled to a target level, or until the entire storage section 93 reaches the target fill level. For example, the system 11, 111, 211 can repeat the process of temporarily increasing the speed or velocity of the receiving vehicle relative to the harvesting vehicle during each sampling interval that the image processing module 18 detects is that the front volume of the container 4 is currently filled to a target level, or until the entire container 4 reaches the desired fill level.

[0033] In another configuration, as soon as container 4 of the receiving vehicle begins to fill, the agricultural product in container 4 was, is, or will become visible to the first imaging device 10, the second imaging device 12, or both. Next, imaging device 10 or 12 can scan or profile the height or level of the agricultural product in container 4. Based on an accurate profile of the agricultural product level in container 4, the system can then execute a filling strategy appropriate to the situation.

[0034] The system and method are well suited for controlling the steering and speed of the harvesting vehicle and the receiving vehicle via location receivers and wireless communication devices. Furthermore, the system and method facilitate the detection of how the receiving vehicle's container is filled, in order to adjust the relative lateral and front / back alignment between the unloading pipe 47 or unloading pipe end 13A and the container circumference or rim, thus achieving uniform filling or a uniformly distributed level of agricultural product in the container 4. Uniform filling of agricultural product within the container 4 can be achieved to minimize certain errors that would otherwise result, for example, from fatigue, inexperience, or a lack of expertise on the part of the vehicle drivers.

[0035] The system architecture and software modules described above operationally execute a system referred to herein as "Machine Sync". Fig. Figure 11 is a process flow diagram of 'Machine Sync' logic for controlling the relative position between the combine harvester and the grain trailer. S1100: The tractor driver activates the system by pressing a button on the tractor's graphical user interface (GUI). The combine harvester driver can activate the system when the tractor is close enough to the combine. Once the system is active, the tractor will accept ground speed and steering commands from the combine.

[0036] S1101: A GPS receiver mounted on the tractor sends GPS location coordinates to the tractor's CAN bus. A controller on the tractor's CAN bus, connected to a wireless communication transceiver, reads the tractor's movement dynamics (GPS position, GPS heading, and yaw rate) from the CAN bus and transmits the movement dynamics data using the wireless transmitter.

[0037] S1102: A controller on the combine harvester, connected to a wireless communication transceiver, receives the motion dynamics data sent by the tractor.

[0038] S1103: The same controller on the combine harvester reads the information about the combine's movement dynamics, which is transmitted by a GPS receiver mounted on the combine. The controller feeds the movement dynamics of the tractor and the combine harvester into a control algorithm.

[0039] S1104: The control algorithm takes the motion dynamics of the tractor and the combine harvester as inputs and outputs a ground speed and a steering angle suitable for positioning the tractor relative to the combine harvester (or maintaining its position if already suitable) such that the relative position of the tractor to the combine harvester is suitable for cooperative unloading while the combine harvester continues to harvest.

[0040] S1105: The controller on the combine harvester sends the outputs of the control algorithm to the combine harvester using the wireless transmitter.

[0041] S1106: The controller on the tractor, which is connected to a wireless communication transceiver, receives the recommended ground speed and steering angle sent by the combine harvester.

[0042] S1107: The controller on the tractor sends the recommended ground speed and steering angle via the CAN bus to the controller on the tractor, which is responsible for control with feedback of the ground speed and steering angle.

[0043] S1108: The controller on the tractor controls the tractor's speed over ground and the steering angle according to commanded values.

[0044] In the present invention, which is described in Fig. 1A and Fig. As illustrated in Figure 3, a stereo camera (first imaging device 10) is mounted on the unloading screw conveyor 47 of a combine harvester 2, looking towards a grain trailer 4 being pulled by a tractor 6 (see Figure 3). Fig. 4) is pulled when the unloading screw conveyor 47 is turned away from the chassis 7 of the combine harvester 2. The stereo camera 10 is mounted high enough to provide a view into the grain trailer 4 (see Fig. 2) to have. This gives the imaging device 10 the ability to observe and profile the surface 3 of the grain as the grain trailer 4 fills up. Fig. Figure 1B illustrates the first imaging device 12 on the unloading pipe of a forage harvester 31.

[0045] Furthermore, another perception sensor 12 (stereo camera - second imaging device) is mounted on the chassis 7 of the combine harvester 2, directly to the left of the combine harvester 2 (the side of the unloading screw conveyor 13), as shown in Fig. Figure 3 shows the view. The chassis-mounted stereo camera 12 is further away from the grain trailer 4 during the unloading process than the sensor 10 mounted on the screw conveyor. This results in a better view of the grain trailer 4 for the chassis-mounted stereo camera 12, which makes it easier to track the relative position of the grain trailer 4 to the combine harvester 2.

[0046] When the combine harvester operator activates the system (push button on the hydraulic handle), it determines whether a grain trailer 4 is correctly positioned below the collar 13A of the auger conveyor so that no grain will spill when it starts to flow. If a trailer 4 is detected and correctly positioned, the system commands the combine harvester 2 to engage the unloading auger conveyor 47. The system then monitors the relative position between the combine harvester 2 and the grain trailer 4, as well as the fill level. As soon as the grain trailer 4 begins to fill, the system uses the grain surface profile 3 to execute the filling strategy selected by the operator (back-to-front, front-to-back, front-to-back-to-front, etc.). To execute the filling strategy, the system must unload grain into areas of the trailer 4 that contain less grain.The system can adjust the unloading point by various means: commanding the auger conveyor 47 to rotate, commanding the combine harvester to change its ground speed, commanding the combine harvester operator to manually adjust the combine's speed, and / or commanding the "Machine Sync" system (discussed above and in US 7,062,381 and US 8,060,283, both incorporated herein by reference) to change the tractor 6's relative position. When the entire trailer 4 is filled to the level selected by the operator, the system shuts off the unloading auger conveyor 47. Camera orientation

[0047] Tilting the optical axis (not shown) of the chassis-mounted stereo camera 12 downwards by 10-25 degrees from the horizontal has several potential advantages. First, less of the sky is in the stereo camera's field of view. This helps to create a more uniform image intensity profile and mitigates potential dynamic range problems due to bright sunlight. Second, the lower part of the grain trailer 4 becomes more visible. This allows the stereo camera 12 to capture images of the wheel(s) 5 of the grain trailer. The wheel 5 is a feature on the grain trailer 4 that can be robustly tracked using image processing techniques. Third, tilting the stereo camera 12 downwards can reduce the accumulation of dust or other dirt on the lenses or the external window of the camera 14. Scanning the rotation of the screw conveyor

[0048] If the combine harvester 2 is equipped with a rotating position sensor (not shown) for measuring the rotation angle φ of the unloading auger conveyor, it is possible to merge the data from the auger conveyor and the chassis-mounted stereo cameras 10, 12. This allows the system to create a virtual profile of the grain level distribution in the grain trailer 4, even if the entire surface 3 of the grain is not visible to the stereo camera 10 mounted on the auger conveyor. Having this virtual profile of the entire surface 3 of the grain, the system enables it to intelligently execute a filling strategy for the grain trailer (discussed above). Setting the relative position

[0049] In general, the system will attempt to use the rotation angle φ of the auger conveyor as the primary means of adjusting the area into which the grain is unloaded in the grain trailer 4. The use of proportional control valves (not shown) on the hydraulic cylinder (not shown) that rotates the auger conveyor 47 facilitates finer adjustments to the auger conveyor's position. The end result is a trailer 4 filled with a very uniform profile. Combine harvesters are typically equipped with hydraulic cylinders with non-proportional valves. Systems on combine harvesters 2 with non-proportional valves will generally fill the grain trailer with multiple discrete stacks and will have zones in the trailer that are locally high (above the desired fill level) and locally low (below the desired fill level).

[0050] The default position for the screw conveyor 47 is at 90 degrees (perpendicular to the center line 21 of the combine harvester 2) as shown in Fig. 2 and Fig. Figure 7 shows that the screw conveyor on a conventional combine harvester 2 may only rotate a further 17 degrees clockwise. In contrast, the system is capable of performing operations up to any physical stop. The end 13B of the screw conveyor 47 describes an arc as it rotates around an axis of rotation (see Figure 7). Fig. 7) rotates. Due to these limitations, it is sometimes the case that a grain trailer 4 cannot be filled using only the rotation R of the screw conveyor. This applies especially to long grain trailers 4. Therefore, the relative speed between the combine harvester 2 and the grain trailer 4 must change in order to fill a grain trailer 4 to capacity. Fig. Figure 7 also illustrates a front-to-back zoned trailer (zones 1-6) suitable for a front-to-back filling strategy. This zoning arrangement also includes "keep-out" zones along the sides of trailer 4 to minimize crop overflow.

[0051] In particular, the Fig. Figure 7 illustrates how the lateral displacement (offset) between a combine harvester and the trailer can control the rotation limits of the automated unloading system. The path that the tip of the unloading screw conveyor's sleeve would follow as the screw conveyor rotates from its rest position to its maximum rotation angle is shown by the dashed arc. Trailer 2 has keep-out areas along its near and far edges. The system will not unload any product into these areas due to an increased likelihood of spillage. The keep-out area is the space between the two keep-out areas, extending from the front to the rear of the trailer. The system divides the keep-out area into a number of zones (6 in this example) of approximately equal size.

[0052] In this example, the combine harvester's unloading auger conveyor is rotated 90 degrees from its rest position. If the lateral displacement (offset) and the front / backward displacement between the combine and the trailer remain constant during the unloading process, the automated unloading system can discharge product into zones 1, 2, 3, and 4. The system will adjust its lower rotation limit to an angle such that the tip of the auger's sleeve remains above the fillable area of ​​the trailer.

[0053] Now consider a scenario where the front / backward displacement remains unchanged, and the lateral displacement between the combine harvester and the trailer is slightly reduced. The tip of the auger sleeve is now positioned closer to the far edge of the trailer. The lower rotation limit is a smaller angle, and the system could potentially unload into Zone 5 as well. Adjusting the relative position by regulating the speed of the combine harvester

[0054] The combine's speed can be electronically controlled if it is equipped with systems such as the Harvest Smart™ feed rate control system. With this system, the ground speed of the combine is automatically regulated by an electro-hydraulic control valve (not shown). The ground speed is increased or decreased depending on the volume of crop entering the combine, as measured by a load pressure sensor (not shown) on the combine's rotor. To prevent plugging of the combine's header 62, the combine's speed must not exceed the maximum speed permitted by the feed rate control system, regardless of the ground speed commanded by the automated unloading system. Setting the relative position through integration with the "Machine Sync" system

[0055] The speed of the tractor 6, which pulls the grain trailer 4, can be controlled via the “Machine Sync” system. This would be the preferred method for changing the relative position. If the automatic unloading system described herein is used in conjunction with the “Machine Sync” system, the combined system would behave as follows: 1. The automatic unloading system determines that an adjustment of the relative position is required to fill an area of ​​the grain trailer that is below the desired fill level. 2. The automatic unloading system sends a CAN message to the “Machine Sync” system, which commands the tractor 6 to change its position relative to the combine harvester 2. 3. The “Machine Sync” system on the combine harvester 2 wirelessly sends a message to the tractor 6, which commands a change in the relative position. 4. The “Machine Sync” system on the tractor regulates its speed and steering to execute the relative position change.

[0056] Combining automated unloading and machine sync systems could potentially automate the entire unloading process. The process steps include: 1. A tractor 6, pulling a grain trailer 4, approaches a combine harvester 2 and the “Machine Sync” system steers the tractor 6 into a position that corresponds to the speeds of the tractor 6 and the combine harvester 2. 2. The automated unloading system automatically rotates the screw conveyor 47 from its rest position on the combine harvester 2 and switches on the unloading screw conveyor 47. 3. The automated unloading system executes the filling strategy and switches off the screw conveyor 47 when it reaches the desired fill level. 4. The “Machine Sync” system determines that the tractor 6 and the grain trailer 4 have driven away from the combine harvester 2. 5. The automated unloading system moves the screw conveyor 47 into its rest position.

[0057] The end of the auger conveyor rises (increases) as it rotates clockwise and lowers (is lowered) as it rotates counterclockwise. To automatically extend or retract the auger conveyor 47, the system must ensure that the relative position between the grain trailer 4 and the combine harvester 2 is such that the auger conveyor 47 will not make contact with the grain trailer 4 as it rotates. Any contact will likely result in damage and wear to the auger conveyor assembly.

[0058] One embodiment of the present invention calculates the height of the end 13A or the sleeve of the screw conveyor 47 by the following equation: Hb=Hr+φ / 90*La tan θ where H b = Cuff height, H r = Height of the cuff in rest position, φ = Angle through which the screw conveyor has rotated from the rest position, L a = Length of the discharge screw conveyor, θ = Incline angle of the screw conveyor and its rotation. Example H r = 3 meters L a = 6.9 meters θ = 8 degrees

[0059] The equation above is an example and any suitable equation can be incorporated into the present invention.

[0060] Another major advantage of integrating the automated unloading system with the "Machine Sync" system is that the chassis-mounted stereo camera 12 is no longer needed to track the relative position of the grain trailer with the combine harvester 2. The "Machine Sync" system already tracks changes in relative position by exchanging GPS coordinates via wireless communication between the combine harvester 2 and the tractor 6. Setting the relative position by sending a message to the tractor driver on the display

[0061] If tractor 6 and combine harvester 2 do not have the "Machine Sync" system activated, but possess the necessary hardware for communication between the vehicles, the automated unloading system could wirelessly send commands to adjust the relative position of combine harvester 2 to tractor 6. Upon receiving the command, the display could emit an audible signal to the tractor driver and provide specific instructions regarding the magnitude and direction of the adjustment requested by the automated unloading system. Setting the relative position by sending a message to the combine harvester driver on the display

[0062] Another method for adjusting the relative position is for the automated unloading system to simply send a message to the combine harvester display requesting a relative position adjustment. The display would then emit an audible signal to the combine operator and provide specific instructions regarding the magnitude and direction of the requested adjustment. Target-based (see FIG. 4) and non-target-based tracking---

[0063] The system is designed to track the relative position of the grain trailer to the combine harvester without the aid of reference points (targets that are easily identifiable through image processing). However, a system that operates without the aid of targets is more difficult to achieve for several reasons. First, there is a great deal of diversity among the grain trailers the system might encounter in the field. Designing an image processing algorithm that works with all of them is challenging. Second, some trailers lack features that are useful for tracking. Third, some trailers are more difficult to track at night without an excessive amount of light from the combine harvester shining directly onto the grain trailer. Trailer identification

[0064] In addition to providing easily identifiable tracking features or targets designed to encode information regarding the identity of the grain trailer or grain wagon, a trailer or container identification module (as discussed above and in US 8,649,940, incorporated herein by reference) can be incorporated into the system architecture to identify trailer features such as edges and sides. The trailer identity could be linked to other information (grain moisture, seed variety, yield, etc.) that can be sampled by other sensors on the machine. Likewise, the trailer identity could be used to automatically load system settings such as the desired fill level and filling strategy (front to back, back to front, etc.). Lighting for nighttime operation

[0065] For the system to operate at night, the combine harvester 2 must provide additional lighting to illuminate the grain trailer 4 and make it visible to the stereo cameras 10 and 12. The lamps should be mounted in a position relative to the cameras 10 and 12 such that backscattering of light into the camera lenses is minimized. This is achieved by placing as much distance as possible between the cameras 10 and 12 and the lamps, and by orienting the lamps so that their direction differs significantly from the optical axis of the cameras. User interface

[0066] The system could also have the capability to allow the customer to view a video feed from any of the cameras 10, 12 in the system on a display. The system could also send images to the display that are annotated by the image processing software (disclosed above) to outline and highlight features on the trailer that the system is tracking. Adjusting the relative displacement (offset) between the combine harvester 2 and the grain trailer 4

[0067] Now with reference to the Fig. 2, in order to achieve optimal system performance, it is advantageous to consider the rotation angle φ of the screw conveyor and the front / backward displacement between the combine harvester 2 and the grain trailer 4. Fig. Figure 5 illustrates the process steps for an embodiment of the present invention.

[0068] S500: The automated unloading system is activated by the driver by pressing a button on the hydraulic handle. On the combine harvester, the auger conveyor automatically rotates to its maximum rotation angle (and therefore height). On the SPFH (self-propelled forage harvester), the unloading auger automatically rotates to a predefined position suitable for unloading to the left or right of the machine. The automated unloading system must then identify and track a trailer before actively controlling the machine.

[0069] S501: Once the automated unloading system is tracking a trailer, it records inputs such as images from the stereo camera, the current rotation angle of the screw conveyor or the unloading pipe, static parameters (machine dimensions, mounting location and orientation of the stereo cameras, etc.), and driver inputs to the graphical user interface (desired filling strategy and fill level). See also Fig. 6 for image processing details.

[0070] S502: If used on a combine harvester, the system performs calculations to determine the rotation limits of the auger conveyor to prevent obstruction of the trailer. This step is unnecessary on the SPFH (self-propelled forage harvester) because the incline and flap on the unloading auger can be adjusted to prevent obstruction of the trailer, and the SPFH drives the harvested material through the unloading auger at high speed. Instead, the rotation limits of the SPFH are predefined in the system software. On the combine harvester, there are three possible scenarios in which the auger conveyor can interfere with the trailer. The first scenario is that the auger conveyor's sleeve can interfere with the grain as it accumulates in the trailer. If grain builds up to the discharge point on the sleeve, damage to the auger conveyor or its drive system can occur.The following equations will determine the minimum ag = height of the screw conveyor sleeve: HB > Ht+HF - H b = Height of the cuff −Hb=Hr+φ / 90*La tan θ - H t = Height of the trailer - H f The "Height of Fill" value is used when the desired fill is to exceed the height of the trailer; otherwise, 0 should be used. Hb=Hr+φ / 90*La*tan θ>Ht+Hf φ>(90 / (La*tan θ))*(Ht+Hf−Hr)

[0071] The second scenario is that the sleeve of the screw conveyor can interfere with the edge of the trailer, and that the screw conveyor rotates. HB>HT - H b = Height of the cuff, where H b = H r + φ / 90 * La * tan θ - H t = Height of the trailer Hb=Hr+φ / 90*La*tan θ>Ht φ>(90 / (La*tan θ))*(Ht−Hr)

[0072] The third scenario is that the screw conveyor or the unloading pipe 47 may interfere with or touch the upper edge 4A of the trailer or the storage part 4, as in Fig. Figure 7A illustrates this. To determine an appropriate lateral displacement (offset) to ensure that there is no contact between the screw conveyor or discharge pipe 47 and the top edge 4A of the trailer or storage part 4, the lateral displacement must be greater than the vertical distance B from a rotational bearing point P of the discharge pipe P to the top edge 4A of the storage part (B = H). t - H p is, where Ht is the height of the top edge of the trailer 4 above the ground, H pThe processor performs the following steps to ensure there is no interference or contact between the screw conveyor / unloading pipe 47 and the top edge of the trailer: (the height of the rotational bearing point of the unloading pipe / screw conveyor above the ground) divided by the tangent of the current elevation angle θ of the unloading pipe (minimum lateral displacement). Determining a vertical distance from a rotational bearing point of the discharge pipe to the upper edge of the storage part; Calculating a tangent to the current elevation angle θ of the discharge pipe; Divide the vertical distance by the tangent to the current elevation angle θ of the discharge pipe to calculate a minimum lateral displacement; and Adjusting the current lateral displacement to a subsequent lateral displacement if the current lateral displacement is greater than the minimum lateral displacement.

[0073] First, the system uses stereo vision distance measurement to determine the trailer's relative position to the combine harvester, as well as the trailer's height and length. The system then calculates the auger's rotation limits using static parameters, the desired fill level (if set above the trailer's top level), the trailer's height, and its relative distance to the combine harvester.

[0074] S503: The system then compares the current rotation angle of the screw conveyor or unloading auger with the rotation limits. If it is determined that the screw conveyor or unloading auger is at or near the rotation limits, the process proceeds to the next step. If not, the system feeds back to gather input, and the screw conveyor's rotation limits are recalculated. It is necessary to periodically evaluate the rotation limits because changes in terrain can alter the trailer's height relative to the combine harvester or SPFH (self-propelled forage harvester) or the trailer's lateral displacement. At the combine harvester, changes in lateral displacement can affect the rotation limits because the screw conveyor rotates in an arc, and the harvested material is conveyed downwards at a low speed.

[0075] S504: The system then determines whether the harvested crop is unloaded into the front or rear zone of the trailer.

[0076] S505: Once the system determines that harvested material is being unloaded into the front or rear zone of the trailer, it determines whether it can rotate toward the center of the trailer without exceeding the rotation limits. If the system cannot rotate toward the center without exceeding the rotation limits, there is a greater chance of grain spillage if the trailer's relative speed suddenly changes. For example, consider the scenario where the system is filling the front zone of the trailer and the unloading auger or screw conveyor is rotating to its lower rotation limit. If the trailer's ground speed suddenly decreases, harvested material will spill over the front edge of the trailer, and the system's only option to limit spillage is to stop the screw conveyor drive on the combine harvester.Conversely, if the screw conveyor or unloading pipe is not at its lower rotation limit, and it is unloading into the front zone, and the speed over the ground of the trailer suddenly increases, the system can rotate the screw conveyor or unloading pipe towards the rear of the trailer to mitigate or prevent bulk loss.

[0077] S506: The system then considers the desired filling strategy to determine the next zone in the trailer into which the system will unload when the current zone reaches the desired fill level. The system calculates the required angle of the auger or discharge pipe to unload into the next filling zone. If the angle required to unload into the next zone violates the rotation limits, the system will not be able to rotate the discharge pipe when the zone it is currently unloading into reaches the desired fill level. On the combine harvester, the system will be forced to stop the auger drive if this occurs. On the SPFH (self-propelled forage harvester), the system will overfill the zone it is currently unloading into until the next zone can be reached without violating the discharge pipe's rotation limits.

[0078] S507: The system then takes measures to adjust the relative front / backward displacement of the combine harvester or SPFH to the trailer. If "Machine Sync" is present, the combine harvester or SPFH can command the tractor to temporarily change its ground speed until the next filling zone is within the rotation limits, or, if the system is unloading into the front or rear zone of the trailer, until the adjacent zone in the trailer is within the rotation limits. If "Machine Sync" is not present, the system can adjust the ground speed of the combine harvester or SPFH. If "Machine Sync" is not present and the system is unable to adjust the ground speed of the combine harvester or SPFH, the system can send a CAN message to the graphical user interface and sound an alarm to indicate to the operator that the system has reached its rotation limits.The system or the operator will shut off the drive (S508) of the screw conveyor when the filling strategy is complete or other limits are violated and would result in overflow if continued.

[0079] For example, when executing a back-to-front filling strategy, if the auger conveyor 47 is fully rotated to its maximum extension and grain is being unloaded into the center of the grain trailer 4, the system will not be able to further extend the auger conveyor's rotation angle φ to fill the front portion of the trailer 4. Instead, the system should adjust the front / back offset between the vehicles (via "Machine Sync" or combine harvester speed control) so that the auger conveyor 47 is not fully extended when filling the center of the grain trailer. Auger conveyor rotation φ is more responsive than adjusting the front / back offset, and it is important to be able to unload grain into another area of ​​the trailer 4 as quickly as possible once the current area reaches its desired fill level.

[0080] Another reason to consider the rotation angle φ and the front / backward displacement is to ensure that grain spillage can be easily mitigated. If the auger conveyor is fully extended, and grain is being unloaded into the rear of the grain trailer 4, and the forward speed of the grain trailer 4 is greater than that of the combine harvester 2, the only recourse the automatic unloading system has to mitigate grain spillage is to shut off the unloading auger conveyor 47. If the auger conveyor 47 is not fully rotated in this scenario, the system can fully rotate it and shut off the flow to further reduce the risk of grain spillage.

[0081] A stereo camera (not shown) could also be mounted on the cab of a tractor. Overcoming conditions that reduce system performance---

[0082] Occasionally, the system may lose its ability to track the combine harvester's relative position to the grain trailer or to measure the fill level. Multiple cameras are used to improve the accuracy and robustness of trailer tracking and fill level measurement. Each camera can be positioned on the combine's chassis or at the end of the unloading auger / screw conveyor to optimize the system's trailer tracking and fill level function. Data fusion algorithms are used to register and combine the output from the multiple cameras to produce a single, accurate, and robust measurement of the trailer's position and fill level. All the trailer position information is integrated using a filtering algorithm, such as a Kalman filter, to produce the estimated trailer position and orientation.Simultaneously, all fill level information is integrated using a model-based filter to produce an accurate fill level measurement. One embodiment of the present invention includes a built-in switch to handle failure in one or more cameras. If one or more cameras fail and are disabled during operation, the filtering and recording algorithms automatically use information only from the remaining camera or "Machine Sync" data, if "Machine Sync" is available. The same switch functionality can also be used to handle occlusion that blocks one or more camera fields of view. Failure detection utilizes consistency in measurements from multiple cameras. Trailer tracking information is used to perform selective stereo processing for fill level measurement and improved real-time performance. Fig. Figure 6 illustrates the process steps for an embodiment of the present invention.

[0083] S600: After the pictures in S501 from Fig. Once command 5 is detected, the system determines whether "Machine Sync" is present. The presence of "Machine Sync" is detected by monitoring for specific "Machine Sync" messages on the CAN bus. If so, the process continues with S602. Otherwise, the system proceeds to S502. Fig. 5 return to perform calculations using stereo vision distance measurement to determine relative trailer position (S601).

[0084] S602: Capturing “Machine Sync” offsets between the combine harvester or SPFH (self-propelled forage harvester) and the tractor.

[0085] S603: "Machine Sync" calculates the relative position of the GPS receiver on the tractor to the position of the GPS receiver on the combine harvester or SPFH (Single-Finger Trailer). Since the position of the GPS receiver on the combine harvester or SPFH is known, the relative displacement between the GPS receiver on the tractor and on the combine harvester can be translated to a reference point on the vehicle (e.g., the center of the rear axle). The automated unloading system first calculates the relative position of the trailer corners to the stereo camera. Since the position of the camera on the combine harvester or SPFH is known, the relative position of the trailer corners can be translated to a reference point on the vehicle (e.g., the center of the rear axle). Once the position of the GPS receiver on the tractor and the corners of the trailer are translated to the same reference frame, the displacement between the GPS receiver and the trailer corners can be calculated.

[0086] S604: There are countless possibilities for sensor merging when the "Machine Sync" offsets and the automated unloading system offsets are translated onto the same coordinate frame. Here are a few: 1. The system could simply use the offset from the data source with the higher confidence. "Machine Sync" might have low confidence if the GPS receiver is tracking few satellites, if the GPS signal strength is weak, or if interference is preventing wireless radio communication. The automated unloading system might have low confidence if the camera is dirty, there is dust in the air, or the trailer is a large distance from the camera (the accuracy of stereo vision distance measurement degrades quadratically with distance).For example, "Machine Sync" confidence could be the product of the combine harvester's GPS signal strength-time (0-100%) multiplied by the tractor's GPS signal strength (0-100%) multiplied by the communication link's signal integrity (0-100%). The confidence of the system's vision processing portion could be the percentage of features the system can track from the previous frame to the current frame, or related features. 2. The system could use the offsets calculated by the system's vision processing portion as long as it has high confidence and revert to using the "Machine Sync" offsets when the vision processing confidence is low. 3. The offset estimates from the "Machine Sync" and the system's vision processing portion could be tightly merged in a Kalman filter.The “Kalman” filter weights the contributions from the “Machine Sync” and the vision processing part of the system based on the reported trust and error model of each data source.

[0087] S605: Use the output of the merging algorithm to determine the relative trailer position. The system becomes S502 from Fig. Return to step 5 to perform calculations using the merged data.

[0088] Possible causes include: Thick dust between the combine harvester and the grain trailer makes it difficult or impossible for camera 12 on the grain tank (chassis) to see identifying features on the grain trailer. This condition can occur when harvesting with a tailwind. - Bright sunlight from a setting sun shining directly into the grain tank camera 12 saturates the camera's imaging device. Dirt or other residues build up on the camera window.

[0089] In the case of thick dust, camera 12 on the grain tank cannot see grain trailer 4. If the automated unloading system cannot track grain trailer 4, it must shut down screw conveyor 47, and the operator must unload manually. The automated unloading system could detect a loss of tracking from camera 12 on the grain tank and use camera 10 on the screw conveyor to detect features on grain trailer 4 and track those features until camera 12 on the grain tank is able to resume tracking. Camera 10 on the screw conveyor is better able to detect features on grain trailer 4 because it is closer to the grain trailer 4 and has less dust obscuration to penetrate.

[0090] In the event of bright sunlight from the setting sun, camera 12 on the grain tank could become saturated and lose its tracking capability. The system could also use camera 10 on the screw conveyor to track features on the grain trailer 4 during such incidents. Camera 10 on the screw conveyor is less likely to be saturated by sunlight because it has a steeper downward viewing angle than camera 12 on the grain tank.

[0091] When the automated unloading system is used in conjunction with "Machine Sync," the relative position between combine harvester 2 and grain trailer 4 can be calculated using GPS data from combine harvester 2 and the tractor 6 pulling grain trailer 4. This calculated relative position could be an input to a "Kalman" filter, along with the relative position scanned by camera 12 of the grain tank. If camera 12 of the grain tank is compromised by dust or direct sunlight, the relative position calculated from the GPS data could be weighted much more heavily in the "Kalman" filter.

[0092] The automated unloading system can also measure the amount of residue on the camera window. The system periodically captures and analyzes an image for signs of residue buildup on the lens. Several commercially available image processing algorithms are designed to calculate a metric of image sharpness or blurriness that could correlate with the dirtiness of the camera window. The measured dirtiness can then be displayed to the driver on the user interface. The driver can then decide whether it is necessary to clean the window immediately or wait for a more convenient opportunity.

[0093] Although the disclosure has been described in detail and with reference to specific embodiments thereof, it will be obvious to a person skilled in the art that various changes and modifications can be made to it without deviating from the meaning and scope of the embodiments. Therefore, it is intended that the present disclosure covers the modifications and deviations of this disclosure, provided they fall within the scope of the appended claims and their equivalents.

Claims

[1] System (11, 111, 211) for facilitating the transfer of agricultural product from a transferring vehicle (2) to a receiving vehicle (6), wherein the system (11, 111, 211) comprises: a discharge pipe (47), comprising: a length, a rotation end which is rotatably connected to the transfer vehicle and a discharge end (13B), an actuator for rotation to rotate the unloading pipe (47) about a vertical axis of rotation from a rest position to a maximum position along a maximum arc, a vertical actuator to raise and lower the discharge end (13B) vertically between a rest position height and a maximum position height, one or more sensors to determine a current rotation angle φ of the unloading pipe (47) and a current elevation angle θ of the unloading pipe (47), and a screw conveyor drive to transfer agricultural product from the transferring vehicle (2) through the unloading pipe (47) into the receiving vehicle (6); wherein the receiving vehicle (6) comprises a driven part for driving the receiving vehicle (6) and a storage part (4) for storing agricultural product, wherein the storage part (4) comprises a front, a back and two longitudinal sides joined at corners, each side having a top edge; a first imaging device (10) connected to the unloading pipe (47) and pointing forward towards the storage part (4) of the receiving vehicle (6), wherein the first imaging device (10) collects initial image data; a second imaging device (12) connected to a chassis of the transferring vehicle (2) and pointing towards the storage part (4) of the receiving vehicle (6), wherein the second imaging device (12) collects second image data; and an image processing module (18) associated with the transferring vehicle (2), wherein the image processing module (18) comprises a processor for executing software to position the discharge end (13B) of the unloading pipe (47) in relation to the front or rear or top edge of the storage part (4) of the receiving vehicle based on a filling strategy which uses the first image data and the second image data. [2] System (11, 111, 211) according to claim 1, wherein the transferring vehicle (2) and the receiving vehicle (6) each comprise a location receiver (42) and a wireless communication device (48, 148) and, wherein the processor further processes data from the location receiver (42) of the receiving vehicle (6), which is transmitted between the wireless communication devices (48, 148) and data from the location receiver (42) of the transferring vehicle (2) to determine current displacements (offsets). [3] System (11, 111, 211) according to claim 1, wherein the first imaging device (10) and the second imaging device (12) each comprise a stereoscopic camera. [4] Method for facilitating the transfer of agricultural product from a transferring vehicle (2) to a receiving vehicle (6) comprising a system (11, 111, 211) according to claim 1, 2 or 3, wherein the processor performs a step to determine a rotation limit of the discharge end (13B) of the unloading pipe (47) within the maximum arc of the discharge end (13B) of the unloading pipe (47). [5] Method according to claim 4, wherein the rotation limit is defined as a part of the maximum arc that traverses the filling zones defined by the filling strategy. [6] Method according to claim 4, wherein the step for determining a rotation limit of the discharge end (13B) of the unloading pipe (47) within the maximum arc of the discharge end (13B) of the unloading pipe (47) further comprises a step for determining a current lateral displacement (offset) between the storage part (4) and the transferring vehicle (2). [7] Method according to claim 6, wherein the step for determining a rotation limit of the discharge end (13B) of the unloading pipe (47) within the maximum arc of the discharge end (13B) of the unloading pipe (47) further comprises the step for determining a disturbance point of the unloading pipe (47) with the upper edge of the storage part (4). [8] Method according to claim 7, wherein the step to determine a disturbance point of the discharge pipe (47) and the upper edge of the storage part (4) comprises the following steps: Determining a vertical distance from a rotational bearing point of the discharge pipe to the top edge of the storage part (4); Calculating a tangent to the current elevation angle θ of the discharge pipe (47); Dividing the vertical distance by the tangent of the current elevation angle θ of the discharge pipe (47) to calculate a minimum lateral displacement; and Adjusting the current lateral displacement to a subsequent lateral displacement if the current lateral displacement is greater than the minimum lateral displacement. [9] The method of claim 4, wherein the processor further comprises the steps: Merging the current front-to-back shift (offset) and the current lateral shift with the first image data and the second image data; calculating a relative position of corners of the memory part (4). [10] Method according to claim 9, wherein the processor further performs a step to determine a position of the memory part (4) relative to the transferring vehicle (2). [11] Method according to claim 4, wherein the processor further performs a step to identify whether the discharge end (13B) of the unloading pipe (47) is close to the rotation limit. [12] Method according to claim 11, wherein the processor further performs a step to identify whether the discharge end (13B) of the unloading pipe (47) fills into the foremost or rearmost filling zone. [13] Method according to claim 12, wherein the processor further performs a step to identify whether the discharge end (13B) of the unloading pipe (47) is capable of rotating to one or more adjacent filling zones. [14] Method according to claim 13, wherein the processor further comprises the steps: Adjusting the current forward-backward displacement to a subsequent forward-backward displacement; and rotating the discharge pipe (47) to one or several adjacent filling zones. [15] The method of claim 13, wherein the processor further performs the steps of: Adjusting the current forward-backward displacement to a subsequent forward-backward displacement; and maintaining a rotational alignment of the discharge pipe (47) .

Images