Mobile work machines with object detection and machine path visualization

Abstract

A method of controlling a mobile work machine on a work site includes receiving an indication of an object detected on the work site, determining a position of the object relative to the mobile work machine, receiving an image of the work site, associating the determined position of the object with a portion of the image, and generating a control signal that controls a display device to display a representation of the image with a visual object indication representing the detected object on the portion of the image.

Description

Technical Field

[0001] The present disclosure generally relates to object detection systems for mobile work machines. More particularly, but not by way of limitation, the present disclosure relates to an object detection system for a mobile work machine that uses radar detection to detect objects and generates a visualization of the detected objects relative to a rearward path of the work machine. Background Art

[0002] There are many different types of work machines. These may include construction machines, turf management machines, forestry machines, agricultural machines, etc. Many such mobile devices have controllable subsystems that include mechanisms that are controlled by an operator when performing an operation.

[0003] For example, a construction machine may have multiple different mechanical, electrical, hydraulic, pneumatic, and electromechanical systems, all of which can be operated by an operator. Depending on the job site operation, a construction machine is typically responsible for transporting materials across the job site, into the job site, or out of the job site. Different job site operations can include moving materials from one location to another or leveling the job site. During job site operations, a variety of construction machines may be used, including articulated dump trucks, wheel loaders, graders, and excavators.

[0004] Job site operations can involve numerous steps or phases and can be quite complex. Furthermore, job site operations often require precise operator control of the machine. Certain maneuvers on the job site require the operator to maneuver the work machine in the opposite direction to return across the job site. When doing so, there are often blind spots or areas that are difficult for the operator to observe, even with the use of mirrors or rearview cameras. This increases the risk of unintended contact between the work machine and objects on the job site, such as other machines, people, job site materials, and the like.

[0005] The above discussion provides general background information only and is not intended to be used as an aid in determining the scope of the claimed subject matter. Summary of the Invention

[0006] A method of controlling a mobile work machine on a work site includes receiving an indication of an object detected on the work site, determining a position of the object relative to the mobile work machine, receiving an image of the work site, associating the determined position of the object with a portion of the image, and generating a control signal that controls a display device to display a representation of the image with a visual object indication representing the detected object on the portion of the image.

[0007] This Summary is provided to introduce some concepts in a simplified form that are further described in the Detailed Description below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all of the disadvantages identified in the Background. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram illustrating one example of a work machine architecture including a mobile work machine.

[0009] Figure 2 It is a perspective view showing an example of a mobile working machine.

[0010] Figure 3 is a block diagram illustrating one example of an object detection system.

[0011] Figure 4 is a flow chart illustrating an exemplary operation of an object detection system.

[0012] Figure 5A and 5B is a flow chart illustrating an exemplary operation of an object detection system.

[0013] Figure 6 is a flow chart illustrating exemplary operations of path determination and control signal generation.

[0014] Figure 7 is an exemplary user interface display illustrating a path of a mobile work machine.

[0015] Figure 8 is an exemplary user interface display illustrating a path of a mobile work machine.

[0016] Figure 9 is an exemplary user interface display illustrating a path of a mobile work machine.

[0017] Figure 10 It shows the deployment in the remote server architecture Figure 1 A block diagram of an example of the architecture is shown.

[0018] Figures 11 to 13 An example of a mobile device that may be used in the architecture shown in the previous figures is shown.

[0019] Figure 14 is a block diagram illustrating one example of a computing environment that may be used in the architecture shown in the previous figures. DETAILED DESCRIPTION

[0020] The present disclosure generally relates to object detection systems for mobile work machines. More particularly, but not by way of limitation, the present disclosure relates to an object detection system for a mobile work machine that uses radar detection to detect objects and generates a visualization of the detected objects relative to a rearward path of the work machine.

[0021] Figure 1 is a block diagram illustrating an example of a work machine architecture 100 including a mobile work machine 102. The work machine 102 includes a control system 104 configured to control a set of controllable subsystems 106 that perform operations on a work site. For example, an operator 108 can interact with and control the work machine 102 through an operator interface mechanism 110. The operator interface mechanism 110 may include items such as a steering wheel, pedals, control levers, joysticks, buttons, dials, linkages, and the like. In addition, the operator interface mechanism may include a display device that displays user-actuable elements (e.g., icons, links, buttons, and the like). In the case where the device is a touch-sensitive display, these user-actuable items can be actuated through touch gestures. Similarly, in the case where the mechanism 110 includes a voice processing mechanism, the operator 108 can provide input and receive output through a microphone and a speaker, respectively. The operator interface mechanism 110 may include any of a variety of other audio, visual, or tactile mechanisms.

[0022] Work machine 102 includes a communication system 112 that is configured to communicate with other systems or machines in architecture 100. For example, communication system 112 can communicate with other local machines, such as other machines operating at the same work site as work machine 102. In the illustrated example, communication system 112 is configured to communicate with one or more remote systems 114 via a network 116. Network 116 can be any of a variety of different types of networks. For example, network 116 can be a wide area network, a local area network, a near field communication network, a cellular communication network, or any of a variety of other networks or a combination of networks.

[0023] Remote user 118 is shown interacting with remote system 114, such as by receiving or sending communications from or to work machine 102 via communication system 112. For example, and without limitation, remote user 118 may receive communications, such as notifications, requests for assistance, etc., from work machine 102 via a mobile device.

[0024] Figure 1Work machine 102 is also shown to include one or more processors 122, one or more sensors 124, an object detection system 126, data storage 128, and may also include other items 130. Sensors 124 may include any of a variety of sensors, depending on the type of work machine 102. For example, sensors 124 may include object detection sensors 132, material sensors 134, position / route sensors 136, speed sensors 138, work site imaging sensors 140, and may also include other sensors 142.

[0025] Material sensor 134 is configured to sense material being moved, processed, or otherwise manipulated by work machine 102. Speed ​​sensor 138 is configured to output a signal indicative of the speed of work machine 102.

[0026] Position / route sensor 136 is configured to identify the position of work machine 102 and the corresponding route (e.g., heading) of work machine 102 as the work machine traverses the work site. Sensor 136 includes sensors configured to generate signals indicating the angle or turning radius of work machine 102. Sensors may include, but are not limited to, steering angle sensors, articulation angle sensors, wheel speed sensors, differential drive signals, and gyroscopes, to name a few.

[0027] Work site imaging sensor 140 is configured to obtain images of the work site that can be processed to identify objects or conditions at the work site. Examples of imaging sensor 140 include, but are not limited to, a camera (e.g., a monocular camera, a stereo camera, etc.) that captures still images, a time series of images, and / or a video feed of an area of ​​the work site. For example, a camera's field of view (FOV) may include an area of ​​the work site that is behind work machine 102 and that may not be visible to operator 108 in the operator's compartment or cab of work machine 102.

[0028] The object detection sensor 132 may include an electromagnetic radiation (EMR) transmitter and receiver (or transceiver) 162. Examples of EMR transmitter / receivers include radio frequency (RF) devices 164 (such as RADAR), LIDAR devices 166, and may also include other devices 168. The object detection sensor 132 may also include a sonar device 170 and may also include other devices 172.

[0029] For the purpose of illustration but not limitation, examples will be discussed below in the context of RADAR. Of course, other types of detection sensors can also be utilized in those examples.

[0030] The control system 104 may include a setup control logic system 144, a route control logic system 146, a power control logic system 148, a display generator logic system 149, and may include other items 150. The controllable subsystems 106 may include a propulsion subsystem 152, a steering subsystem 154, a material handling subsystem 155, one or more different actuators 156 that may be used to change machine settings, machine configuration, etc., an electronic system 158, and may include a variety of other systems 160, some of which are described below. In one example, the controllable subsystems 106 include operator interface mechanisms 110, such as a display device, an audio output device, a tactile feedback mechanism, and an input mechanism. Examples are discussed in further detail below.

[0031] The setup control logic system 144 can control one or more of the subsystems 106 to change machine settings based on an object, jobsite conditions, or characteristics. For example, the setup control logic system 144 can actuate actuators 156 that change the operation of the material handling subsystem 155, the propulsion subsystem 152, and / or the steering subsystem 154.

[0032] Path control logic system 146 may control steering subsystem 154. By way of example and not limitation, if object detection system 126 detects an object, path control logic system 146 may control propulsion subsystem 152 and / or steering subsystem 154 to avoid the detected object.

[0033] The power control logic system 148 generates control signals to control the electronic systems 158. For example, the power control logic system 148 can distribute power to different subsystems, generally increasing power usage or decreasing power usage, etc. These are merely examples, and a variety of other control systems can also be used to control other controllable subsystems in different ways.

[0034] Display generator logic system 149 illustratively generates control signals to control the display device to generate a user interface display for operator 108. The display may be an interactive display having a user input mechanism for interaction by operator 108.

[0035] Object detection system 126 is configured to receive signals from object detection sensor 132 and, based on these signals, detect objects adjacent to work machine 102 at the work site (e.g., in the rear path of work machine 102). Thus, object detection system 126 can help operator 108 avoid objects when backing up. Before discussing object detection system 126 in more detail, reference will be made to Figure 2 Discuss the example of mobile work machinery.

[0036] As mentioned above, mobile work machines can take a variety of different forms. Figure 2 1 is a diagram illustrating one example of a mobile work machine 200 in the form of an off-highway construction vehicle having an object detection system 201 (e.g., system 126) and a control system 202 (e.g., 104). While work machine 200 illustratively includes a wheel loader, a variety of other mobile work machines may be used. Work machine 200 may include other construction machines (e.g., bulldozers, motor graders, etc.), agricultural machines (e.g., tractors, combines, etc.), to name a few.

[0037] Work machine 200 includes a cab 214 having a display device 215, ground engaging elements 228 (e.g., wheels), a motor 204, a speed sensor 206, a frame 216, and a boom assembly 218. Boom assembly 218 includes a boom 222, a boom cylinder 224, a bucket 220, and a bucket cylinder 226. Boom 222 is pivotally connected to frame 216 and can be raised and lowered by extending or retracting boom cylinder 224. Bucket 220 is pivotally connected to boom 222 and can be moved by extending or retracting bucket cylinder 226. During operation, mobile work machine 200 can be controlled by an operator in cab 214, where mobile work machine 200 can traverse a work site. In one example, each of motors 204 is illustratively connected to and configured to drive wheels 228 of mobile work machine 200. A speed sensor 206 is illustratively connected to each of motors 204 to detect motor operating speed.

[0038] In the example shown, work machine 200 includes an articulated body in which a front portion 229 is pivotally connected to a rear portion 231 at a pivot joint 233. An articulation sensor can be used to determine the articulation angle at pivot joint 233, which can be used to determine the path of work machine 200. In another example in which the body of work machine 200 is non-articulated, the angles of the front and / or rear wheels 228 can rotate relative to the frame.

[0039] Object detection system 201 detects objects within range of work machine 200. In the example shown, object detection system 201 receives signals from an object detection sensor 205 and an imaging sensor 207 (e.g., a monocular camera), which are illustratively mounted at rear end 209 of work machine 200. In one example, components of system 201 and / or system 202 communicate over a CAN network of work machine 200.

[0040] Object detection sensor 205 is configured to transmit a detection signal from rear end 209 of work machine 200 and receive reflections of the detection signal to detect one or more objects behind work machine 200. In one example, the detection signal comprises electromagnetic radiation transmitted to the rear of work machine 200. For example, the detection signal may comprise a radio frequency (RF) signal. Some specific examples include radar and LORAN, to name a few.

[0041] In other examples, the object detection sensor 205 utilizes sonar, ultrasound, and light (eg, LIDAR) to image an object. Exemplary LIDAR systems utilize ultraviolet light, visible light, and / or near infrared light to image an object.

[0042] Of course, other types of object detectors may be utilized. In any case, object detection system 201 generates an output indicative of an object that may be utilized by control system 202 to control the operation of work machine 200 .

[0043] Some work machines utilize a rearview camera that displays a rear view from the work machine to the operator, as well as a radar system that provides an audible indication of the presence of an object behind the work machine. Such systems visually cover the area directly behind the work machine that cannot be seen by the operator using mirrors. However, it is often difficult for the operator to determine what objects are relevant (e.g., detection of actual obstacles versus non-obstacles or false alarms), as well as an indication of where the objects are actually located around the work machine (e.g., whether the objects are in the path of the work machine). For example, some radar systems have a large range, but have a tendency to generate false alarms, which are alarms when no objects are present. Typically, this is due to multipath reflections or ground reflections. Therefore, it may be difficult for the operator to distinguish between true alarms and false alarms. In addition, some radar systems can detect objects at a long distance behind the work machine, but may not be able to detect objects that are very close to the sensor (i.e., near the rear end 209 of the work machine 200).

[0044] Furthermore, some machine systems utilizing CAN communication may have limited bandwidth to communicate over the CAN bus. Consequently, the signals received from the radar object detection system may include limited information about the tracked object, providing low-quality information. Consequently, the system cannot determine size information or range / angular resolution, increasing the likelihood of false positive detections.

[0045] Figure 3An example of an object detection system 300 is shown. The object detection system 300 is configured to combine object detection information received from an object detection sensor, such as sensor 132, with visual recognition using images captured by an imaging sensor on or otherwise associated with the work machine. Thus, rather than simply providing an operator with the ability to look behind the work machine, the object detection system 300 utilizes the image as a sensor to detect objects and their corresponding positions relative to the work machine. Furthermore, objects detected by the object detection system 300 can be fused with images acquired by the imaging sensor to provide an indication to the operator of where the detected object is located in the image frame. This can enable the operator to quickly determine whether the detected object is a false alarm, a true alarm that the operator is already aware of, or a true alarm that the operator is unaware of. Furthermore, the object detection system 300 facilitates a wider coverage area for object detection without significantly increasing the detection of false alarms.

[0046] For purposes of illustration and not limitation, Figure 1 Object detection system 300 is described with reference to the illustrated case of mobile work machine 102 .

[0047] Object detection system 300 includes activation logic system 302 configured to activate and control object detection performed by object detection system 300. For example, object detection system 300 may determine, in response to mode selector 304, that work machine 102 has entered a particular mode for which object detection system 300 is to be activated. For example, object detection system 300 may determine, in response to sensing operator input and / or machine settings, that work machine 102 is backing up, preparing to back up, etc.

[0048] Sensor control logic 306 is configured to control object detection sensor 132 and imaging sensor 140. Logic 306 controls sensor 132 to transmit detection signals and receive corresponding reflections of the detection signals, which are used by object detection logic 308 to detect the presence of objects on the work site. Logic 306 controls sensor 140 to acquire images of the work site.

[0049] Object position determination logic 310 is configured to determine the position of an object detected by logic system 308 . Object / image association logic 312 is configured to associate the object position determined by logic system 310 with a portion of an image captured by imaging sensor 140 .

[0050] Visual recognition system 313 is configured to perform visual recognition on images to evaluate objects detected by logic system 308. Illustratively, visual recognition system 313 includes image processing logic 314, which is configured to perform image processing on images, and object evaluation logic 316, which is configured to evaluate objects based on the image processing performed by logic system 314. This may include, but is not limited to, object size detection 318, object shape detection 320, object classification performed by object classifier 322, false positive determination 324, and may also include other items 326.

[0051] Path determination logic 328 is configured to determine a path for work machine 102, and control signal generator logic 330 is configured to generate control signals, either by itself or in conjunction with control system 104. Object detection system 300 is shown with one or more processors 332 and may also include other items 334.

[0052] Figure 4 A flowchart 400 illustrates exemplary operation of the object detection system 300. For purposes of illustration and not limitation, the operation will be described in the context of the mobile work machine 102. Figure 4 .

[0053] At block 402 , activation logic system 302 activates object detection. This may be in response to manual input from operator 108 , such as operator 108 actuating an input mechanism. This is represented by block 404 . Alternatively or additionally, the object detection system may be automatically activated, such as in response to detecting that work machine 102 has entered a predetermined operating mode, such as being switched to reverse. This is represented by block 406 . Of course, the object detection system may also be activated in other ways. This is represented by block 408 .

[0054] At block 410, the sensor control logic system 306 controls the transmitter to transmit a detection signal. In the example shown, this includes a radar transmitter transmitting a radar signal, represented by block 412. Alternatively or additionally, the detection signal may include a LIDAR device 414, and may also include other types of detection signals. This is represented by block 416.

[0055] At block 418, the receiver receives a reflection of the detection signal sent at block 410. At block 420, based on the received reflections, the object detection logic system 308 obtains an indication of one or more objects (e.g., representing possible obstacles on the work site). Each of these objects may be labeled with a unique object identifier. This is represented by block 422. The unique object identifier may be used by the object detection system 300 for subsequent processing of the detected object.

[0056] At block 424, the position of each object relative to work machine 102 is determined. This may include objects currently detected using the radar signal. This is represented by block 426. Furthermore, the position of objects previously detected using the radar signal may also be determined. This is represented by block 428. To illustrate, object detection logic system 308 may initially detect a particular object, but as the work machine traverses the work site, object detection system 300 may no longer detect that particular object in the radar signal. As discussed in further detail below, these objects may still be tracked by object detection system 300.

[0057] Determining the location of each object may include identifying the location of the object on a ground plane, ie, determining its approximate location at ground level at the work site. This is represented by block 430 .

[0058] When determining the location of each object, block 424 may use the distance from the sensor mounting location on work machine 102, i.e., the estimated distance of the object from the radar sensor. This is represented by block 432. Alternatively or additionally, determining the location at block 424 may include determining the angle at which the detected reflection was received relative to the orientation of the sensor mounted on work machine 102. This is represented by block 434.

[0059] At block 436, images of the work site are received from imaging sensor 140. As mentioned above, one example of imaging sensor 140 is a monocular camera. The images received at block 436 may include a time series of images or a video. This is represented by block 438. Of course, images may also be received in other ways. This is represented by block 440.

[0060] At block 442, the location of each object determined at block 424 is associated with a portion of the image received at block 436. Illustratively, the association is related to the field of view of the imaging sensor. This is represented by block 444. For example, block 442 utilizes the angles and / or distances determined at blocks 432 and 434 to identify an area of ​​the imaging sensor's field of view that corresponds to the detected object.

[0061] At block 446, image processing is performed on the portion of the image associated with each object from block 442. At block 448, each detected object is evaluated based on the image processing performed at block 446. This may include, but is not limited to, determining the object size (block 450), determining the object shape (block 452), and / or applying an image classifier to detect the object type (block 454). For example, image classification may determine that the object represented in the portion of the image is a person, another work machine, or another type of object. Of course, these are for example purposes only.

[0062] Furthermore, at block 448, the false alarm determination logic system 324 may determine the likelihood that the detected object is a false alarm. This is represented by block 456. In one example, this includes generating a metric or score indicating the likelihood that the portion of the image represents an actual object on the work site, and then comparing this metric or score to a threshold. Based on this comparison, the logic system 324 determines that the object detected from the radar signal is likely a false alarm. Of course, other assessments may also be performed. This is represented by block 458.

[0063] At block 460, control signals are generated based on the object evaluation performed at block 448 to control the work machine 102. The work machine 102 can be controlled in any of a variety of ways. In one example, one or more controllable subsystems 106 are controlled by the control signal generator logic system 330 and / or the control system 104. This is represented by block 462. Alternatively or additionally, an operator interface mechanism can be controlled to present a visual, auditory, tactile, or other type of output to the operator 108 indicating the detected object. This is represented by block 464.

[0064] In one example, an object detection sensor is controlled. This is represented by block 465. For example, the settings of a radar transmitter may be adjusted. In another example utilizing LIDAR (or other similar transmitter), the control signal may cause the transmitter to steer or direct the beam to an area of ​​the work site to perform another (e.g., higher accuracy) scan to identify possible objects.

[0065] Of course, work machine 104 may be controlled in other ways as well. This is represented by block 466. At block 468, if operation of object detection system 300 is to continue, operation returns to block 410.

[0066] Figure 5A and 5B 5 ) illustrates a flow chart 500 of exemplary operation of the object detection system 300 in evaluating an object using image processing. For purposes of illustration and not limitation, FIG5 will be described in the context of the mobile work machine 102.

[0067] At block 502, a set of objects detected by the object detection logic system 308 using the object detection sensor 132 (e.g., radar) is identified. In the example shown, this includes objects from Figure 4 4 and 5. As described above, the objects may include objects currently detected by the object detection sensor 132 (block 504), as well as objects that were previously detected (block 506) but may not be currently detected.

[0068] At block 508, one of the objects is selected from the group for processing. Block 510 determines whether the selected object is already being tracked by the image processing logic system 314. If not, block 512 identifies the location of the object (as detected by the object detection logic system 308) and creates a window in the image with a nominal window size. The nominal window size is illustratively a first predefined size to begin visual tracking of the object.

[0069] At block 514, the image processing logic system 314 performs adaptive thresholding using blob segmentation to separate the object from the background within the window. Thresholding is used to segment an image by setting all pixels with intensity values ​​above a threshold to a foreground value and all remaining pixels to a background value. In adaptive thresholding, the threshold value is dynamically changed across the image, which can adapt to changes in lighting conditions in the image (e.g., caused by lighting gradients or shadows).

[0070] In one example, semantic image segmentation analyzes the portion of an image within a window to associate pixels or groups of pixels with class labels. The class labels can be used to determine whether a pixel represents an object or non-object region of the work site.

[0071] Referring again to block 510 , if the object is already being tracked by image processing logic system 314 , logic system 314 uses any changes in the object's position detected by object detection logic system 308 and determined by position determination logic system 312 to move the previously identified window for the object within the image.

[0072] At block 518, a pixel buffer may be added around the window to aid in blob segmentation at block 514. At block 520, the image processing logic system 314 determines whether a closed blob (e.g., a group of pixels representing a particular object class) has been identified. If no closed blob has been identified, this may indicate that a portion of the object extends outside the current window. At block 522, the image processing logic system 314 increases the window size. If the maximum window size (which may be predetermined or otherwise determined) is reached at block 524, the object is marked as a possible false positive by the false positive determination logic system 324 at block 526.

[0073] If a closed blob is identified at block 520, the window size and position of the object are recorded at block 528. At block 530, blocks 508 / 530 are repeated for any additional objects. At block 532, the set of blobs or objects tracked by the image processing logic system 314 is identified. At block 534, one of the objects in the set is selected, and block 536 visually tracks the object's movement in the image frames. For example, block 536 determines the difference in the object's position between subsequent image frames. This can include using sensor information about the movement of the work machine to predict how the object is likely to move between frames. This is represented by block 538.

[0074] At block 540, a window is determined for the blob. In one example, this is similar to the process described above with respect to blocks 510-530. At block 542, the window around the blob is passed to the image object classifier 322 to perform image classification. This may include detecting the type of object represented within the window in the image.

[0075] At block 544, the actual size of the object is estimated based on the size of the pixels representing the object in the window. In one example, if the blob was tracked in a previous frame, the previous size estimate and / or previous classification is used to refine the current estimate of the object's size and / or classification. This is represented by block 536. In one example, if the difference in the estimate between the current frame and the previous frame exceeds a threshold, the object can be marked as blurry at block 548. Block 550 determines whether there are any more objects in the group.

[0076] Figure 6 is a flowchart 600 illustrating exemplary operation of the path determination logic system 328 and the control signal generator logic system 330 in generating control signals for controlling a display device on or associated with the work machine 102. For purposes of illustration and not limitation, the description will be made in the context of a mobile work machine 102. Figure 6 .

[0077] At block 602, sensor signals are received from sensors 124. The sensor signals may include signals indicating wheel speed (604), steering angle (606), articulation angle (608), differential drive speed (610), gyroscope (612), and may also include other signals (614).

[0078] Reference Figure 2 In the example shown, steering angle 606 represents the angle of the front wheels and / or rear wheels relative to the frame of work machine 200. Articulation angle 608 indicates the articulation angle between front portion 228 and rear portion 231.

[0079] Differential drive speed 610 represents the difference between the drive speeds of the traction elements on opposite sides of the work machine. For example, in the case of a skid steer, the differential drive speed represents the difference in direction and / or speed between the wheels or tracks on the left side of the work machine and the wheels or tracks on the right side of the work machine. The average of the two speeds provides the forward or reverse speed. The radius of curvature of the work machine's path can be determined based on the speed or ground speed of the work machine's wheels or tracks and the degree / second the work machine travels around the curve.

[0080] In any case, based on the sensor signals, a path for the work machine is determined at block 616. At block 618, images (such as a time series of images or a video) are displayed to operator 108 using a display device on or otherwise associated with work machine 102. At block 620, a visual indication of the path determined at block 616 is displayed.

[0081] At block 622 , a set of detected objects is identified. Illustratively, these objects include objects detected using radar signals transmitted by sensor 132 .

[0082] At block 624, for each detected object, a visual indication of the object is displayed on the displayed image. This can be accomplished in any of a variety of ways. For example, the location of the object can be identified in the image. This is represented by block 626. Alternatively or additionally, the outline of the object can be shown on the image. This is represented by block 628.

[0083] The visual indication may also indicate the size of the object (block 630), and / or may include a label indicating the classification of the object (block 632). For example, a label may be displayed on a display area near the object indicating the type of object, such as a person, another machine, etc.

[0084] Furthermore, the visual indication may include an indication of whether the object is a suspected false positive. This is represented by block 634. For example, if the likelihood that a detected object is an actual object on the worksite is below a threshold, the display may be modified to indicate this, such as by color-coding an area of ​​the display or otherwise displaying an indication that the object is a suspected false positive. This allows the operator to visually inspect the area of ​​the worksite to confirm the presence of the object. Of course, the visual indication may also be displayed in other ways. This is represented by block 636.

[0085] At block 638, object detection system 300 determines whether any of the objects have boundaries that overlap (or are within a threshold distance) with the path of the work machine determined at block 616. If so, an overlap is indicated at block 640. This may include changing the visual indication of the path at block 642, changing the visual indication of the overlapping object at block 644, presenting an audible alarm at block 646, or other indication (block 648).

[0086] Figures 7 to 9 An exemplary user interface display is shown in FIG. 1 , and for purposes of illustration and not limitation, will be described in the context of a mobile work machine 102. Figures 7 to 9 .

[0087] Figure 7 An exemplary user interface display 650 is shown that displays an image of a work site 652 and a visual path indication 654 representing a planned path for work machine 102 on work site 652. The field of view of imaging sensor 140 is located behind work machine 102. Figure 7 In one embodiment, no objects are detected on the work site, or any objects that may have been detected are outside a threshold detection range. In any case, the display 650 indicates that the path is free of objects, for example by displaying the visual path indication 654 in a first state (i.e., clear, without shading or highlighting).

[0088] Figure 8 User interface display 650 is shown with subsequent image frames obtained by imaging sensor 140 after operator 108 has begun reversing work machine 102 while turning. Based on the received sensor signals (e.g., received at block 602), system 126 determines a new planned path, which is represented by modifying visual path indication 654. Here, system 126 has detected one or more objects 656 and 660 on the work site and added visual object indications 658 and 662 corresponding to the detected objects to the display. Illustratively, indications 658 and 662 increase the perceptibility of detected objects 656 and 660. Indications 658 and 662 can have any of a variety of display characteristics. For example, indications 658 and 662 can flash on the display screen, have a particular color (e.g., yellow), and / or have a particular shape (e.g., a circle, an outline of the boundary of object 656, etc.). Furthermore, the display characteristics can vary based on the evaluation of object detection performed by the visual recognition system. For example, if the radar sensor detects an object, but the visual recognition system does not identify the object at that location (or otherwise generates a low likelihood metric), the visual object indication can be set with a selected display feature (e.g., green) to indicate an area of ​​the image containing a possible false positive detection.

[0089] Display 650 may also be modified to indicate the location of detected objects 656 and 660 relative to the path. Figure 8 , system 126 determines that neither object 656 or 660 overlaps the planned path. Therefore, visual path indication 654 can be modified to indicate this, such as by changing the color (e.g., yellow) of visual path indication 654. Alternatively or additionally, a warning indication 664 (e.g., a solid or flashing yellow triangle) can be added to the screen.

[0090] Figure 9 A user interface display 650 is shown in which subsequent image frames are obtained by the imaging sensor 140 after the operator 108 further reverses the work machine 102 on the work site. At this time, the system 126 determines that at least one of the objects 656 overlaps with the planned path of the work machine 102. In response to this determination, the display 650 is further modified to indicate the overlap. Illustratively, the alert level is increased by changing the display characteristics of the visual path indication 654. This can include changing the color (e.g., to red) or causing the indication 654 to flash, to name a few examples. Alternatively or in addition, a different warning indication 666 (e.g., a red stop sign) can be displayed. Furthermore, as described above, icons, text labels, or other identifiers can be added based on the image classification. For example, a descriptive text label can be added to the display 650 near the object indication 658.

[0091] An audible warning may also be generated, and the volume (or other means) may vary based on the detected distance to object 656 (i.e., the audible warning becomes louder as work machine 102 approaches object 656). Additionally, as described above, the steering subsystem and / or propulsion subsystem may be automatically controlled to avoid contact with object 656 (e.g., by automatically braking the wheels, stopping the engine, shifting gears, etc.).

[0092] The present discussion has referred to processors and servers. In one embodiment, the processors and servers comprise computer processors with associated memory and timing circuitry (not shown separately). The processors and servers are functional components of and activated by the systems or devices to which they belong, and facilitate the functionality of other components or items in those systems.

[0093] It should be noted that the above discussion has described a variety of different systems, components and / or logical systems. It should be understood that such systems, components and / or logical systems may include hardware items (such as processors and associated memory, or other processing components, some of which will be described below) that perform the functions associated with these systems, components and / or logical systems. In addition, as described below, systems, components and / or logical systems may include software that is loaded into memory and subsequently executed by a processor or server or other computing component. Systems, components and / or logical systems may also include different combinations of hardware, software, firmware, etc., some of which examples will be described below. These are merely some examples of different structures that can be used to form the above-mentioned systems, components and / or logical systems. Other structures may also be used.

[0094] In addition, multiple user interface displays have been discussed. The multiple user interface displays can take a variety of different forms and can have a variety of different user-actuated input mechanisms arranged thereon. For example, the user-actuated input mechanism can be a text box, a check box, an icon, a link, a drop-down menu, a search box, etc. The input mechanism can also be actuated in a variety of different ways. For example, a pointing device (such as a control ball or a mouse) can be used to actuate the input mechanism. A hardware button, switch, joystick or keyboard, finger switch or finger pad, etc. can be used to actuate the input mechanism. A virtual keyboard or other virtual actuator can also be used to actuate the input mechanism. In addition, when the screen on which the input mechanism is displayed is a touch-sensitive screen, touch gestures can be used to actuate the input mechanism. In addition, when the device displaying the input mechanism has a voice recognition component, voice commands can be used to actuate the input mechanism.

[0095] Multiple data stores are also discussed. It should be noted that these data stores can be divided into multiple types of data stores. They can all be local to the system accessing these data stores, or they can all be remote, or some can be local and others remote. All of these configurations are contemplated herein.

[0096] In addition, the accompanying drawings show a plurality of frames, wherein each frame is endowed with certain functions. It should be noted that fewer frames can be used so that functions are performed by fewer components. Additionally, more frames can be used, wherein functions are distributed among more components.

[0097] Figure 10 yes Figure 11 is a block diagram of an example of a work machine architecture 100, wherein a work machine 102 communicates with elements of a remote server architecture 700. In an example, the remote server architecture 700 can provide computing, software, data access, and storage services without requiring the end user to be aware of the physical location or configuration of the system delivering the services. In various examples, the remote server can deliver the services over a wide area network (e.g., the Internet) using appropriate protocols. For example, the remote server can deliver applications over the wide area network, and the remote server can be accessed via a web browser or any other computing component. Figure 1 The software or components shown in and the corresponding data can be stored on a server located at a remote location. The computing resources in a remote server environment can be consolidated at a remote data center location, or the computing resources can be decentralized. Remote server architectures can deliver services through a shared data center, even though these remote server architectures appear to users as a single access point. Therefore, remote server architectures can be used to provide the components and functionality described herein from a remote server located at a remote location. Alternatively, the components and functionality can be provided from a conventional server, or the components and functionality can be installed directly on the client device, or provided in other ways.

[0098] exist Figure 10 In the example shown, some items are related to Figure 1 Items shown are similar and are similarly numbered. Figure 10 Specifically shown is that system 126 and data storage 128 may be located at remote server location 702. Thus, work machine 102 accesses these systems through remote server location 702.

[0099] Figure 10 Another example of a remote server architecture is also depicted. Figure 10 It is also conceivable to Figure 1 Some elements of the system 126 may be located at the remote server location 702 while other elements may not be located at the remote server location 702. For example, the data store 128 may be located at a location separate from the location 702 and may be accessed via a remote server located at the location 702. Alternatively or additionally, the system 126 may be located at a location separate from the location 702 and may be accessed via a remote server located at the location 702.

[0100] Regardless of where these components are located, the work machine 102 can access them directly via a network (wide area network or local area network), these components can be hosted at a remote site via a service, or these components can be provided as a service or can be accessed by a connection service located in a remote location. In addition, data can be stored in essentially any location and can be intermittently accessed by or forwarded to relevant parties. For example, a physical carrier wave can be used instead of or in addition to an electromagnetic carrier wave. In this example, in the case of poor or no cellular coverage, another mobile machine (such as a refueling truck) can have an automatic information collection system. When the work machine approaches the refueling truck for refueling, the system automatically collects information from the work machine or transmits information to the work machine using any type of dedicated (ad-hoc) wireless connection. Then, when the refueling truck arrives at a location with cellular coverage (or other wireless coverage), the collected information can be forwarded to the main network. For example, when traveling to refuel other machines or at a main fuel storage location, the refueling truck can enter a covered location. All of these architectures are considered herein. In addition, information can be stored on the work machine until the work machine enters a covered location. The work machine itself can then send / receive information to / from the main network.

[0101] It should also be noted that Figure 1 The components or parts of the components can be arranged on a variety of different devices. Some of these devices include servers, desktop computers, laptop computers, tablet computers or other mobile devices, such as handheld computers, mobile phones, smart phones, multimedia players, personal digital assistants, etc.

[0102] Figure 11 1 is a simplified block diagram of an illustrative example of a handheld or mobile computing device that can be used as a user or customer handheld device 16 and in which the present system (or a portion of the present system) can be deployed. For example, the mobile device can be deployed in the cab of the work machine 102 or as a remote system 114. Figure 12-13 are examples of handheld or mobile devices.

[0103] Figure 11 Provides a general block diagram of the components of the client device 16, which can operate Figure 1 Some of the components shown may be Figure 1 Some of the components shown interact, or can run Figure 1 Some of the components shown can also be used with Figure 1Some of the components shown interact. In device 16, a communication link 13 is provided that allows the handheld device to communicate with other computing devices and, in some embodiments, provides a channel for automatically receiving information (e.g., by scanning). Examples of communication link 13 include protocols that allow communication via one or more communication protocols, such as wireless services for providing cellular access to a network, and protocols that provide local wireless connectivity to a network.

[0104] In other examples, the application may be received on a removable secure digital (SD) card connected to the interface 15. The interface 15 and the communication link 13 communicate with a processor 17 (which may also be implemented as a processor or server in the previous figures) along a bus 19, which is also connected to a memory 21 and input / output (I / O) components 23, as well as a clock 25 and a positioning system 27.

[0105] In one example, I / O components 23 are provided to facilitate input and output operations. The I / O components 23 of various embodiments of device 16 may include input components, such as buttons, touch sensors, optical sensors, microphones, touch screens, proximity sensors, accelerometers, and orientation sensors, and output components, such as displays, speakers, and / or printer ports. Other I / O components 23 may also be used.

[0106] The clock 25 illustratively includes a real-time clock component that outputs the time and date. The clock can also illustratively provide a timing function for the processor 17.

[0107] Positioning system 27 illustratively includes components for outputting the current geographic location of device 16. The positioning system may include, for example, a Global Positioning System (GPS) receiver, a LORAN system, a dead reckoning system, a cellular triangulation system, or other positioning systems. The positioning system may also include, for example, mapping software or navigation software that generates desired maps, navigation routes, and other geographic functions.

[0108] Memory 21 stores operating system 29, network settings 31, applications 33, application configuration settings 35, data storage 37, communication drivers 39, and communication configuration settings 41. Memory 21 may include all types of tangible, volatile, and non-volatile computer-readable storage devices. Memory may also include computer storage media (described below). Memory 21 stores computer-readable instructions that, when executed by processor 17, cause the processor to perform computer-implemented steps or functions in accordance with the instructions. Processor 17 may also be activated by other components to facilitate its functions.

[0109] Figure 12 An example of device 16 being a tablet computer 750 is shown. Figure 12, computer 750 is shown with a user interface display screen 752. Screen 752 can be a touch screen or a pen-activated interface that receives input from a pen or stylus. A virtual keyboard on the screen can also be used. Of course, the screen can also be attached to a keyboard or other user input device via a suitable attachment mechanism (e.g., a wireless link or a USB port). Computer 750 can also illustratively receive voice input.

[0110] Figure 13 The device is shown to be a smartphone 71. Smartphone 71 has a touch-sensitive display 73 that displays icons or tiles or other user input mechanisms 75. A user can use mechanisms 75 to run applications, make calls, perform data transfer operations, etc. Typically, smartphone 71 is built on a mobile operating system and provides more advanced computing power and connectivity than a feature phone.

[0111] It should be noted that other forms of device 16 are possible.

[0112] Figure 14 is an example of a computing environment in which, for example, Figure 1 Component or part thereof. Figure 14 , an exemplary system for implementing some embodiments includes a computing device in the form of a computer 810. Components of the computer 810 may include, but are not limited to, a processing unit 820 (which may include the processors or servers of the previous figures), a system memory 830, and a system bus 821 that connects various system components including the system memory to the processing unit 820. The system bus 821 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. Figure 1 The memory and program described can be deployed in Figure 14 in the corresponding part of .

[0113] Computer 810 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computer 810, and includes volatile and non-volatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media is different from, and does not include, modulated data signals or carrier waves. Computer storage media includes hardware storage media, which includes volatile and non-volatile media and removable and non-removable media, which are implemented in any method or technology to store information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to: RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by computer 810. Communication media can be implemented as computer-readable instructions, data structures, program modules, or other data in a transmission mechanism, and includes any information transmission media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0114] The system memory 830 includes computer storage media in the form of volatile and / or nonvolatile memory, such as read-only memory (ROM) 831 and random access memory (RAM) 832. A basic input / output system 833 (BIOS), containing the basic routines that help to transfer information between elements within the computer 810 (e.g., during startup), is typically stored in ROM 831. RAM 832 typically contains data and / or program modules that are immediately accessible to and / or currently being executed by the processing unit 820. For example, and not limitation, Figure 14 Operating system 834 , application programs 835 , other program modules 836 , and program data 837 are shown.

[0115] The computer 810 may also include other removable / non-removable, volatile / non-volatile computer storage media. For example only, Figure 14 Shown are a hard disk drive 841 that reads from or writes to non-removable, non-volatile magnetic media, an optical drive 855, and a non-volatile optical disk 856. The hard disk drive 841 is typically connected to the system bus 821 through a non-removable storage interface, such as interface 840, and the optical drive 855 is typically connected to the system bus 821 through a removable storage interface, such as interface 850.

[0116] Alternatively or additionally, the functions described herein may be performed or at least partially performed by one or more hardware logic components. For example, but not limitation, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0117] The above discussed and Figure 14 The drives and their associated computer storage media shown in FIG. 8 provide storage of computer readable instructions, data structures, program modules and other data for the computer 810. Figure 14 844, application programs 845, other program modules 846, and program data 847. Note that these components may be the same as or different from operating system 834, application programs 835, other program modules 836, and program data 837.

[0118] A user can enter commands and information into the computer 810 through input devices such as a keyboard 862, a microphone 863, and a pointing device 861 (e.g., a mouse, a control ball, or a touch pad). Other input devices (not shown) may include joysticks, gamepads, satellite dishes, scanners, and the like. These and other input devices are typically connected to the processing unit 820 via a user input interface 860 connected to the system bus, but may be connected via other interfaces and bus structures. A visual display 891 or other type of display device is also connected to the system bus 821 via an interface such as a video interface 890. In addition to a monitor, the computer may also include other peripheral output devices such as speakers 897 and a printer 896, which may be connected via an output peripheral interface 895.

[0119] The computer 810 operates in a network environment using logical connections (eg, a local area network - LAN, a wide area network - WAN, or a controller area network - CAN) to one or more remote computers (eg, remote computer 880).

[0120] When used in a LAN networking environment, the computer 810 is connected to the LAN 871 through a network interface or adapter 870. When used in a WAN networking environment, the computer 810 typically includes a modem 872 or other device for establishing communications over the WAN 873 (e.g., the Internet). In a networking environment, program modules may be stored in the remote memory storage device. Figure 14 For example, remote application programs 885 are shown as being located on remote computer 880 .

[0121] It should also be noted that the different examples described herein can be combined in different ways. That is, parts of one or more examples can be combined with parts of one or more other examples. All of these aspects are contemplated herein.

[0122] Example 1 is a method of controlling a mobile work machine on a work site, the method comprising:

[0123] receiving an indication of an object detected on the work site;

[0124] determining a position of the object relative to the mobile work machine;

[0125] receiving an image of the work site;

[0126] associating the determined position of the object with a portion of the image; and

[0127] A control signal is generated that controls a display device to display a representation of the image with a visual object indication representing the detected object on the portion of the image.

[0128] Example 2 is a method according to any or all of the preceding examples, further comprising:

[0129] determining a planned path for the mobile working machine;

[0130] The display device is controlled to present a visual path indication representing the planned path on the image.

[0131] Example 3 is a method according to any or all of the preceding examples, further comprising:

[0132] A visual characteristic of the visual path indication is changed based on a determination that at least a portion of the detected object is located within the planned path.

[0133] Example 4 is a method according to any or all of the preceding examples, wherein changing a visual characteristic of the visual path indication comprises changing a display color of the visual path indication.

[0134] Example 5 is a method according to any or all of the preceding examples, further comprising:

[0135] In response to determining that the detected object is not located within the planned path, changing the visual path indication to a first color; and

[0136] In response to determining that the detected object is located within the planned path, the visual path indication is changed to a second color.

[0137] Example 6 is a method according to any or all of the preceding examples, wherein the mobile work machine includes ground-engaging traction elements, and determining the planned path of the mobile work machine includes:

[0138] determining a differential drive speed of the first ground-engaging traction element relative to the second ground-engaging traction element; and

[0139] The planned path is determined based on the differential drive speed.

[0140] Example 7 is a method according to any or all of the preceding examples, wherein the mobile work machine comprises an articulated machine, and determining the planned path of the mobile work machine comprises:

[0141] The planned path is determined based on an articulation angle of the articulated machine.

[0142] Example 8 is a method according to any or all of the preceding examples, wherein the indication of the object is generated by an object detection system that sends a detection signal and receives a reflection of the detection signal.

[0143] Example 9 is a method according to any or all of the preceding examples, wherein the detection signal comprises a radar signal.

[0144] Example 10 is a method according to any or all of the preceding examples, further comprising:

[0145] evaluating the object by performing image processing on the portion of the image; and

[0146] The visual object indication is generated based on the evaluation.

[0147] Example 11 is a method according to any or all of the preceding examples, wherein:

[0148] Evaluating the object includes determining a likelihood that detection of the object by the object detection system includes a false positive detection; and

[0149] The visual object indication includes a false positive indication based on the likelihood.

[0150] Example 12 is a method according to any or all of the preceding examples, further comprising:

[0151] applying an image classifier to the portion of the image to determine an object type of the object; and

[0152] The visual object indication is displayed based on the determined object type.

[0153] Example 13 is a method according to any or all of the preceding examples, wherein the mobile work machine includes a frame and an implement supported by the frame and configured to move material on the work site.

[0154] Example 14 is a mobile working machine, comprising:

[0155] an object detection sensor configured to generate a signal indicative of an object detected on the work site;

[0156] object position determination logic configured to determine a position of the object relative to the mobile work machine;

[0157] an image association logic system configured to receive an image of the work site and associate the determined position of the object with a portion of the image;

[0158] A control signal generator logic system is configured to generate a control signal that controls a display device to display a representation of the image with a visual object indication representing a detected object on the portion of the image.

[0159] Example 15 is a mobile work machine according to any or all of the preceding examples, further comprising:

[0160] a path determination logic system configured to determine a planned path for the mobile work machine; and

[0161] The control signal generator logic system is configured to control the display device to present a visual path indication on the image that indicates the planned path, and the control signal generator logic system is configured to change a visual characteristic of the visual path indication based on a determination that at least a portion of the detected object is located in the planned path.

[0162] Example 16 is the mobile work machine of any or all of the preceding examples, wherein the mobile work machine includes ground-engaging traction elements, and the path determination logic system is configured to determine the planned path based on at least one of:

[0163] a differential drive speed of a first ground-engaging traction element relative to a second ground-engaging traction element; or

[0164] The articulation angle of the mobile work machine.

[0165] Example 17 is the mobile work machine of any or all of the preceding examples, wherein the object detection sensor is configured to transmit a detection signal, receive a reflection of the detection signal, and generate the signal based on the received reflection, and the mobile work machine further comprises:

[0166] Image processing logic is configured to perform image processing on the portion of the image, wherein the control signal generator logic is configured to generate the visual object indication based on the evaluation.

[0167] Example 18 is a mobile work machine according to any or all of the preceding examples, further comprising:

[0168] racks; and

[0169] A tool is supported by the frame and configured to move material on the job site.

[0170] Example 19 is a method of controlling a mobile work machine on a work site, the method comprising:

[0171] receiving an indication of an object detected on the work site;

[0172] determining a position of the object relative to the mobile work machine;

[0173] receiving an image of the work site;

[0174] associating the determined position of the object with a portion of the image;

[0175] determining a planned path for the mobile work machine; and

[0176] A control signal is generated that controls a display device to display a representation of the image having a visual object indication indicating a detected object on the portion of the image and a visual path indication indicating the planned path.

[0177] Example 20 is a method according to any or all of the preceding examples, wherein the indication of the object is generated by an object detection system that transmits a radar signal and receives reflections of the radar signal, and the method further comprises:

[0178] evaluating the object by performing image processing on the portion of the image; and

[0179] The visual object indication is generated based on the evaluation.

[0180] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed merely as exemplary forms of implementing the claims.

Claims

1. A method for controlling a mobile work machine at a work site, the method comprising: receiving an indication of an object detected on the work site; determining a position of the object relative to the mobile work machine; receiving an image of the work site; associating the determined position of the object with a portion of the image; generating a control signal that controls a display device to display a representation of the image with a visual object indication representative of the detected object on the portion of the image; evaluating the object, which includes determining a likelihood that detection of the object by an object detection system includes a false positive detection; the visual object indication including a false positive indication based on the likelihood, wherein determining the likelihood that detection of the object by the object detection system includes a false positive detection includes generating a metric or score indicating a likelihood that the portion of the image represents an actual object on the work site, then comparing the metric or score to a threshold, and determining whether the detected object is a false positive based on the comparison.

2. The method according to claim 1, further comprising: determining a planned path for the mobile working machine; The display device is controlled to present a visual path indication representing the planned path on the image.

3. The method according to claim 2, further comprising: A visual characteristic of the visual path indication is changed based on a determination that at least a portion of the detected object is located within the planned path.

4. The method according to claim 3, wherein: Changing a visual characteristic of the visual path indication includes changing a display color of the visual path indication.

5. The method according to claim 3, further comprising: In response to determining that the detected object is not located within the planned path, changing the visual path indication to a first color; as well as In response to determining that the detected object is located within the planned path, the visual path indication is changed to a second color.

6. The method according to claim 2, wherein: The mobile work machine includes ground-engaging traction elements, and determining the planned path of the mobile work machine includes: determining a differential drive speed of the first ground-engaging traction element relative to the second ground-engaging traction element; and The planned path is determined based on the differential drive speed.

7. The method according to claim 2, wherein: The mobile work machine includes an articulated machine, and determining the planned path of the mobile work machine includes: The planned path is determined based on an articulation angle of the articulated machine.

8. The method according to claim 1, wherein The indication of the object is generated by the object detection system, which sends a detection signal and receives a reflection of the detection signal.

9. The method according to claim 8, wherein The detection signal includes a radar signal.

10. The method according to claim 8, further comprising: evaluating the object by performing image processing on the portion of the image; as well as The visual object indication is generated based on the evaluation.

11. The method according to claim 1 , further comprising: applying an image classifier to the portion of the image to determine an object type of the object; as well as The visual object indication is displayed based on the determined object type.

12. The method according to claim 1, wherein The mobile work machine includes a frame and an implement supported by the frame and configured to move material on the work site.

13. A mobile working machine comprising: an object detection sensor configured to generate a signal indicative of an object detected on the work site; object position determination logic configured to determine a position of the object relative to the mobile work machine; an image association logic system configured to receive an image of the work site and associate the determined position of the object with a portion of the image; a control signal generator logic system configured to generate a control signal that controls a display device to display a representation of the image with a visual object indication representing a detected object on the portion of the image, image processing logic configured to perform image processing on the portion of the image to evaluate the object, wherein the control signal generator logic is configured to generate the visual object indication based on the evaluation, wherein evaluating the object comprises determining a likelihood that detection of the object by an object detection system comprises a false positive detection; and the visual object indication comprises a false positive indication based on the likelihood, wherein determining the likelihood that detection of the object by the object detection system includes a false positive detection includes generating a metric or score indicating a likelihood that the portion of the image represents an actual object on the work site, then comparing the metric or score to a threshold, and determining whether the detected object is a false positive based on the comparison.

14. The mobile work machine according to claim 13, further comprising: a path determination logic system configured to determine a planned path for the mobile work machine; as well as The control signal generator logic system is configured to control the display device to present a visual path indication on the image that indicates the planned path, and the control signal generator logic system is configured to change a visual characteristic of the visual path indication based on a determination that at least a portion of the detected object is located in the planned path.

15. The mobile working machine according to claim 14, wherein: The mobile work machine includes ground-engaging traction elements, and the path determination logic system is configured to determine the planned path based on at least one of: a differential drive speed of a first ground-engaging traction element relative to a second ground-engaging traction element; or The articulation angle of the mobile work machine.

16. The mobile working machine according to claim 13, wherein: The object detection sensor is configured to transmit a detection signal, receive a reflection of the detection signal, and generate the signal based on the received reflection.

17. The mobile work machine according to claim 13, further comprising: frame; as well as A tool is supported by the frame and configured to move material on the job site.

18. A method of controlling a mobile work machine at a work site, the method comprising: receiving an indication of an object detected on the work site; determining a position of the object relative to the mobile work machine; receiving an image of the work site; associating the determined position of the object with a portion of the image; determining a planned path for the mobile working machine; generating a control signal that controls a display device to display a representation of the image having a visual object indication indicating a detected object on the portion of the image and a visual path indication indicating the planned path; as well as evaluating the object, which includes determining a likelihood that detection of the object by an object detection system includes a false positive detection; the visual object indication including a false positive indication based on the likelihood, wherein determining the likelihood that detection of the object by the object detection system includes a false positive detection includes generating a metric or score indicating a likelihood that the portion of the image represents an actual object on the work site, then comparing the metric or score to a threshold, and determining whether the detected object is a false positive based on the comparison.

19. The method according to claim 18, wherein The indication of the object is generated by an object detection system that transmits a radar signal and receives reflections of the radar signal, and the method further comprises: evaluating the object by performing image processing on the portion of the image; and The visual object indication is generated based on the evaluation.

Images