Display system for work machine, work machine, and display method for work machine

Abstract

This display system for a work machine comprises: an acquisition unit that acquires detection data including an image of the periphery of a work machine; a level difference detection unit that detects, on the basis of the detection data, any level difference between a first ground surface of a work site on which the work machine travels and a second ground surface present below the first ground surface; a display data generation unit that generates display data including the image of the periphery and a position image indicating the position of the level difference; and an output unit that displays the display data on the display device.

Description

Work machine display system, work machine, and work machine display method

[0001] The present disclosure relates to a work machine display system, a work machine, and a work machine display method.

[0002] BACKGROUND ART In the technical field of work machines, a bulldozer such as that disclosed in Patent Document 1 is known.

[0003] JP 2012-062620 A

[0004] A work machine travels through a work site. There may be large steps, such as cliffs, at the work site. If the steps are in the blind spot of the work machine operator, it may be difficult for the operator to recognize the steps. Furthermore, depending on the condition of the ground at the work site, it may be difficult for the operator to recognize the steps. The ground at the work site is often earth and sand. If the ground at the work site is earth and sand, it may be difficult for the operator to see the steps at the work site. Furthermore, there is a high possibility that the ground at the work site is uneven. If the ground at the work site is uneven, it may be difficult for the operator to see the steps at the work site.

[0005] The present disclosure aims to make it easier for a work machine operator to recognize steps that exist at a work site.

[0006] According to the present disclosure, there is provided a work machine display system comprising: an acquisition unit that acquires detection data including a peripheral image that shows an image of the area around the work machine; a step detection unit that detects a step between a first ground surface at a work site where the work machine is traveling and a second ground surface that is located below the first ground surface based on the detection data; a display data generation unit that generates display data that includes the peripheral image and a position image that shows the position of the step; and an output unit that displays the display data on a display device.

[0007] According to the present disclosure, it becomes easier for a work machine operator to recognize steps that exist at a work site.

[0008] FIG. 1 is a side view schematically showing a work machine according to the first embodiment. FIG. 2 is a plan view schematically showing a work machine according to the first embodiment. FIG. 3 is a view schematically showing a cab of a work machine according to the first embodiment. FIG. 4 is a hardware configuration diagram showing a display controller according to the first embodiment. FIG. 5 is a block diagram showing a display system for a work machine according to the first embodiment. FIG. 6 is a flowchart showing a display method for a work machine according to the first embodiment. FIG. 7 is a diagram illustrating the operation of a work machine according to the first embodiment. FIG. 8 is a diagram illustrating the operation of a work machine according to the first embodiment. FIG. 9 is a diagram illustrating an example of display data displayed on a display device according to the first embodiment. FIG. 10 is a diagram illustrating an example of display data displayed on a display device according to the first embodiment. FIG. 11 is a diagram illustrating an example of display data displayed on a display device according to the first embodiment. FIG. 12 is a diagram illustrating an example of display data displayed on a display device according to the first embodiment. FIG. 13 is a diagram schematically showing a remote control system for a work machine according to a second embodiment.

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.

[0010] In the embodiment, the positional relationship of each part is described using the terms front, rear, left, right, top, and bottom. These terms indicate relative positions or directions based on the center of the work machine 1. These terms may be considered to be positions or directions in a local coordinate system set for the work machine 1.

[0011] First Embodiment A first embodiment will be described.

[0012] <Working Machine> Fig. 1 is a side view that schematically shows a working machine 1 according to an embodiment. Fig. 2 is a plan view that schematically shows a working machine 1 according to an embodiment. The working machine 1 works at a work site. In the embodiment, the working machine 1 is a bulldozer. As shown in Figs. 1 and 2 , the working machine 1 includes a vehicle body 2, a traveling device 3, a working implement 4, a vehicle body controller 5, a monitoring device 6, a display device 7, and a display controller 8.

[0013] The vehicle body 2 has a cab 15. An operator of the work machine 1 sits in the cab 15. The traveling device 3 supports the vehicle body 2 and travels. The traveling device 3 has a pair of tracks 9. The tracks 9 rotate to allow the work machine 1 to travel around the work site.

[0014] The work implement 4 performs excavation work, soil dozing work, or ground leveling work. The work implement 4 is attached to the vehicle body 2. At least a portion of the work implement 4 is disposed in front of the vehicle body 2. The work implement 4 has an excavation blade 10, a lift frame 11, a tilt cylinder 12, and a lift cylinder 13.

[0015] The excavating blade 10 is disposed in front of the vehicle body 2. A lift frame 11 supports the excavating blade 10. The tilt cylinder 12 and the lift cylinder 13 each operate the excavating blade 10. The tilt cylinder 12 is driven to tilt the excavating blade 10. The lift cylinder 13 is driven to move the excavating blade 10 up and down. The tilt angle of the excavating blade 10 changes as the tilt cylinder 12 extends and retracts. The excavating blade 10 moves up and down as the lift cylinder 13 extends and retracts.

[0016] The vehicle body controller 5 includes a computer. The vehicle body controller 5 outputs control commands for controlling the traveling device 3 and the work machine 4, respectively.

[0017] The monitoring device 6 monitors the periphery of the work machine 1. The monitoring device 6 includes an imaging device 61 that images the periphery of the work machine 1, and a three-dimensional measuring device 62 that detects the three-dimensional shape of the periphery of the work machine 1. The monitoring device 6 is disposed on the vehicle body 2. In the embodiment, the monitoring devices 6 are disposed at the front and rear of the vehicle body 2.

[0018] The objects monitored by the monitoring device 6 include objects in the vicinity of the work machine 1. The objects monitored by the monitoring device 6 are the objects imaged by the imaging device 61. The objects monitored by the monitoring device 6 are the objects detected by the three-dimensional measuring device 62. The objects in the vicinity of the work machine 1 include the ground 100 at the work site.

[0019] The three-dimensional measurement device 62 detects the three-dimensional shape of the monitored object without contacting the monitored object. The three-dimensional measurement device 62 detects the distance to the surface of the monitored object. The three-dimensional measurement device 62 detects the three-dimensional shape of the monitored object's surface by detecting the relative distance or relative position to each of a plurality of detection points DP on the monitored object's surface. Shape data indicating the three-dimensional shape of the monitored object includes three-dimensional point cloud data consisting of the plurality of detection points DP. The shape data includes the relative distance or relative position between the three-dimensional measurement device 62 and each of the plurality of detection points DP on the monitored object's surface. The shape data includes height data for each of the plurality of detection points DP.

[0020] A stereo camera is exemplified as the monitoring device 6. The stereo camera has the functions of an imaging device 61 and a three-dimensional measuring device 62. The monitoring device 6 may include a monocular camera and a three-dimensional optical sensor. An example of the three-dimensional optical sensor is a laser sensor (LIDAR: Light Detection and Ranging) that detects a monitoring target by emitting laser light.

[0021] As shown in FIG. 2 , the monitoring device 6 has a monitoring area 14. The monitoring device 6 monitors a monitoring target placed in the monitoring area 14. The monitoring area 14 is an imaging area of ​​an imaging device 61. The monitoring area 14 is a detection area of ​​a three-dimensional measuring device 62. The imaging area of ​​the imaging device 61 and the detection area of ​​the three-dimensional measuring device 62 at least partially coincide with each other. In an embodiment, the imaging area of ​​the imaging device 61 and the detection area of ​​the three-dimensional measuring device 62 substantially coincide with each other. The imaging device 61 captures an image of the monitoring target placed in the monitoring area 14. The three-dimensional measuring device 62 detects the three-dimensional shape of the monitoring target placed in the monitoring area 14.

[0022] The monitoring device 6 includes a monitoring device 6F arranged at the front of the vehicle body 2, and a monitoring device 6R arranged at the rear of the vehicle body 2. The monitoring area 14 includes a monitoring area 14F of monitoring device 6F and a monitoring area 14R of monitoring device 6R. At least a portion of the monitoring area 14F is defined in front of the work machine 1. At least a portion of the monitoring area 14R is defined behind the work machine 1. The monitoring device 6F monitors a monitoring target in front of the work machine 1. The imaging device 61 of the monitoring device 6F images the monitoring target in front of the work machine 1. The 3D measurement device 62 of the monitoring device 6F detects the three-dimensional shape of the monitoring target in front of the work machine 1. The monitoring device 6R monitors a monitoring target behind the work machine 1. The imaging device 61 of the monitoring device 6R images the monitoring target behind the work machine 1. The 3D measurement device 62 of the monitoring device 6R detects the three-dimensional shape of the monitoring target behind the work machine 1.

[0023] 3 is a diagram schematically showing the cab 15 of the work machine 1 according to the embodiment. The cab 15 is provided on the vehicle body 2. The work machine 1 is equipped with a travel operation device 16, a work operation device 17, a display device 7, and a display controller 8. The travel operation device 16, the work operation device 17, the display device 7, and the display controller 8 are each disposed in the cab 15 of the work machine 1. Note that the display controller 8 does not have to be disposed in the cab 15. The display controller 8 may be disposed in any position on the vehicle body 2.

[0024] A seat 18 is arranged in the operator's cab 15. The operator of the work machine 1 can operate each of the travel operation device 16 and the work operation device 17 while sitting in the seat 18. In the example shown in FIG. 3 , the travel operation device 16 is arranged on the left side of the seat 18. The work operation device 17 is arranged on the right side of the seat 18. The display device 7 is arranged on the front side of the seat 18.

[0025] The travel operation device 16 includes an operation lever. The operator operates the travel operation device 16 when traveling the work machine 1. An operation signal generated by operating the travel operation device 16 is transmitted to the vehicle body controller 5. The vehicle body controller 5 travels the work machine 1 based on the operation signal from the travel operation device 16. The vehicle body controller 5 travels the work machine 1 forward or backward based on the operation signal from the travel operation device 16.

[0026] The work operation device 17 includes an operation lever. When the operator operates the work operation device 17, an operation signal generated by operating the work operation device 17 is transmitted to the vehicle body controller 5. The vehicle body controller 5 operates the work machine 4 based on the operation signal from the work operation device 17.

[0027] At least one of the travel operation device 16 and the work operation device 17 does not have to be an operation lever, and may be, for example, a pedal or a switch.

[0028] The display device 7 provides display data to the operator. The display device 7 includes a flat panel display such as a liquid crystal display or an organic EL display. The display controller 8 generates display data and causes the display device 7 to display the data.

[0029] <Display Controller> FIG. 4 is a hardware configuration diagram showing a display controller 8 according to an embodiment. The display controller 8 includes a computer. The display controller 8 has a processor 8A such as a CPU (Central Processing Unit), a main memory 8B including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 8C, and an interface 8D including an input / output circuit and a communication circuit. The functions of the display controller 8 are stored in the storage 8C as a computer program. The processor 8A reads the computer program from the storage 8C, loads it into the main memory 8B, and executes processing in accordance with the computer program. The computer program may be distributed to the display controller 8 via a network.

[0030] The vehicle body controller 5 also includes a computer. Like the display controller 8, the vehicle body controller 5 has a processor, a main memory, a storage, and an interface.

[0031] <Display System> Figure 5 is a block diagram showing a display system 20 of a work machine 1 according to an embodiment. The work machine 1 is equipped with a display system 20. The display system 20 has a monitoring device 6, a travel operation device 16, a display controller 8, and a display device 7. The monitoring device 6 and the display controller 8 are capable of communication. The travel operation device 16 and the display controller 8 are capable of communication. The display controller 8 and the display device 7 are capable of communication. The display controller 8 has an acquisition unit 21, a step detection unit 22, a display data generation unit 23, and an output unit 24.

[0032] The acquisition unit 21 acquires detection data including a peripheral image 30 that shows an image of the periphery of the work machine 1. The detection data includes shape data that shows the three-dimensional shape of the periphery of the work machine 1. The acquisition unit 21 acquires the peripheral image 30 from the imaging device 61. The acquisition unit 21 acquires the shape data from the three-dimensional measurement device 62.

[0033] The step detection unit 22 detects steps at the work site based on the shape data acquired by the acquisition unit 21. As described above, the object monitored by the monitoring device 6 includes the ground surface 100 at the work site. The three-dimensional shape of the area around the work machine 1 includes the three-dimensional shape of the ground surface 100 at the work site. The three-dimensional shape of the area around the work machine 1 includes the topography of the work site.

[0034] The step detection unit 22 detects steps between the first ground surface 101 at the work site where the work machine 1 is traveling and the second ground surface 102 that exists below the first ground surface 101, based on the shape data acquired by the acquisition unit 21. The step detection unit 22 determines whether or not there are steps at the work site, based on the shape data acquired by the acquisition unit 21. Steps at the work site include cliffs. In the following description, steps at the work site will be referred to as cliffs 105 as appropriate.

[0035] The step detection unit 22 detects cliffs 105 in the traveling direction of the work machine 1. The traveling direction of the work machine 1 includes a forward direction in front of the work machine 1 and a reverse direction behind the work machine 1. When the work machine 1 is traveling forward, the step detection unit 22 determines whether or not there is a cliff 105 in front of the work machine 1 based on shape data detected by the three-dimensional measurement device 62 of the monitoring device 6F. When the work machine 1 is traveling backward, the step detection unit 22 determines whether or not there is a cliff 105 behind the work machine 1 based on shape data detected by the three-dimensional measurement device 62 of the monitoring device 6R. The step detection unit 22 receives an operation signal from the travel operation device 16. The step detection unit 22 can determine the traveling direction of the work machine 1 based on the operation signal from the travel operation device 16. The step detection unit 22 can determine whether the work machine 1 is traveling forward or backward based on the operation signal from the travel operation device 16.

[0036] When the step detection unit 22 detects a cliff 105, the display data generation unit 23 generates a position image 31 that indicates the position of the cliff 105. The display data generation unit 23 generates display data that includes a peripheral image 30 and a position image 31 acquired by the acquisition unit 21. When the work machine 1 is moving forward, the display data generation unit 23 generates display data that includes a peripheral image 30 in front of the work machine 1 that has been captured by the imaging device 61 of the monitoring device 6F, and a position image 31 that indicates the position of the cliff 105 that exists in front of the work machine 1. When the work machine 1 is moving backward, the display data generation unit 23 generates display data that includes a peripheral image 30 behind the work machine 1 that has been captured by the imaging device 61 of the monitoring device 6R, and a position image 31 that indicates the position of the cliff 105 that exists behind the work machine 1. The display data generation unit 23 can determine whether the work machine 1 is moving forward or backward, based on the operation signal from the travel operation device 16.

[0037] The output unit 24 displays the display data generated by the display data generation unit 23 on the display device 7. When the work machine 1 is moving forward, the output unit 24 displays on the display device 7 display data including at least an image 30 of the surroundings in front of the work machine 1 captured by the imaging device 61 of the monitoring device 6F. When the work machine 1 is moving in reverse, the output unit 24 displays on the display device 7 display data including at least an image 30 of the surroundings behind the work machine 1 captured by the imaging device 61 of the monitoring device 6R. The output unit 24 can determine whether the work machine 1 is moving forward or in reverse based on an operation signal from the travel operation device 16. When the travel operation device 16 is operated so that the work machine 1 moves forward, the output unit 24 displays on the display device 7 display data including at least an image 30 of the surroundings in front of the work machine 1. When the travel operation device 16 is operated so that the work machine 1 moves in reverse, the output unit 24 displays on the display device 7 display data including at least an image 30 of the surroundings behind the work machine 1. The display device 7 can function as a so-called backup monitor.

[0038] <Display method> Figure 6 is a flowchart showing a display method for the work machine 1 according to the embodiment. In the following description, it is assumed that the monitoring device 6 is a stereo camera. The stereo camera has a first camera and a second camera. The stereo camera has the functions of an imaging device 61 and a three-dimensional measuring device 62. One of the first camera and the second camera functions as the imaging device 61 (single camera). An image captured by one of the first camera and the second camera (single camera image) is a surrounding image 30 that shows an image of the surroundings of the work machine 1. Shape data that shows the three-dimensional shape of the surroundings of the work machine 1 is calculated by stereo processing the image captured by the first camera and the image captured by the second camera.

[0039] The work machine 1 travels on the ground 100 at a work site. The monitoring device 6, which includes a first camera and a second camera, captures images of the surroundings of the work machine 1. The acquisition unit 21 acquires images captured by the first camera and images captured by the second camera (step S1).

[0040] The acquisition unit 21 performs stereo processing on the image captured by the first camera and the image captured by the second camera to generate three-dimensional image data of the monitored object (step S2). The three-dimensional image data of the monitored object includes a parallax image of the monitored object. The three-dimensional image data of the monitored object includes distance data from the monitoring device 6 to the surface of the monitored object.

[0041] The acquisition unit 21 generates three-dimensional point cloud data of the monitoring target from the three-dimensional image data of the monitoring target generated in step S2 (step S3).

[0042] The shape data indicating the three-dimensional shape of the monitored object includes three-dimensional point cloud data of the monitored object. The three-dimensional point cloud data includes a point cloud defined on the surface of the monitored object. The three-dimensional point cloud data is made up of a plurality of detection points DP defined on the surface of the monitored object in the periphery of the work machine 1. The monitored object in the periphery of the work machine 1 includes the ground surface 100 in the periphery of the work machine 1. The acquisition unit 21 can recognize the three-dimensional shape of the ground surface 100 in the periphery of the work machine 1 by performing point cloud processing on the three-dimensional point cloud data.

[0043] Figures 7 and 8 are diagrams illustrating the operation of the work machine 1 according to the embodiment. Figures 7 and 8 each show a state in which the work machine 1 is reversing. The direction of travel of the work machine 1 is reverse. When the work machine 1 is reversing, the acquisition unit 21 performs stereo processing on images taken by the first camera and the second camera of the monitoring device 6R provided at the rear of the vehicle body 2, to generate three-dimensional image data of the ground surface 100 behind the work machine 1. The acquisition unit 21 generates three-dimensional point cloud data of the ground surface 100 behind the work machine 1.

[0044] The work machine 1 travels on a ground surface 100 at a work site. In the example shown in Figures 7 and 8, the ground surface 100 includes a first ground surface 101 and a second ground surface 102 that exists below the first ground surface 101. The work machine 1 travels on the first ground surface 101 at a work site. The second ground surface 102 exists behind the first ground surface 101.

[0045] A plurality of detection points DP are defined on the surface of the ground 100. Each of the plurality of detection points DP includes height data. The step detection unit 22 classifies each of the plurality of detection points DP into mutually different height groups. For example, if the height of the ground with which the crawler 9 of the work machine 1 is in contact is defined as a reference height Ho, the step detection unit 22 classifies detection points DP that exist in a first height range from [Ho] to [Ho-1×α] into a first height group. The step detection unit 22 classifies detection points DP that exist in a second height range from [Ho-1×α] to [Ho-2×α] into a second height group. The step detection unit 22 classifies detection points DP that exist in a third height range from [Ho-2×α] to [Ho-3×α] into a third height group. Similarly, the step detection unit 22 classifies detection points DP that exist in the nth height range from [Ho-(n-1)×α] to [Ho-n×α] into the nth height group. α is, for example, 0.1 m. n is a natural number. Note that the first height group may include detection points DP that exist at positions higher than the reference height Ho. The first height group may also include detection points DP that exist in the range of [Ho+1×α].

[0046] The step detection unit 22 determines whether the difference in height ΔH between adjacent height groups in the direction of travel of the work machine 1 is equal to or greater than a predetermined threshold value Sh. When the detection point DP belonging to one of the height groups adjacent to each other in the direction of travel of the work machine 1 is designated as the first detection point DP1 and the detection point DP belonging to the other height group is designated as the second detection point DP2, the step detection unit 22 determines whether the difference in height ΔH between the first detection point DP1 and the second detection point DP2 is equal to or greater than a predetermined threshold value Sh (step S4). The threshold value is, for example, 1.5 m.

[0047] In step S4, if it is determined that the difference ΔH between the height of the first detection point DP1 and the height of the second detection point DP2 is greater than or equal to the threshold value Sh (step S4: Yes), the step detection unit 22 determines that there is a cliff 105 between the first detection point DP1 and the second detection point DP2 (step S5).

[0048] In the example shown in FIG. 7 , the first ground surface 101 and the second ground surface 102 are adjacent to each other in the front-to-rear direction. A first detection point DP1 is defined on the first ground surface 101, and a second detection point DP2 is defined on the second ground surface 102. In the example shown in FIG. 7 , each of the first ground surface 101 and the second ground surface 102 is substantially flat. The first detection point DP1 belongs to a first height group. The second detection point DP2 belongs to a height group lower than the first height group. In the example shown in FIG. 7 , the difference ΔH between the heights of the first detection point DP1 and the second detection point DP2 is equal to or greater than the threshold value Sh. Therefore, the step detection unit 22 determines that a cliff 105 exists between the first ground surface 101 and the second ground surface 102.

[0049] In the example shown in FIG. 8 , the first ground surface 101 and the second ground surface 102 are adjacent to each other in the front-to-rear direction. A first detection point DP1 is defined on the first ground surface 101, and a second detection point DP2 is defined on the second ground surface 102. In the example shown in FIG. 8 , the first ground surface 101 is substantially flat. The second ground surface 102 slopes downward from the rear end of the first ground surface 101 toward the rear. The first detection point DP1 belongs to a first height group. The second detection point DP2 belongs to a height group lower than the first height group. In the example shown in FIG. 8 , the difference ΔH between the height of the first detection point DP1 and the height of the front second detection point DP2 is less than the threshold value Sh. Therefore, the step detection unit 22 determines that there is no cliff 105 between the first ground surface 101 and the second ground surface 102.

[0050] Note that the step detection unit 22 may determine that a cliff 105 exists between the first detection point DP1s and the second detection point DP2s when the difference in height ΔH between the first detection point DP1s and the second detection point DP2s that are adjacent to each other in the traveling direction of the work machine 1 is equal to or greater than the threshold value Sh. In the example shown in Fig. 7, the difference in height ΔH between the first detection point DP1s and the second detection point DP2s that are adjacent to each other in the front-to-rear direction is equal to or greater than the threshold value Sh. In the example shown in Fig. 8, the difference in height ΔH between the first detection point DP1s and the second detection point DP2s that are adjacent to each other in the front-to-rear direction is less than the threshold value Sh.

[0051] If it is determined that a cliff 105 exists in the direction of travel of the work machine 1, the display data generator 23 generates a position image 31 that indicates the position of the cliff 105 (step S6).

[0052] The display data generation unit 23 generates display data including the peripheral image 30 of the work machine 1 acquired in step S1 and the position image 31 generated in step S6 (step S7). The output unit 24 displays the display data generated in step S7 on the display device 7 (step S8).

[0053] In step S4, if it is determined that the difference ΔH between the height of the first detection point DP1 and the height of the second detection point DP2 is not greater than the threshold value Sh (step S4: No), the step detection unit 22 determines that there is no cliff 105 between the first detection point DP1 and the second detection point DP2 (step S9).

[0054] If it is determined that there is no cliff 105 in the direction of travel of the work machine 1, the display data generation unit 23 does not generate the position image 31. The display data generation unit 23 generates display data that does not include the position image 31 but includes the surrounding image 30 (step S10). The output unit 24 displays the display data generated in step S10 on the display device 7 (step S8).

[0055] Fig. 9 is a diagram showing an example of display data displayed on the display device 7 according to the embodiment. As shown in Fig. 9, the display data includes a peripheral image 30 showing an image behind the work machine 1, and a position image 31 showing the position of a cliff 105. The display data generation unit 23 generates the display data so that the position image 31 is superimposed on the peripheral image 30.

[0056] The display data generation unit 23 generates display data such that the position image 31 is superimposed on the cliff 105 in the peripheral image 30. The display data generation unit 23 generates display data such that the position image 31 is superimposed on an edge of the first ground surface 101 in the peripheral image 30. The edge of the first ground surface 101 is the boundary between the first ground surface 101 and the cliff 105. The edge of the first ground surface 101 may be considered to be the upper edge of the cliff 105. By displaying the position image 31, the cliff 105 is emphasized in the peripheral image 30.

[0057] In the embodiment, the position image 31 is a line extending along the longitudinal direction of the edge of the first ground surface 101 in the peripheral image 30. In the example shown in Fig. 9, the edge of the first ground surface 101 is long in the left-right direction. The position image 31 is a line extending in the left-right direction on the display screen of the display device 7.

[0058] The position image 31 does not have to be a line extending along the longitudinal direction of the cliff 105. It may be a symbol (icon) superimposed on the cliff 105 in the peripheral image 30.

[0059] In the embodiment, when the work machine 1 moves backward, the position image 31 is displayed so as to emphasize the cliff 105 between the first ground surface 101 and the second ground surface 102 that is located below and to the rear of the first ground surface 101. In other words, when the work machine 1 moves backward, the display data generation unit 23 generates the position image 31 so as to emphasize the cliff 105 that faces backward. Note that there may be cases where a cliff that faces forward exists at the work site. When the work machine 1 moves backward, the display data generation unit 23 may generate the position image 31 so as to emphasize the cliff 105 that faces forward, or it may not be necessary to generate a position image 31 for the cliff 105 that faces forward.

[0060] A message 40 indicating the distance from the work machine 1 to the cliff 105 is displayed at the top of the display screen of the display device 7. The distance from the work machine 1 to the cliff 105 is the shortest distance between the work machine 1 and the cliff 105. A symbol 32 is displayed at the part of the cliff 105 where the distance to the work machine 1 is the shortest. In this embodiment, the symbol 32 is a circular mark.

[0061] FIG. 10 is a diagram showing an example of display data displayed on the display device 7 according to the embodiment. FIG. 10 shows display data in a state in which the work machine 1 has come even closer to the cliff 105 than in the state shown in FIG. 9. As shown in FIGS. 9 and 10, the display data generator 23 changes the display form of the position image 31 based on the distance between the work machine 1 and the cliff 105. In each of the display data shown in FIG. 9 and the display data shown in FIG. 10, the position image 31 is in the form of a line that follows the longitudinal direction of the edge of the first ground surface 101 in the peripheral image 30. The position image 31A shown in FIG. 9 is displayed in a first color. The position image 31B shown in FIG. 10 is displayed in a second color that is different from the first color. The first color is, for example, green, and the second color is, for example, red. The position image 31A shown in FIG. 9 is displayed as a dotted line. The position image 31B shown in FIG. 10 is displayed as a solid line. In addition, the display data generation unit 23 generates display data so that when the distance from the work machine 1 to the cliff 105 becomes less than a predetermined distance threshold, a message 41 warning the driver is displayed at the top of the display screen.

[0062] Fig. 11 is a diagram showing an example of display data displayed on the display device 7 according to the embodiment. As shown in Fig. 11 , when multiple cliffs 105 are detected in the traveling direction of the work machine 1, the display data generation unit 23 generates display data so that the position image 31 is superimposed on the cliff 105 closest to the work machine 1. In the example shown in Fig. 11 , a cliff 105A is present behind the work machine 1, a cliff 105B is present behind cliff 105A, and a cliff 105C is present behind cliff 105B. Of the multiple cliffs 105A, 105B, and 105C, cliff 105A is closest to the work machine 1. The display data generation unit 23 generates display data so that the position image 31 is superimposed on the cliff 105A closest to the work machine 1, but so that the position image 31 is not superimposed on cliffs 105B and 105C. The display data generation unit 23 generates display data including the position image 31 so that the cliff 105A closest to the work machine 1 is emphasized and the cliffs 105B and 105C are not emphasized.

[0063] FIG. 12 is a diagram illustrating an example of display data displayed on the display device 7 according to the embodiment. As illustrated in FIG. 12, the position image 33 does not have to be linear. As illustrated in FIG. 12, the position image 33 may include a first area image 33A of a first color superimposed on the first ground surface 101 in the peripheral image 30 and a second area image 33B of a second color superimposed on the second ground surface 102. The first area image 33A is displayed so as to cover the first ground surface 101 in the peripheral image 30. The second area image 33B is displayed so as to cover the second ground surface 102 in the peripheral image 30. Each of the first area image 33A and the second area image 33B is a translucent image. The operator can view the peripheral image 30 through the first area image 33A and the second area image 33B. The first color is, for example, translucent yellow, and the second color is, for example, translucent blue. The operator can recognize the position of the cliff 105 from the boundary between the first area image 33A and the second area image 33B.

[0064] 6 to 12 , the processing of the display system 20 when the work machine 1 approaches a cliff 105 while traveling backward has been described. The processing of the display system 20 when the work machine 1 approaches a cliff 105 while traveling forward is the same as the processing of the display system 20 when the work machine 1 approaches a cliff 105 while traveling backward. When the work machine 1 moves forward, the display data generation unit 23 generates a position image 31 based on the shape data acquired from the three-dimensional measurement device 62 of the monitoring device 6F so as to emphasize the cliff 105 between the first ground surface 101 and the second ground surface 102 that exists below and in front of the first ground surface 101. In other words, when the work machine 1 moves forward, the display data generation unit 23 generates a position image 31 so as to emphasize the cliff 105 facing forward. When the work machine 1 moves forward, the display data generation unit 23 may generate a position image 31 so as to emphasize the cliff 105 facing backward, or may not generate a position image 31 for the cliff 105 facing backward.

[0065] <Effects> As described above, the display system 20 of the work machine 1 according to the embodiment comprises an acquisition unit 21 that acquires a peripheral image 30 that shows an image of the periphery of the work machine 1 and shape data that shows the three-dimensional shape of the periphery of the work machine 1; a step detection unit 22 that detects, based on the shape data, a step between a first ground surface 101 at the work site where the work machine 1 is traveling and a second ground surface 102 that is located below the first ground surface 101; a display data generation unit 23 that generates display data including the peripheral image 30 and a position image 31 that shows the position of the step; and an output unit 24 that displays the display data on the display device 7.

[0066] According to the embodiment, the position of the cliff 105 is emphasized by the position image 31. This makes it easier for the operator of the work machine 1 to recognize the cliff 105 that exists at the work site. If the ground 100 at the work site is earth and sand, it may be difficult for the operator to visually recognize the cliff 105 at the work site. Furthermore, if the ground 100 at the work site is uneven, it may be difficult for the operator to visually recognize the cliff 105 at the work site. When the work machine 1 moves in reverse, the cliff 105 may be in the operator's blind spot, so the operator often operates the travel operation device 16 while checking the display data displayed on the display device 7 that functions as a backup monitor. When the work machine 1 moves in reverse, the position of the cliff 105 is emphasized by the position image 31 on the display device 7, making it easier for the operator of the work machine 1 to recognize the cliff 105 that exists at the work site.

[0067] The shape data includes three-dimensional point cloud data made up of a plurality of detection points DP defined on the ground 100 around the work machine 1. If the difference ΔH between the height of the first detection point DP1 and the height of the second detection point DP2 is equal to or greater than a predetermined threshold value Sh, the step detection unit 22 can determine that a cliff 105 exists between the first detection point DP1 and the second detection point DP2.

[0068] The display data generation unit 23 generates display data so that the position image 31 is superimposed on the peripheral image 30. Because the position image 31 is superimposed on the peripheral image 30, the operator of the work machine 1 can easily recognize a cliff 105 that exists at the work site.

[0069] The display data generation unit 23 generates display data so that the position image 31 is superimposed on the cliff 105 in the peripheral image 30. This makes it easier for the operator of the work machine 1 to recognize the cliff 105 that exists at the work site.

[0070] The display data generation unit 23 generates display data such that the position image 31 is superimposed on the edge of the first ground surface 101 in the peripheral image 30. When the work machine 1 moves backward, the display data generation unit 23 generates display data such that the position image 31 is superimposed on the rear edge of the first ground surface 101 in the peripheral image 30. When the work machine 1 moves forward, the display data generation unit 23 generates display data such that the position image 31 is superimposed on the front edge of the first ground surface 101 in the peripheral image 30. This makes it easier for the operator of the work machine 1 to recognize cliffs 105 that exist at the work site.

[0071] The position image 31 is in the form of a line that follows the longitudinal direction of the end of the first ground surface 101 in the peripheral image 30. Because the position image 31 is in the form of a line that follows the longitudinal direction of the cliff 105, it becomes easier for the operator of the work machine 1 to recognize the cliff 105 that exists at the work site.

[0072] The position image 33 may include a first area image 33A in a first color superimposed on the first ground surface 101 in the peripheral image 30, and a second area image 33B in a second color superimposed on the second ground surface 102. This makes it easier for the operator of the work machine 1 to recognize cliffs 105 that exist at the work site.

[0073] The step detection unit 22 detects cliffs 105 in the direction of travel of the work machine 1. When the work machine 1 moves backward, the step detection unit 22 detects cliffs 105 behind the work machine 1. When the work machine 1 moves forward, the step detection unit 22 detects cliffs 105 in front of the work machine 1. This makes it easier for the operator of the work machine 1 to recognize cliffs 105 that exist in the direction of travel of the work machine 1.

[0074] The display data generation unit 23 changes the display format of the position image 31 based on the distance between the work machine 1 and the cliff 105. This makes it easier for the operator of the work machine 1 to recognize the distance to the cliff 105.

[0075] When multiple cliffs 105 (105A, 105B, 105C) are detected in the direction of travel of the work machine 1, the display data generation unit 23 generates display data such that the position image 31 is superimposed on the cliff 105A that is closest to the work machine 1. This makes it easier for the operator of the work machine 1 to recognize the cliff 105A that is closest to the work machine 1.

[0076] Second Embodiment A second embodiment will be described below. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and descriptions of those components will be simplified or omitted.

[0077] Figure 13 is a diagram that schematically shows a remote operation system 50 for a work machine 1 according to an embodiment. In the above-described embodiment, the display device 7 is provided on the work machine 1. As shown in Figure 13, a display device 53 that displays display data may also be located outside the work machine 1.

[0078] The remote operation system 50 remotely controls the work machine 1 working at a work site. At least a part of the remote operation system 50 is arranged in a remote operation room 51. The remote operation room 51 is installed outside the work site. The remote operation system 50 includes a remote operation device 52, a display device 53, and a remote controller 54. The remote operation device 52, the display device 53, and the remote controller 54 are each arranged in the remote operation room 51.

[0079] The remote controller 54 communicates with the body controller 5 and display controller 8 of the work machine 1 via a communication system 60. Examples of the communication system 60 include the Internet, a local area network (LAN), a mobile phone communication network, or a satellite communication network.

[0080] The remote control device 52 is operated by an operator in a remote control room 51. The operator can operate the remote control device 52 while sitting in a seat 55. An operation signal generated by operating the remote control device 52 is transmitted to the vehicle body controller 5 via the remote controller 54 and the communication system 60. The vehicle body controller 5 causes the work machine 1 to travel based on the operation signal from the remote control device 52. The vehicle body controller 5 causes the work machine 1 to move forward or backward based on the operation signal from the remote control device 52. The vehicle body controller 5 operates the work implement 4 based on the operation signal from the remote control device 52.

[0081] The peripheral image 30 captured by the imaging device 61 of the monitoring device 6 and the shape data detected by the three-dimensional measuring device 62 are transmitted to the remote controller 54 via the display controller 8 and the communication system 60. The remote controller 54 detects a cliff 105 based on the shape data. When the remote controller 54 detects the cliff 105, it generates a position image 31 (33) indicating the position of the cliff 105 and causes the display device 53 to display display data including the peripheral image 30 and the position image 31 (33).

[0082] The operator operates the remote control device 52 while checking the surrounding image 30 of the work site displayed on the display device 53. If there is a cliff 105 at the work site, the cliff 105 is highlighted by the position image 31 (33), making it easier for the operator to recognize the cliff 105 present at the work site.

[0083] [Other Embodiments] In the above-described embodiment, the monitoring device 6 monitors a monitoring target in front of or behind the vehicle body 2. The monitoring device 6 may be disposed in any location on the vehicle body 2 so as to be able to monitor any direction of the work machine 1. The any direction includes at least one of the front, rear, left, right, left front, left rear, right front, and right rear directions of the vehicle body 2.

[0084] In the above-described embodiment, the step detection unit 22 may detect a cliff 105 in any direction, regardless of the cliff 105 in the traveling direction of the work machine 1. For example, when the work machine 1 is moving forward, the step detection unit 22 may detect a cliff 105 in front, a cliff 105 behind, or a cliff 105 in any direction. Furthermore, when the work machine 1 is moving backward, the step detection unit 22 may detect a cliff 105 in front, a cliff 105 behind, or a cliff 105 in any direction. Furthermore, when the work machine 1 is stopped, the step detection unit 22 may detect a cliff 105 in front, a cliff 105 behind, or a cliff 105 in any direction.

[0085] In the above-described embodiment, the step detection unit 22 may set the direction in which the monitoring device 6 monitors based on instructions from an input device (not shown). That is, the position or orientation of the monitoring area 14 around the vehicle body 2 may be changed based on input data from the input device. For example, the display device 7 may be a touch panel type, and the direction in which the cliff 105 is detected may be set by operating the touch panel. Alternatively, the direction in which the cliff 105 is detected may be set by a hardware switch disposed in the driver's cab 15 or the remote control room 51.

[0086] In the above-described embodiment, the cliff 105 may be a naturally formed cliff or an artificially formed cliff. Furthermore, the step at the work site does not have to be the cliff 105.

[0087] In the above-described embodiment, the monitoring device 6 does not have to include the three-dimensional measuring device 62. In this case, the level difference detection unit 22 may use detection data including the peripheral image 30 of the monitoring device 6 to detect a level difference between the first ground surface 101 at the work site where the work machine 1 is traveling and the second ground surface 102 that exists below the first ground surface 101, through image processing such as pattern matching. The level difference detection unit 22 may use detection data including the peripheral image 30 of the monitoring device 6 to detect a cliff 105 through image processing such as pattern matching.

[0088] In the above-described embodiment, the acquisition unit 21, the step detection unit 22, the display data generation unit 23, and the output unit 24 may be configured as separate hardware (computers).

[0089] In the above-described embodiment, the work machine 1 is a bulldozer. However, the work machine 1 may be another work machine such as a hydraulic excavator, a wheel loader, or a motor grader.

[0090] 1...work machine, 2...body, 3...traveling device, 4...working machine, 5...body controller, 6...monitoring device, 6F...monitoring device, 6R...monitoring device, 7...display device, 8...display controller, 8A...processor, 8B...main memory, 8C...storage, 8D...interface, 9...track, 10...excavation blade, 11...lift frame, 12...tilt cylinder, 13...lift cylinder, 14...monitoring area, 14F...monitoring area, 14R...monitoring area, 15...operator's cab, 16...traveling operation device, 17...work operation device, 18...seat, 20...display system, 21...acquisition unit, 22...step detection unit, 23...display data generation unit, 24...output unit, 30...peripheral image, 31...position image, 31A...position image, 31B...position image, 32...symbol, 33...position image, 33A...first area image, 33B...second area image, 40...message, 41...message, 50...remote control system, 51...remote control room, 52...remote control device, 53...display device, 54...remote controller, 55...seat, 60...communication system, 61...imaging device, 62...3D measuring device, 100...ground, 101...first ground, 102...second ground, 105...cliff, 105A...cliff, 105B...cliff, 105C...cliff, DP...detection point, DP1...first detection point, DP2...second detection point.

Claims

1. A work machine display system comprising: an acquisition unit that acquires detection data including a peripheral image that shows an image of the area around the work machine; a step detection unit that detects a step between a first ground surface at a work site where the work machine is traveling and a second ground surface that is below the first ground surface based on the detection data; a display data generation unit that generates display data including the peripheral image and a position image that shows the position of the step; and an output unit that displays the display data on a display device.

2. A display system for a work machine as described in claim 1, wherein the detection data includes shape data indicating the three-dimensional shape of the surrounding area, the shape data includes three-dimensional point cloud data consisting of a plurality of detection points defined on the surface of the surrounding area, and the step detection unit determines that there is a cliff between the first detection point and the second detection point when the difference in height between the first detection point and the second detection point is equal to or greater than a predetermined threshold.

3. A work machine display system according to claim 1, wherein the display data generation unit generates the display data so that the position image is superimposed on the peripheral image.

4. A work machine display system according to claim 3, wherein the display data generation unit generates the display data so that the position image is superimposed on a step in the peripheral image.

5. A work machine display system according to claim 4, wherein the display data generation unit generates the display data so that the position image is superimposed on an edge of the first ground surface in the peripheral image.

6. A display system for a work machine according to claim 5, wherein the position image is a line shape that follows the longitudinal direction of the edge of the first ground surface in the peripheral image.

7. A work machine display system as described in claim 1, wherein the position image includes a first area image in a first color superimposed on the first ground surface in the peripheral image, and a second area image in a second color superimposed on the second ground surface.

8. A display system for a work machine according to claim 1, wherein the step detection unit detects steps in the direction of travel of the work machine.

9. A work machine display system according to claim 8, wherein the display data generation unit changes the display format of the position image based on the distance between the work machine and the step.

10. A work machine display system as described in claim 8, wherein, when multiple steps are detected in the traveling direction, the display data generation unit generates the display data so that the position image is superimposed on the step closest to the work machine.

11. A display system for a work machine as described in claim 2, wherein the work machine has an imaging device that images the surroundings and a three-dimensional measuring device that detects the three-dimensional shape of the surroundings, and the acquisition unit acquires the surrounding images from the imaging device and acquires the shape data from the three-dimensional measuring device.

12. A work machine display system according to claim 11, wherein the imaging device captures an image of the area behind the work machine, and the three-dimensional measuring device detects the three-dimensional shape of the area behind the work machine.

13. A work machine comprising the work machine display system according to claim 1.

14. A method for displaying a work machine, comprising: acquiring detection data including a peripheral image showing an image of the periphery of the work machine; detecting a step between a first ground surface at a work site where the work machine is traveling and a second ground surface located below the first ground surface based on the detection data; generating display data including the peripheral image and a position image showing the position of the step; and displaying the display data on a display device.

Images