Display control device, display control method, and working machine

Abstract

One embodiment of the present disclosure is a display control device including: an overhead image generation unit that generates an overhead image of a work machine based on one or more captured images captured by one or more imaging devices provided on an upper swing body of the work machine, the work machine including a lower traveling body and the upper swing body supported on the lower traveling body in a swingable manner; an overlapping unit that generates a display image in which an attention range image indicating a swing attention range accompanying a swing of the upper swing body and a traveling attention range accompanying forward and backward movement of the lower traveling body is overlaid on the overhead image; and a display image output unit that outputs the display image.

Description

Technical Field

[0001] This disclosure relates to display control devices, display control methods, and operating machinery.

[0002] This application claims priority to Japan Patent Application No. 2021-075949, filed on April 28, 2021, the contents of which are incorporated herein by reference. Background Technology

[0003] In the vision assist device for the operating machinery described in Patent Document 1, a display image is displayed on a monitor installed in the operator's cab. Specifically, the display image described in Patent Document 1 consists of an external image of the operating machinery viewed from above, a composite camera image of the machinery's surroundings from a bird's-eye view, and multiple camera images. Furthermore, the external image consists of a bitmap image pre-set for each type of operating machinery, including a rotating body image representing a slewing body. Additionally, a traveling body image representing a moving body is overlaid on the rotating body image representing the slewing body. It should be noted that the external image is a fixed image, while the traveling body image rotates around the center of rotation as the slewing body rotates.

[0004] Prior art literature

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2010-59653 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] According to the vision assistance device for the operating machinery described in Patent Document 1, appropriate vision information can be provided to the driver, enabling the driver to always recognize the situation of the direction of travel. However, the following issues still exist: the driver cannot easily grasp the range of attention that should be paid when the upper rotating body of the operating machinery is turned around, or the range of attention that should be paid when the operating machinery is moving forward or backward by passing the lower traveling body, in view of surrounding obstacles, etc.

[0009] This disclosure is made in view of the above circumstances, and its purpose is to provide a display control device, display control method, and working machinery that enable the driver to easily grasp the scope of attention required while driving the accompanying working machinery.

[0010] Methods for solving problems

[0011] To address the aforementioned issues, one aspect of this disclosure is a display control device comprising: a top-view image generation unit that generates a top-view image of the surroundings of a work machine based on one or more images captured by one or more imaging devices mounted on the upper rotating body of the work machine, which has a lower traveling body and an upper rotating body supported rotatably on the lower traveling body; an overlay unit that generates a display image that overlays an image representing a range of interest indicating rotational movement accompanying the rotation of the upper rotating body and a range of interest indicating forward and backward movement accompanying the movement of the lower traveling body onto the top-view image; and a display image output unit that outputs the display image.

[0012] Invention Effects

[0013] According to the various methods disclosed herein, the driver can easily grasp the scope of attention required while driving the accompanying work machinery. Attached Figure Description

[0014] Figure 1 This is a schematic diagram showing the structure of the working machine according to the embodiment.

[0015] Figure 2 This is a top view schematically showing the shooting range of the multiple cameras equipped on the work machinery of the embodiment.

[0016] Figure 3 This is a diagram showing the internal structure of the driver's cab in an embodiment.

[0017] Figure 4 This is a schematic block diagram showing the structure of the display control system.

[0018] Figure 5 This is a diagram showing an example of a display screen illustrating an implementation method.

[0019] Figure 6 This is a diagram showing an example of a display screen illustrating an implementation method.

[0020] Figure 7 This is a diagram showing an example of a display screen illustrating an implementation method.

[0021] Figure 8 This is a flowchart illustrating the operation of the display control device according to an embodiment.

[0022] Figure 9 These are other examples of display screens showing implementation methods.

[0023] Figure 10 These are other examples of display screens showing implementation methods. Detailed Implementation

[0024] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be noted that in the various figures, the same reference numerals are used for the same or corresponding structures, and descriptions are omitted where appropriate.

[0025] Figure 1 This is a schematic diagram showing the structure of the work machine 100 according to the embodiment. Figure 2 This is a top view schematically showing the shooting range of the multiple cameras (shooting devices) 121A to 121D provided by the work machine 100 of the embodiment. Figure 3 This is a diagram showing the internal structure of the driver's cab 140 according to an embodiment. Figure 4 This is a schematic block diagram showing an example of the structure of the display control system 60. Figures 5-7 This is a diagram showing an example of a display screen illustrating an implementation method. Figure 8 This is a flowchart illustrating an example of the operation of the display control device 61 according to an embodiment. Figures 9-10 These are other examples of display screens showing implementation methods.

[0026] like Figure 1 As shown, in this embodiment, a local coordinate system is established for the operating machinery 100, and the positional relationships of each part are explained with reference to the local coordinate system. In the local coordinate system, the first axis extending along the left-right direction (vehicle width direction) of the operating machinery 100 (upper rotating body 120) is designated as the x-axis, the second axis extending along the front-back direction of the operating machinery 100 is designated as the y-axis, and the third axis extending along the up-down direction of the operating machinery 100 is designated as the z-axis. The x-axis is orthogonal to the y-axis. The y-axis is orthogonal to the z-axis. The z-axis is orthogonal to the x-axis. The arrow direction of the x-axis is left, and the opposite direction is right. The arrow direction of the y-axis is forward, and the opposite direction is backward. The arrow direction of the z-axis is upward, and the opposite direction is downward.

[0027] (Structural example of machine tool 100)

[0028] Figure 1 The following is a structural example of the construction machinery 100 according to the embodiment. The construction machinery 100 operates at a construction site to carry out construction work on objects such as sand and soil. As an example, the construction machinery 100 of the embodiment is a hydraulic excavator. The construction machinery 100 includes a lower traveling body 110, an upper slewing body 120, and a working device 130. The upper slewing body 120 is equipped with a cab 140 and a display and control device 61.

[0029] The lower traveling body 110 supports the working machine 100 so that it can move. The lower traveling body 110 has, for example, a pair of tracks 110a (also called left track 110a) and track 110b (also called right track 110b).

[0030] The upper rotating body 120 is supported on the lower traveling body 110 so as to be able to rotate around the rotation center c. The working device 130 is driven hydraulically. The working device 130 is supported on the front of the upper rotating body 120 so as to be able to be driven in the up and down direction. The cab 140 is a space for the operator (driver) to sit in and operate the working machine 100. The cab 140 is located at the left front of the upper rotating body 120. Here, the part of the upper rotating body 120 in which the working device 130 is installed is referred to as the front part. In addition, regarding the upper rotating body 120, based on the front part, the part opposite is referred to as the rear part, the part on the left is referred to as the left part, and the part on the right is referred to as the right part.

[0031] It should be noted that each of the left and right tracks 110a and 110b can be driven independently by its drive wheels (forward and backward). If the left track 110a and the right track 110b are moved forward simultaneously, the lower traveling body 110 moves forward; if the left track 110a and the right track 110b are moved backward simultaneously, the lower traveling body 110 moves backward. Furthermore, if the drive wheels of one track and the other track are driven in opposite directions—for example, if the right track 110b moves forward while the left track 110a moves backward—the lower traveling body 110 can rotate around its rotation center. This method of rotation is called in-place rotation.

[0032] It should be noted that the rotation center of the lower traveling body 110 when it turns in place can be the same as the rotation center c of the upper rotating body 120, or they can be different. The "rotation center" described in the claims can be either the rotation center when turning in place or the rotation center c of the upper rotating body 120.

[0033] In addition, the operating machine 100 is equipped with a rotation angle sensor 160. The rotation angle sensor 160 measures the rotation angle of the upper rotating body 120 relative to the lower traveling body 110 from a predetermined reference angle. The rotation angle sensor 160 is used to determine the relative positional relationship between the upper rotating body 120 and the lower traveling body 110. The rotation angle sensor 160 can be constructed using, for example, a rotary potentiometer or a rotary encoder.

[0034] (Example of camera structure)

[0035] Multiple cameras (front camera 121A, left-side camera 121B, rear camera 121C, and right-side camera 121D) are arranged around the shooting machine 100 on the upper rotating body 120. It should be noted that... Figure 1In the example shown, the front camera 121A is installed inside the driver's cab 140. Alternatively, when collectively referred to as the front camera 121A, the left-side camera 121B, the rear camera 121C, and the right-side camera 121D, they are referred to as multiple cameras 121A to 121D. Figure 2 The shooting range of the multiple cameras 121A to 121D provided by the work machine 100 of the embodiment is schematically shown.

[0036] Specifically, the upper rotating body 120 is equipped with a front camera 121A for capturing the front area Ra around the upper rotating body 120, a left camera 121B for capturing the left side area Rb around the upper rotating body 120, a rear camera 121C for capturing the rear area Rc around the upper rotating body 120, and a right camera 121D for capturing the right side area Rd around the upper rotating body 120. It should be noted that the shooting ranges of the multiple cameras 121A to 121D may overlap or not overlap. Alternatively, multiple cameras 121A to 121D may be used in place of each other, for example, a single omnidirectional camera.

[0037] The shooting range of multiple cameras 121A to 121D is not limited to Figure 2 The example shown. For example, the shooting of the left front area that can be visually confirmed from the driver's cab 140 can also be omitted. The number of cameras 121A to 121D and the shooting range can also be adjusted accordingly. Figure 1 as well as Figure 2 The examples shown are different.

[0038] (Structural example of working device 130)

[0039] The working device 130 includes a boom 131, a stick 132, a bucket 133, a boom cylinder 131C, a stick cylinder 132C, and a bucket cylinder 133C.

[0040] The base end of the boom 131 is mounted to the upper slewing body 120 via boom pin 131P. The stick 132 connects the boom 131 to the bucket 133. The base end of the stick 132 is mounted to the front end of the boom 131 via stick pin 132P. The bucket 133 has bucket teeth for digging sand and the like, and a receiving portion for collecting the excavated sand and soil. The base end of the bucket 133 is mounted to the front end of the stick 132 via bucket pin 133P.

[0041] Boom cylinder 131C is a hydraulic cylinder used to operate boom 131. The base end of boom cylinder 131C is mounted on upper slewing body 120. The front end of boom cylinder 131C is mounted on boom 131.

[0042] The boom cylinder 132C is a hydraulic cylinder used to drive the boom 132. The base end of the boom cylinder 132C is mounted on the boom 131. The front end of the boom cylinder 132C is mounted on the boom 132.

[0043] Bucket cylinder 133C is a hydraulic cylinder used to drive bucket 133. The base end of bucket cylinder 133C is mounted on stick 132. The front end of bucket cylinder 133C is mounted on a connecting rod member connected to bucket 133.

[0044] (Structural example of driver's cab 140)

[0045] Figure 3 An example of the internal structure of the driver's cab 140 according to the embodiment is shown. The driver's seat 141, the operating device 142, and the display input device 145 are provided in the driver's cab 140.

[0046] The operating device 142 is a device for driving the lower traveling body 110, the upper rotating body 120, and the working device 130 through manual operation by an operator. The operating device 142 includes a left operating lever 142LO, a right operating lever 142RO, a left foot pedal 142LF, a right foot pedal 142RF, a left travel lever 142LT, and a right travel lever 142RT.

[0047] The left control lever 142LO is located to the left of the driver's seat 141. The right control lever 142RO is located to the right of the driver's seat 141.

[0048] The left operating lever 142LO is an operating mechanism for rotating the upper slewing body 120 and performing digging / unloading operations on the boom 132. Specifically, when the operator of the working machine 100 reverses the left operating lever 142LO forward, the boom 132 performs an unloading operation. Conversely, when the operator of the working machine 100 reverses the left operating lever 142LO backward, the boom 132 performs a digging operation. Furthermore, when the operator of the working machine 100 reverses the left operating lever 142LO to the right, the upper slewing body 120 rotates to the right. Additionally, when the operator of the working machine 100 reverses the left operating lever 142LO to the left, the upper slewing body 120 rotates to the left. It should be noted that in other embodiments, the upper slewing body 120 may rotate to the right or left when the left operating lever 142LO is reversed forward or backward, and the boom 132 may perform digging or unloading operations when the left operating lever 142LO is reversed to the left or right.

[0049] The right operating lever 142RO is an operating mechanism for performing digging / unloading actions of the bucket 133 and raising / lowering actions of the boom 131. Specifically, when the operator of the working machine 100 tilts the right operating lever 142RO forward, the boom 131 is lowered. Conversely, when the operator of the working machine 100 tilts the right operating lever 142RO backward, the boom 131 is raised. Conversely, when the operator of the working machine 100 tilts the right operating lever 142RO to the right, the bucket 133 unloads material. Conversely, when the operator of the working machine 100 tilts the right operating lever 142RO to the left, the bucket 133 digs. It should be noted that in other embodiments, the bucket 133 may unload or dig when the right operating lever 142RO is tilted forward or backward, and the boom 131 may raise or lower when the right operating lever 142RO is tilted left or right.

[0050] The left foot pedal 142LF is located on the left side of the floor surface in front of the driver's seat 141. The right foot pedal 142RF is located on the right side of the floor surface in front of the driver's seat 141. The left travel lever 142LT is pivotally supported on the left foot pedal 142LF, configured such that the tilting of the left travel lever 142LT is linked to the depressing of the left foot pedal 142LF. The right travel lever 142RT is pivotally supported on the right foot pedal 142RF, configured such that the tilting of the right travel lever 142RT is linked to the depressing of the right foot pedal 142RF.

[0051] The left foot pedal 142LF and the left travel lever 142LT correspond to the rotational drive of the left track 110a of the lower traveling body 110. Specifically, when the operator of the working machine 100 reverses the left foot pedal 142LF or the left travel lever 142LT forward, the left track 110a rotates in the forward direction. Conversely, when the operator of the working machine 100 reverses the left foot pedal 142LF or the left travel lever 142LT backward, the left track 110a rotates in the backward direction.

[0052] The right foot pedal 142RF and right travel lever 142RT correspond to the rotation drive of the right track 110b of the lower traveling body 110. Specifically, when the operator of the working machine 100 reverses the right foot pedal 142RF or the right travel lever 142RT forward, the right track 110b rotates in the forward direction. Conversely, when the operator of the working machine 100 reverses the right foot pedal 142RF or the right travel lever 142RT backward, the right track 110b rotates in the backward direction.

[0053] The display input device 145 is a device for displaying information related to the various functions of the machine tool 100, or for inputting various instruction operations. The display input device 145 includes a display 145D. The display 145D is, for example, composed of a touch panel.

[0054] (Example of the structure of the display control system 60)

[0055] Figure 4 An example of the structure of the display control system 60 according to the embodiment is shown. The display control system 60 includes a display control device 61, a plurality of cameras 121A to 121D, an operation device 142, a rotation angle sensor 160, and a display input device 145.

[0056] The display input device 145 includes a display unit 145A and a selection unit 145B, forming a functional structure composed of hardware constituting the display 145D and software such as programs for controlling the hardware. The display unit 145A displays the indicated image on the display 145D according to the instructions from the display control device 61. The selection unit 145B, for example, selects whether the crosshairs (described later) are displayed or not, based on the operator's input operation on the display 145D.

[0057] The display control device 61 can be constructed using hardware such as a microcomputer, a CPU (Central Processing Unit), and computer peripheral circuits and peripheral devices. Furthermore, the display control device 61 includes an input / output unit 62, a top-view image generation unit 63, an upper rotating body image overlay unit 64, a baseline overlay unit 65, a crosshair overlay unit 66, a travel direction image generation unit 67, and a display image output unit 68, all of which are functional structures composed of hardware and software such as programs executed by a computer.

[0058] It should be noted that the display control device 61 and the display input device 145 can also be constructed using a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device). Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions implemented by the processor can also be implemented using this integrated circuit.

[0059] Alternatively, for example, the display 145D and the operating device 142 can also be configured as a remote operating room located away from the machine 100.

[0060] The input / output unit 62 repeatedly performs the following operations at a predetermined cycle: inputting an image transmission signal representing images captured by multiple cameras 121A to 121D, or inputting operation information of the operation device 142, or inputting a signal representing the rotation angle measured by the rotation angle sensor 160, or inputting an input operation signal to the selection unit 145B of the display input device 145.

[0061] The overhead image generation unit 63 generates an overhead image based on images captured by multiple cameras 121A to 121D, displaying an image of the surroundings of the work machinery 100 as if viewed from above. Figure 5 The overhead view image G20 is generated by using conversion information stored in a designated storage unit to perform coordinate transformation on the image data. This transforms the images captured by multiple cameras 121A-121D into an image projected from an imaginary viewpoint above the machine 100 onto a designated imaginary projection plane. The overhead view image generation unit 63, after converting each image data captured by the four cameras 121A-121D into an overhead view image, synthesizes the converted image data to generate an overhead view image of the machine 100's surroundings. The overhead view image generation unit 63 synthesizes the surrounding images, for example, using the rotation center c as a reference. At this time, the overhead view image generation unit 63 generates the overhead view image with the front of the cab 140 always facing upwards. In this embodiment, the overhead view generation unit 63 generates an overhead view of the surroundings of the work machine 100 based on one or more images captured by one or more cameras (shooting devices) 121A to 121D mounted on the upper rotating body 120 of the work machine 100, which has a lower traveling body 110 and an upper rotating body 120 supported on the lower traveling body 110 in a rotatable manner.

[0062] The overlapping portion 64 of the upper rotating body image, for example, Figure 5 As shown, the overhead view image G20 generated by the overhead view image generation unit 63 is superimposed with the overhead view image IM1 of the pre-prepared upper rotating body 120 and the working device 130. Figure 5An example of the display 145D (display screen 1451) is shown. Display screen 1451 includes a display image G21 and a travel direction image G22. In display image G21, a top view image IM1, baseline images m1 and m2, a crosshair image L1, and a rotation center image c1 are superimposed on the top view image G20. The upper rotating body image superposition section 64 superimposes the top view image IM1 of the working machine 100 onto the center of the top view image G20 generated by the top view image generation section 63. Thus, viewing... Figure 5 The operator of the displayed image G21 can easily grasp the positional relationship and sense of distance between the surrounding obstacles and the working machinery 100 shown in the overhead image G20. It should be noted that the overhead image G20 includes actual captured images of tracks 110a and 110b.

[0063] The baseline overlap section 65 generates baseline images m1 and m2, representing the rotational focus area accompanying the rotation of the upper rotating body 120, and overlaps them with the overhead view G20. The baseline overlap section 65 overlaps baseline images m1 and m2 (rounded rectangles) with the overhead view G20 in a semi-transparent state. Baseline images m1 and m2 are, for example, different colors. Baseline image m1 is, for example, an image corresponding to the outer edge (outermost trajectory) of the upper rotating body 120 during rotation. Baseline image m1 represents the area (rotational focus area) where, when the upper rotating body 120 has rotated, the possibility of contact, interference, or collision is higher within the range represented by baseline image m1 compared to outside the range, and the operator should pay attention to and observe this area. It should be noted that in this embodiment, "focus" refers to attention, especially observation, and can be replaced with terms such as "attention," "careful attention," "vigilance," or "attentiveness." The baseline image m2 represents a range separated from the upper rotating body 120 by a certain distance (e.g., 2-3 m) from the baseline image m1. For example, the baseline image m1 represents the range that should be of concern during rotation, while the baseline image m2 represents the range that requires less attention compared to the range represented by the baseline image m1 but still requires attention during rotation. It should be noted that the ranges represented by the baseline image m1 and the ranges represented by the baseline image m2 may, for example, correspond to the stop control range and the deceleration control range outside the automatic control based on automatic detection of obstacles, etc., or the warning range and the attention range outside the warning range, or the ranges corresponding to the stop judgment range and the deceleration judgment range outside the deceleration range.

[0064] The crosshair overlap section 66 generates a crosshair image L1 and an image representing the rotation center c, namely the rotation center image c1, based on the rotation angle measured by the rotation angle sensor 160, and overlaps them with the overhead view G20. For example, the crosshair overlap section 66 overlaps the crosshair image L1 with the overhead view G20 in a semi-transparent state. The rotation center image c1 is an image with a predetermined shape representing the rotation center c, such as a semi-transparent or non-transparent circle of a specific color. The crosshair image L1 is an image (range of interest image) representing the range of interest for travel accompanying the forward and backward movement of the lower traveling body 110, and the range of interest for rotation accompanying the rotation of the upper rotating body 120. The area of ​​concern for movement accompanying the lower traveling body 110 is the area within which the operator should pay attention to and observe when obstacles or the like exist within the range (width and length) indicated by crosshair image L1, in the direction (forward or backward) of the lower traveling body 110 moving forward or backward (or when the upper rotating body 120 rotates while moving forward or backward). This range is considered more likely to result in contact with obstacles or the like compared to areas outside the range. Figure 5 In the example shown, the crosshair image L1 includes an arrow image L11 and a line segment image L12. Alternatively, the crosshair image L1 includes an arrow image L11, a line segment image L12, and a rotation center image c1. It should be noted that the crosshair overlap portion 66 overlaps or stops overlapping the crosshair images L1 according to the selection operation performed by the selection portion 145B.

[0065] When the operator of the work machinery 100 simultaneously reverses forward using the left foot pedal 142LF or left travel lever 142LT, and the right foot pedal 142RF or right travel lever 142RT, arrow image L11 indicates the direction of travel of the lower traveling body 110 by the direction of the arrow. Arrow image L11 also uses the length of the arrow to represent a reference for the length of the travel range of interest. Line segment image L12 shows a line segment at the rotation center c of the work machinery 100 that intersects with the arrow in arrow image L11 and shows a length corresponding to the rotation range of interest. Figure 5 In the example shown, the arrow image L11 and the line segment image L12 are orthogonal at the position of the rotation image c1. The arrow image L11 has a length that connects with the reference image m2, and the line segment image L12 has a length that connects with the reference image m1.

[0066] It should be noted that, Figure 6 Showing from Figure 5 Image G21a shows the state after the lower traveling body 110 has turned 45 degrees to the right. Additionally, Figure 7 Showing from Figure 5 The image G21b shows the state after the lower driving body 110 has turned 90 degrees to the right.

[0067] In this embodiment, the crosshair overlap portion 66 (overlap portion) generates a crosshair image L1 (range of interest image) that represents the rotation range of interest accompanying the rotation of the upper rotating body 120 and the driving range of interest accompanying the forward and backward movement of the lower traveling body 110, which is superimposed on the display image G21 of the overhead view image G20. In addition, the crosshair image L1 shows the orientation information (the direction of the arrow in the crosshair image L1) corresponding to the prescribed instruction operation for operating the operating device 142 of the lower traveling body 110 to move forward or backward (the operation of the operator of the work machine 100 simultaneously reversing the left foot pedal 142LF or the left travel lever 142LT and the right foot pedal 142RF or the right travel lever 142RT forward). Additionally, the crosshair image L1 includes an image (arrow image L11) showing an arrow that indicates the length corresponding to the driving range of interest and the orientation corresponding to the above-described instruction operation, and an image (line segment image L12) showing a line segment that intersects the arrow (the arrow shown in arrow image L11) at the rotation center c of the working machine 100 and indicates the length corresponding to the rotation range of interest.

[0068] For example, when the operating device 142 performs a specified instruction operation to indicate whether to move forward or backward, the travel direction image generation unit 67 generates a travel direction image corresponding to the travel direction of the lower traveling body 110 based on images captured by one or more cameras 121A to 121D. Figure 5 The travel direction image G22 shown is displayed when a forward instruction operation is performed on the operating device 142 in the working machine 100 in the state shown in the display image G21. In this case, the travel direction image G22 is, for example, an image captured by the front camera 121A. The travel direction image generation unit 67 can, for example, select one of the images captured by cameras 121A to 121D as the travel direction image when the travel direction is forward, backward, left, or right, or it can cut out an image corresponding to the travel direction from the composite image of the images captured by cameras 121A to 121D as the travel direction image. It should be noted that the travel direction image generation unit 67 can also, for example, generate a travel direction image corresponding to the travel direction of the lower traveling body 110 based on images captured by one or more cameras 121A to 121D when no predetermined forward or backward instruction operation is performed on the operating device 142.

[0069] The display image output unit 68 outputs the display image and the direction of travel image, along with the displayed indication, to the display unit 145A.

[0070] (Example of the operation of the display control device 61)

[0071] Reference Figure 8 and Figure 5 ,right Figure 4 The operation of the display control device 61 shown will be explained using an example. Figure 8 The process shown is repeated at a predetermined cycle. At the beginning... Figure 8 After the processing shown, firstly, the overhead view image generation unit 63 generates an overhead view image G20 (S1). Next, the upper rotating body image overlay unit 64 overlays the overhead view image IM1 onto the overhead view image G20 (S2). Next, the baseline overlay unit 65 overlays the baseline image m1 and the baseline image m2 onto the overhead view image G20 overlaid with the overhead view image IM1 (S3). Next, the crosshair overlay unit 66 overlays the crosshair image L1 and the rotation center image c1 (which is the display image G21) onto the overhead view image G20 overlaid with the overhead view image IM1, the baseline image m1, and the baseline image m2 (S4). Next, the travel direction image generation unit 67 generates a travel direction image G22 (S5). Next, the display image output unit 68 outputs the display image G21 and the travel direction image G22 to the display unit 145A, causing the display 145D to display the display image G21 and the travel direction image G22 (S6).

[0072] (Function / Effect)

[0073] According to this embodiment, the operator (driver) can easily grasp the scope of attention for driving (driving and turning) the accompanying work machinery 100.

[0074] (Modified Example)

[0075] Next, refer to Figure 9 and Figure 10 ,right Figure 5 The following describes variations of the displayed image G21 (display image G21c and display image G21d). Figure 9 The displayed image G21c is relative to Figure 5 The displayed image G21 differs in the following aspects. That is, Figure 9 The displayed image G21c does not include Figure 5 Reference images m1 and m2 are shown, while reference image m3 is included. Reference image m3 shows the region including reference image m1. Figure 9 The entirety of the shaded area Rm1 and the area of ​​the arrow image L11. In this case, the reference image m3 includes the rotational area of ​​interest accompanying the rotation of the upper rotating body 120 and the driving area of ​​interest accompanying the forward and backward movement of the lower traveling body 110. That is, in Figure 9 In the displayed image G21c, the crosshair image L1 is a separate image of interest, and the reference image m3 is also a separate image of interest.

[0076] on the other hand, Figure 10 The displayed image G21d is relative to Figure 5 The displayed image G21 differs in the following aspects. That is, Figure 10 The displayed image G21d does not include Figure 5 The reference images shown are m1, m2, and L11, and include m4. Reference image m4 shows a comparison with... Figure 9 The reference image m3 shows an area with the same shape. However, unlike reference image m3, reference image m4 indicates the orientation of the lower driving body 110 by changing the display mode of the upper half of reference image m4a and the lower half of reference image m4b. The changes in display mode include different colors, different flashing periods, and different thicknesses of dashed lines. In this case, reference image m4 is a separate image of the area of ​​interest and shows the orientation corresponding to the prescribed instruction operation of the operating device 142 for moving forward or backward on the lower driving body 110. In this case, the reference line overlap 65, like the crosshair overlap 66, corresponds to the "overlapping portion".

[0077] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the specific structure is not limited to the above embodiments, and also includes design changes that do not depart from the spirit of the present invention.

[0078] For example, the operating machinery 100 in the above-described embodiment is a hydraulic excavator, but it is not limited to this. For example, the operating machinery 100 in other embodiments may also be other operating machinery such as dump trucks, bulldozers, and wheel loaders.

[0079] In addition, in the above embodiments, part or all of the program executed by the computer can be published via a computer-readable recording medium or communication line.

[0080] Industrial applicability

[0081] According to the various methods disclosed herein, the driver can easily grasp the scope of attention required while driving the accompanying work machinery.

[0082] Explanation of reference numerals in the attached figures:

[0083] 100…Working machinery; 110…Lower traveling body; 120…Upper rotating body; 121A~121D…Camera; 61…Display control device; 62…Input / output unit; 63…Overhead image generation unit; 64…Overlapping image of upper rotating body; 65…Baseline overlap; 66…Cross line overlap; 67…Travel direction image generation unit; 68…Display image output unit; L1…Cross image; m1, m2, m3, m4…Base image; G20…Overhead image; G21…Display image; G22…Travel direction image.

Claims

1. A display control device, wherein, The display control device includes: The overhead image generation unit generates an overhead image of the surroundings of the operating machine based on one or more images captured by one or more imaging devices mounted on the upper rotating body of the operating machine, which has a lower traveling body and an upper rotating body supported on the lower traveling body in a rotatable manner. An overlapping portion is generated by overlaying an image representing the rotational range of interest accompanying the rotation of the upper rotating body and an image representing the driving range of interest accompanying the forward and backward movement of the lower driving body onto the overhead view. as well as The display image output unit outputs the display image. The image representing the driving focus area includes an arrow showing the length corresponding to the driving focus area and the direction corresponding to a prescribed indicative operation for moving forward or backward on the operating device for operating the lower driving body. The arrow image is an arrow-shaped image that is positioned in a positional relationship with the image representing the center of rotation of the working machinery, i.e., the center of rotation image, and extends along the front-rear direction of the lower traveling body. The image representing the driving range of interest includes an image showing a line segment that intersects the arrow image at the center of rotation of the working machinery and shows a length corresponding to the range of rotational interest.

2. The display control device according to claim 1, wherein, A first baseline image representing the area of ​​interest that should be focused on during rotation, and a second baseline image representing the area of ​​interest that should be focused on during rotation but with a lower degree of attention compared to the area represented by the first baseline image, are superimposed on the overhead image. The image representing the line segment has a length that connects to the first baseline image. The arrow image has a length that connects to the second baseline image.

3. The display control device according to claim 1 or 2, wherein, The display control device further includes a travel direction image generation unit, which generates a travel direction image corresponding to the travel direction of the lower driving body based on the one or more captured images. The display image output unit outputs the display image and the travel direction image.

4. A display control method, wherein, The display control method includes the following steps: An overhead view of the surroundings of the working machine is generated based on one or more images captured by one or more cameras mounted on the upper rotating body of the working machine, which has a lower traveling body and an upper rotating body supported on the lower traveling body in a rotatable manner. A display image is generated by overlaying an image representing the rotational range of interest accompanying the rotation of the upper rotating body and an image representing the driving range of interest accompanying the forward and backward movement of the lower driving body onto the overhead view. Output the displayed image. The image representing the driving focus area includes an arrow showing the length corresponding to the driving focus area and the direction corresponding to a prescribed indicative operation for moving forward or backward on the operating device for operating the lower driving body. The arrow image is an arrow-shaped image that is positioned in a positional relationship with the image representing the center of rotation of the working machinery, i.e., the center of rotation image, and extends along the front-rear direction of the lower traveling body. The image representing the driving range of interest includes an image showing a line segment that intersects the arrow image at the center of rotation of the working machinery and shows a length corresponding to the range of rotational interest.

5. A type of operating machinery, wherein, The operating machinery includes: Lower driving body; The upper rotating body is supported on the lower traveling body in a rotatable manner; One or more imaging devices are disposed on the upper rotating body; as well as Display control device, The display control device has: An overhead image generation unit generates an overhead image of the surroundings of the upper rotating body and the lower traveling body based on one or more images captured by the one or more imaging devices; and An overlapping portion is generated by overlaying an image representing the rotational range of interest accompanying the rotation of the upper rotating body and an image representing the driving range of interest accompanying the forward and backward movement of the lower driving body onto the overhead view. as well as The display image output unit outputs the display image. The image representing the driving focus area includes an arrow showing the length corresponding to the driving focus area and the direction corresponding to a prescribed indicative operation for moving forward or backward on the operating device for operating the lower driving body. The arrow image is an arrow-shaped image that is positioned in a positional relationship with the image representing the center of rotation of the working machinery, i.e., the center of rotation image, and extends along the front-rear direction of the lower traveling body. The image representing the driving range of interest includes an image showing a line segment that intersects the arrow image at the center of rotation of the working machinery and shows a length corresponding to the range of rotational interest.

Images