Display system of work machine and display method of work machine

Abstract

The image acquisition unit acquires a captured image of the surroundings of the work machine. The display control unit, while the display unit is displaying a surrounding image based on the captured image, generates a display signal for displaying a display image containing the surrounding image on a migrated screen of the display unit, in the case where a migration signal for migrating the screen is accepted.

Description

Technical Field

[0001] The present disclosure relates to a display system for a work machine and a display method for a work machine.

[0002] This application claims priority from Japanese Patent Application No. 2019-167454 filed in Japan on September 13, 2019, the contents of which are incorporated herein by reference. Background Art

[0003] Patent Document 1 discloses a technique for allowing an operator to visually recognize the surrounding conditions of a work machine during calibration of the work machine.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2017 / 191853 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] According to Patent Document 1, by displaying display data indicating the peripheral conditions of the working machine on a display device, the operator can visually recognize the peripheral conditions of the working machine during correction work. However, the display device of the working machine can be switched to display a screen such as a menu screen for displaying information related to the functions of the working machine. However, in the switched screen, information related to the functions is displayed on the entire display device, and display data indicating the peripheral conditions of the working machine is not displayed. When at least a part of the working machine is in an operable state, the operator may want to operate the working machine while the display device is in a state where information related to any function is displayed.

[0009] An object of the present disclosure is to provide a display system for a work machine and a display method for a work machine that allow an operator to visually recognize the surrounding conditions of the work machine.

[0010] Means for solving problems

[0011] According to a first embodiment of the present invention, a display system of a working machine comprises: an image acquisition unit which acquires a captured image of the surroundings of the working machine; and a display control unit which, when a surrounding image based on the captured image is being displayed on the display unit, generates a display signal for displaying a display image including the surrounding image on the post-migration screen of the display unit upon receiving a migration signal for migrating the screen.

[0012] Effects of the Invention

[0013] According to at least one of the above aspects, the operator can visually recognize the surrounding conditions of the working machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram showing the structure of the working machine according to the first embodiment.

[0015] Figure 2 It is a diagram showing the imaging ranges of a plurality of cameras included in the working machine according to the first embodiment.

[0016] Figure 3 It is a diagram showing the internal structure of the cab according to the first embodiment.

[0017] Figure 4 This is a schematic block diagram showing the configuration of a computer according to at least one embodiment.

[0018] Figure 5 This is a flowchart showing a display control process of the control device according to the first embodiment.

[0019] Figure 6 This is a screen transition diagram showing the transition of screens displayed on the display according to the first embodiment.

[0020] Figure 7 This is a flowchart showing a display control process of the control device according to the second embodiment.

[0021] Figure 8 It is a screen transition diagram showing the transition of screens displayed on the display according to the second embodiment.

[0022] Figure 9 This is a screen transition diagram showing the transition of screens displayed on a display according to another embodiment. DETAILED DESCRIPTION

[0023] <First embodiment>

[0024] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[0025] Structure of operating machinery

[0026] Figure 1 It is a schematic diagram showing the structure of the working machine according to the first embodiment.

[0027] The work machine 100 operates at a construction site, performing construction work on a construction object such as earth and sand. The work machine 100 according to the first embodiment is, for example, a hydraulic excavator. The work machine 100 includes a traveling body 110 , a revolving body 120 , a working machine 130 , and a cab 140 .

[0028] The traveling body 110 supports the working machine 100 so that the working machine 100 can travel. The traveling body 110 is, for example, a pair of left and right endless tracks.

[0029] The rotating body 120 is rotatably supported by the traveling body 110 around a rotation center.

[0030] The work machine 130 is hydraulically driven and supported vertically in front of the revolving structure 120. The operator's cab 140 is a space where an operator can operate the work machine 100. The operator's cab 140 is located on the left front of the revolving structure 120.

[0031] Here, the portion of the rotating body 120 to which the working machine 130 is mounted is referred to as the front portion. Furthermore, with respect to the rotating body 120, the portion opposite the front portion is referred to as the rear portion, the left portion is referred to as the left portion, and the right portion is referred to as the right portion.

[0032] The Structure of a Rotating Body

[0033] The revolving body 120 is provided with a plurality of cameras 121 for capturing images of the surroundings of the working machine 100 . Figure 2 It is a diagram showing the imaging ranges of a plurality of cameras included in the working machine according to the first embodiment.

[0034] Specifically, the rotating body 120 is provided with a left rear camera 121A for capturing images of a left rear range Ra around the rotating body 120, a rear camera 121B for capturing images of a rear range Rb around the rotating body 120, a right rear camera 121C for capturing images of a right rear range Rc around the rotating body 120, and a right front camera 121D for capturing images of a right front range Rd around the rotating body 120. Furthermore, the capturing ranges of the plurality of cameras 121 may partially overlap.

[0035] The imaging range of the plurality of cameras 121 covers the entire periphery of the working machine 100 except for the left front range Re that can be visually identified from the cab 140. In addition, the cameras 121 involved in the first embodiment capture the left rear, rear, right rear, and right front of the rotating body 120, but are not limited to this in other embodiments. For example, the number and imaging range of the cameras 121 involved in other embodiments may also be different. Figure 1 as well as Figure 2 The examples shown are different.

[0036] In addition, the left rear camera 121A is as shown in FIG. Figure 2As shown in the left rear range Ra of the rotating body 120, the left side area and the left rear area of ​​the rotating body 120 are photographed, but either area can be photographed. Similarly, the right rear camera 121C is as shown in the left rear range Ra of the rotating body 120. Figure 2 As shown in the right rear range Rc of the right side, the right side area and the right rear area of ​​the rotating body 120 are photographed, but either area can be photographed. Similarly, the right front camera 121D is as shown in the right rear range Rc of the right side. Figure 2 As shown in the right front range Rd, the right front area and the right side area of ​​the rotating body 120 are imaged, but either area may be imaged.

[0037] Structure of the operating machine

[0038] The work machine 130 includes a boom 131 , an arm 132 , a bucket 133 , a boom cylinder 134 , an arm cylinder 135 , and a bucket cylinder 136 .

[0039] The base end portion of the boom 131 is attached to the rotating body 120 via a boom pin P1.

[0040] The arm 132 is connected to the boom 131 and the bucket 133. The base end portion of the arm 132 is attached to the front end portion of the boom 131 via an arm pin P2.

[0041] The bucket 133 has a blade for digging earth and sand, etc. and a container for storing the excavated earth and sand. The base end of the bucket 133 is attached to the front end of the arm 132 via a bucket pin P3.

[0042] The boom cylinder 134 is a hydraulic cylinder for operating the boom 131. The base end portion of the boom cylinder 134 is attached to the rotating body 120. The front end portion of the boom cylinder 134 is attached to the boom 131.

[0043] The arm cylinder 135 is a hydraulic cylinder for driving the arm 132. The base end of the arm cylinder 135 is attached to the boom 131. The front end of the arm cylinder 135 is attached to the arm 132.

[0044] The bucket cylinder 136 is a hydraulic cylinder for driving the bucket 133. The base end of the bucket cylinder 136 is attached to the arm 132. The front end of the bucket cylinder 136 is attached to a connecting member connected to the bucket 133.

[0045] The structure of the cab

[0046] Figure 3 It is a diagram showing the internal structure of the cab according to the first embodiment.

[0047] In the cab 140, a driver's seat 141, an engine key cylinder 1411 (main switch), a sub-switch 1412, an operation device 142, and a control device 143 are provided.

[0048] The engine key cylinder 1411 receives insertion of an engine key, and by rotating the inserted engine key in a first direction, an engine not shown is driven. Further, in the driving of the engine, the engine key inserted in the engine key cylinder 1411 is rotated in a second direction opposite to the first direction, and the engine is stopped.

[0049] The sub-switch 1412 is provided at a lower portion of the driver's seat 141. The sub-switch 1412 is a switch for stopping the engine in an emergency. In the driving of the engine, if the sub-switch 1412 is pressed, the engine is stopped.

[0050] The operation device 142 is an interface for driving the traveling body 110, the rotating body 120, and the working machine 130 by manual operation of an operator. The operation device 142 has a left operation lever 1421, a right operation lever 1422, a left pedal 1423, a right pedal 1424, a left traveling lever 1425, and a right traveling lever 1426.

[0051] The left operation lever 1421 is provided at a left side of the driver's seat 141. The right operation lever 1422 is provided at a right side of the driver's seat 141.

[0052] The left operation lever 1421 is an operation mechanism for performing a rotating operation of the rotating body 120 and a pulling / pushing operation of the arm 132. Specifically, if the operator of the working machine 100 tilts the left operation lever 1421 toward the front, the arm 132 performs a pushing operation. Further, if the operator of the working machine 100 tilts the left operation lever 1421 toward the rear, the arm 132 performs a pulling operation. Further, if the operator of the working machine 100 tilts the left operation lever 1421 toward the right direction, the rotating body 120 performs right rotation. Further, if the operator of the working machine 100 tilts the left operation lever 1421 toward the left direction, the rotating body 120 performs left rotation. In addition, in other embodiments, the rotating body 120 can perform right rotation or left rotation when the left operation lever 1421 is tilted toward the front-rear direction, and the arm 132 can perform a pulling operation or a pushing operation when the left operation lever 1421 is tilted toward the left-right direction.

[0053] The right operation lever 1422 is an operation mechanism for performing the digging / pouring action of the bucket 133 and the raising / lowering action of the large arm 131. Specifically, if the operator of the working machine 100 tilts the right operation lever 1422 toward the front, the lowering action of the large arm 131 is performed. Further, if the operator of the working machine 100 tilts the right operation lever 1422 toward the rear, the raising action of the large arm 131 is performed. Further, if the operator of the working machine 100 tilts the right operation lever 1422 toward the right direction, the pouring action of the bucket 133 is performed. Further, if the operator of the working machine 100 tilts the right operation lever 1422 toward the left direction, the digging action of the bucket 133 is performed. In addition, in other embodiments, the bucket 133 can perform the pouring action or the digging action when the right operation lever 1422 is tilted toward the front-rear direction, and the large arm 131 can perform the raising action or the lowering action when the right operation lever 1422 is tilted toward the left-right direction.

[0054] The left pedal 1423 is disposed on the left side of the ground in front of the driver's seat 141. The right pedal 1424 is disposed on the right side of the ground in front of the driver's seat 141. The left travel lever 1425 is configured to be supported by the left pedal 1423, and the inclination of the left travel lever 1425 is linked to the depression of the left pedal 1423. The right travel lever 1426 is configured to be supported by the right pedal 1424, and the inclination of the right travel lever 1426 is linked to the depression of the right pedal 1424.

[0055] The left pedal 1423 and the left travel lever 1425 correspond to the rotational drive of the left side track of the traveling body 110. Specifically, if the operator of the working machine 100 tilts the left pedal 1423 or the left travel lever 1425 toward the front, the left side track rotates in the forward direction. Further, if the operator of the working machine 100 tilts the left pedal 1423 or the left travel lever 1425 toward the rear, the left side track rotates in the backward direction.

[0056] The right pedal 1424 and the right travel lever 1426 correspond to the rotational drive of the right side track of the traveling body 110. Specifically, if the operator of the working machine 100 tilts the right pedal 1424 or the right travel lever 1426 toward the front, the right side track rotates in the forward direction. Further, if the operator of the working machine 100 tilts the right pedal 1424 or the right travel lever 1426 toward the rear, the right side track rotates in the backward direction.

[0057] The control device 143 is an input / output device having a display 1431 for displaying information related to a plurality of functions of the working machine 100. The control device 143 is an example of a display system. In addition, the display 1431 is an example of a display unit. The input unit of the control device 143 involved in the first embodiment is a hard key. In addition, in other embodiments, a touch panel, a mouse, or a keyboard may also be used as an input unit. In addition, the control device 143 involved in the first embodiment is provided integrally with the display 1431, but in other embodiments, the display 1431 may also be provided separately from the control device 143.

[0058] Structure of the control device

[0059] Figure 4 This is a schematic block diagram showing the configuration of a computer according to at least one embodiment.

[0060] The control device 143 is a computer including a processor 210 , a main memory 230 , a storage 250 , and an interface 270 .

[0061] The display 1431 is connected to the processor 210 via the interface 270 .

[0062] The storage 250 is a non-transitory tangible storage medium. Examples of the storage 250 include a HDD (Hard Disk Drive), an SSD (Solid State Drive), a magnetic disk, a magneto-optical disk, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), and a semiconductor memory. The storage 250 may be an internal medium directly connected to the bus of the control device 143, or an external medium connected to the control device 143 via an interface 270 or a communication line. The storage 250 stores a program for realizing surrounding monitoring of the working machine 100. In addition, the storage 250 pre-stores a plurality of images including icons, user menu images, service menu images, and function images for display on the display.

[0063] The program may also be a program for realizing a portion of the functions performed by the control device 143. For example, the program may also realize the functions by combining with other programs already stored in the storage 250, or by combining with other programs installed on other devices. In addition, in other embodiments, the control device 143 may have a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above structure. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, part or all of the functions realized by the processor may also be realized by the integrated circuit.

[0064] By executing programs, the processor 210 functions as an image acquisition unit 211, an overhead image generation unit 212, a display image generation unit 213, an input unit 214, a selection unit 215, a display control unit 216, a function execution unit 217, and a vehicle state monitoring unit 218. Furthermore, by executing programs, the processor 210 can also read a plurality of images stored in the storage 250 into the main memory 230.

[0065] The image acquisition unit 211 acquires captured images from the plurality of cameras 121 .

[0066] Bird's-eye image generation section 212 deforms and synthesizes the multiple captured images acquired by image acquisition section 211 to generate a bird's-eye view of the scene from above, centered around work machine 100. Bird's-eye image generation section 212 cuts out portions of each deformed captured image and synthesizes the resulting images to generate the bird's-eye view. An image of work machine 100, viewed from above, is pre-pasted at the center of the bird's-eye view image generated by bird's-eye image generation section 212. The bird's-eye view image is an example of a surrounding image capturing the area surrounding work machine 100.

[0067] The display image generation unit 213 generates a display image representing information for implementing each of the multiple functions of the work machine 100. Examples of the functions of the work machine 100 include a function for displaying a user menu, a function for displaying vehicle information such as the remaining fuel level and coolant temperature, and a function for adjusting the power mode of the work machine 100.

[0068] The display image associated with each function generated by the display image generating unit 213 includes the bird's-eye view image generated by the bird's-eye view image generating unit 212 and a functional image for realizing the function.

[0069] The input unit 214 receives an input signal generated by a user pressing a hard key.

[0070] The selection unit 215 determines the function selected by the user from among the multiple functions of the work machine 100 based on the input signal received by the input unit 214. Specifically, the selection unit 215 receives a selection of any one of the multiple functions of the work machine 100. Furthermore, the selection unit 215 is an example of a transition receiving unit that receives a transition signal for transitioning the screen displayed on the display 1431 of the work machine 100.

[0071] The display control unit 216 generates a display signal for displaying the display image related to the function determined by the selection unit 215 among the display images generated by the display image generation unit 213 , and outputs the signal to the display 1431 .

[0072] The function execution unit 217 executes the function determined by the selection unit 215 .

[0073] The vehicle state monitoring unit 218 monitors the vehicle state including the operating state of the engine. For example, when the engine is stopped by pressing the auxiliary switch 1412, the vehicle state monitoring unit 218 detects the stop of the engine.

[0074] Operation of the control device

[0075] Figure 5 This is a flowchart showing a display control process of the control device according to the first embodiment. Figure 6 This is a screen transition diagram showing the transition of screens displayed on the display according to the first embodiment.

[0076] If the engine key is inserted into the engine key cylinder 1411 and rotated in the first direction, the engine is driven and the control device 143 is started. If the control device 143 is started, the display 1431 is displayed as follows. Figure 6As shown, display image D0 indicates that the work machine 100 is currently operating. While display image D0 is being displayed, the control device 143 has not yet completed activation, and therefore cannot generate an overhead image. Therefore, display image D0 does not include an overhead image. Furthermore, until activation of the control device 143 is complete, the work machine 100 is in an inactive state.

[0077] If the activation of the control device 143 is completed, the execution Figure 5 The display control process shown. First, the image acquisition unit 211 acquires captured images from multiple cameras 121 (step S1). Then, the bird's-eye view image generation unit 212 generates a bird's-eye view image by deforming and synthesizing the multiple captured images acquired in step S1 (step S2). Then, the display image generation unit 213 generates a default display image D1 (step S3) that includes one of the captured images acquired in step S1 (single camera image G12) and the bird's-eye view image G11 generated in step S2. For example, Figure 6 In the display image D1, the image captured by the right rear camera 121C is arranged as the single camera image G12.

[0078] like Figure 6 As shown, control device 143 has six function keys F1-F6 as hard keys directly below display 1431. An icon G13 is located in the lower portion of display image D1, corresponding to function key F6. Icon G13 represents the user menu display function. Specifically, icon G13 indicates that the user menu display function is executed by pressing function key F6. Icon G13 is pre-stored in storage 250 or main memory 230.

[0079] The display control unit 216 outputs a display signal for displaying the display image D1 generated in step S3 to the display 1431 (step S4). Thus, when the control device 143 is activated and the working machine 100 is in operation, a display image including a bird's-eye view image is displayed on the display 1431.

[0080] Next, if input unit 214 detects whether the user has pressed a function key, selection unit 215 determines whether the pressed function key is function key F6, which indicates the user menu display function (step S5). If function key F6 has not been pressed (step S5: NO), control device 143 returns the process to step S1 and continues generating and displaying display image D1. On the other hand, if function key F6 has been pressed (step S5: YES), selection unit 215 accepts selection of the user menu display function (step S6). In other words, selection unit 215 accepts the signal generated by pressing function key F6 as a signal to transition to the user menu display screen.

[0081] Next, the display image generation unit 213 generates a display image D2 (display image D2A, display image D2B) that includes the bird's-eye view image G21 generated in step S3 and a pre-stored user menu image G22 (step S7). The user menu image G22 is a functional image for realizing the display function of the user menu. The user menu image G22 includes a list of multiple functions of the working machine 100, with one of the multiple items in the list in focus. The item in focus is set to a different color from the other items. The user menu image G22 displays multiple tabs indicating the type of list. In other words, as shown in display images D2A and D2B, different lists are arranged in the user menu image G22 depending on the tab with focus. In addition, icons G23-G28 are arranged in the portions corresponding to the function keys F1-F6 at the bottom of the display image D2. Icons G23-G28 are pre-stored in the storage 250 or the main memory 230.

[0082] The icon G23 is an icon indicating a leftward movement. That is, the icon G23 indicates that the focus of the tab of the user menu has moved to the left by pressing the function key F1.

[0083] The icon G24 is an icon indicating right movement. That is, the icon G24 indicates that the focus of the tab of the user menu has moved to the right by pressing the function key F2.

[0084] The icon G25 is an icon indicating downward movement. That is, the icon G25 indicates that the focus of the user menu has moved downward by pressing the function key F3.

[0085] The icon G26 is an icon indicating upward movement. That is, the icon G26 indicates that the focus of the user menu has moved upward by pressing the function key F4.

[0086] The icon G27 is an icon indicating a transition to the previous screen. That is, the icon G27 indicates that the user menu display function is canceled by pressing the function key F5 and the previous screen is displayed.

[0087] The icon G28 is an icon indicating a decision. Specifically, the icon G28 indicates that the function with focus is selected in the user menu by pressing the function key F6, and a display screen related to the function is displayed.

[0088] The display control unit 216 outputs a display signal for displaying the display image D2 generated in step S7 to the display 1431 (step S8). The display image D2 including the bird's-eye view image is displayed on the display 1431. At this time, the working machine 100 is in operation.

[0089] Next, if the input unit 214 detects whether the user has pressed a function key, the selection unit 215 determines whether the pressed function key is the function key F1 or F2 for left and right movement, or the function key F3 or F4 for up and down movement, or the function key F5 for migration to the previous screen, or the function key F6 for decision (step S9).

[0090] When the function key F3 or F4 indicating up / down movement is pressed (step S9: up / down movement), the display control unit 216 changes the focused item based on the pressed function key, returns the process to step S7, and updates the display image D2.

[0091] When the function key F1 or F2 indicating left / right movement is pressed (step S9: left / right movement), the display image generation unit 213 obtains the user menu image G22 after the label switching from the storage 250 or the main memory 230 based on the pressed function key (step S10). Then, the control device 143 returns the process to step S7 and updates the display image D2. For example, when the Figure 6 When the function key F2 is pressed while the display image D2A is shown, the display image generation unit 213 generates the display image D2B.

[0092] When the function key F5 is pressed (step S9 : previous screen), the control device 143 returns the process to step S1 and displays the display image D1 on the display 1431 .

[0093] When function key F6 is pressed (step S9: decision), selection section 215 receives selection of the function associated with the focused item among the plurality of functions (step S11). In other words, selection section 215 receives the signal generated by pressing function key F6 as a transition signal to the function image display screen.

[0094] Next, the function execution unit 217 retrieves a function image G32 for implementing the function selected in step S11 from the storage 250 or main memory 230 (step S12). For example, if the selected function is to adjust the power mode of the work machine 100, the function execution unit 217 retrieves a function image for adjusting the power mode. Alternatively, if the selected function is to set an attachment for the work machine 100, the function execution unit 217 retrieves a function image for setting the flow rate of oil used in the attachments included in the work machine 100. This allows the power of the attachments to be changed.

[0095] Next, the display image generation unit 213 generates a display image D3 including the bird's-eye view image G31 generated in step S2 and the functional image G32 acquired in step S12 (step S13 ). Figure 6In the example shown, icons G33-G36 are respectively arranged at the portion corresponding to the function keys F1-F6 in the lower portion of the display image D3. The icons G33-G36 are stored in the storage 250 or the main memory 230 in advance.

[0096] Icon G33 indicates a downward movement. Icon G34 indicates an upward movement. Icon G35 indicates a transition to the previous screen. Icon G36 indicates a decision. Furthermore, the type and number of icons arranged in display image D3 may vary depending on the selected function. Meanwhile, at least in display image D3, an icon indicating a transition to the previous screen is arranged.

[0097] The display control unit 216 outputs a display signal for displaying the display image D3 generated in step S14 to the display 1431 (step S14). The display image D3 including the bird's-eye view image is displayed on the display 1431. At this time, the working machine 100 is in operation.

[0098] Next, when the input unit 214 detects whether the user has pressed a function key, the selection unit 215 determines whether it is the function key F5 indicating transition to the previous screen or any of the other function keys F1 to F4 and F6 (step S15 ).

[0099] When any one of the function keys F1-F4 and F6 is pressed (step S15: Other), the function execution unit 217 executes the function selected in step S11 based on the pressed function key (step S16), and ends the process. Figure 5 The display control processing shown causes the display image D1 to be displayed on the display 1431 .

[0100] On the other hand, when function key F5 is pressed (step S15: previous screen), the process returns to step S7, and display image D2, which is the previous screen, is displayed on display 1431. That is, if the previous screen is display image D2A, display image D2A is displayed again, and if the previous screen is display image D2B, display image D2B is displayed again.

[0101] in addition, Figure 5 The flowchart shown is an example, and in other embodiments, not all steps may be performed. For example, in other embodiments, if a function can be selected without displaying image D2 using a shortcut button, etc., the control device 143 may not perform steps S6-10. Furthermore, if the display of display image D3 itself constitutes the execution of a function, the control device 143 may not perform step S16.

[0102] Furthermore, if the vehicle state monitoring unit 218 of the control device 143 detects that the engine has stopped due to the pressing of the auxiliary switch 1412, the display control unit 216 outputs a display signal to the display 1431 via an interrupt process, causing the display to display a stop display screen prompting the user to turn off the engine key. This display signal does not include an overhead image. Furthermore, when the engine has stopped due to the pressing of the auxiliary switch 1412, the work machine 100 is in an inoperative state.

[0103] Function and Effect

[0104] As described above, when the control device 143 according to the first embodiment receives a transition signal while the bird's-eye view image is being displayed on the display 1431, the control device 143 also displays the display image including the bird's-eye view image on the screen after the transition. This allows the control device 143 to maintain the bird's-eye view image on the display 1431 even when the screen is transitioned by an operator's operation. This allows the control device 143 to visually recognize the surrounding conditions of the work machine.

[0105] Furthermore, the control device 143 according to the first embodiment displays an image on the display 1431 according to a display signal during the period from when the working machine 100 is in the working state to when it is in the non-working state, as shown in FIG. Figure 6 As shown, the surrounding image is included. In this way, the bird's-eye view image can be constantly displayed on the display 1431 while the operator can operate at least a portion of the working machine 100.

[0106] Furthermore, when the work machine 100 is placed in an inoperative state, such as by pressing the auxiliary switch 1412, the control device 143 according to the first embodiment generates a display signal for displaying an image that does not include a bird's-eye view image. This allows the control device 143 to inform the operator whether the work machine 100 is operable based on the presence or absence of the bird's-eye view image. In other words, the operator can intuitively determine whether the work machine 100 is operable based on the presence or absence of the bird's-eye view image.

[0107] In the first embodiment, the plurality of functions of the working machine 100 include a user menu display function, and the display image D2 includes a bird's-eye view image. Thus, the control device 143 can display the bird's-eye view image on the display 1431 even when the display image D2 is displayed.

[0108] Furthermore, in the first embodiment, the bird's-eye view images G11, G21, and G31 are all displayed at the same position. Therefore, the display position of the bird's-eye view images remains constant regardless of screen transitions, allowing the operator to easily check the surroundings of the work machine 100 before and after screen transitions.

[0109] Furthermore, in the first embodiment, the display images including the user menu image G22 and the function image G32 do not display the engine water temperature, fuel level, or service meter that displays the engine operating time displayed on the display image D1. By displaying only necessary information to the operator after the screen transition, it is possible to minimize display complexity.

[0110] <Second embodiment>

[0111] The control device 143 according to the first embodiment includes a bird's-eye view image in the display image of the working machine 100 during operation. In contrast, the control device 143 according to the second embodiment includes a single-camera image in the display image of the working machine 100 during operation. The single-camera image is an example of a surrounding image capturing the surroundings of the working machine 100.

[0112] The control device 143 according to the second embodiment may not include the bird's-eye view image generating unit 212 .

[0113] Figure 7 This is a flowchart showing a display control process of the control device according to the second embodiment. Figure 8 It is a screen transition diagram showing the transition of screens displayed on the display according to the second embodiment.

[0114] If the engine key is inserted into the engine key cylinder 1411 and rotated in the first direction, the engine is driven and the control device 143 is started. If the control device 143 is started, the display 1431 is displayed as follows. Figure 6 As shown, a display image D0 indicating that the working machine 100 is activated is displayed.

[0115] If the activation of the control device 143 is completed, the execution Figure 7 The display control process shown. First, the image acquisition unit 211 acquires captured images from the plurality of cameras 121 (step S101). Next, the display image generation unit 213 generates a default display image D5 containing two (single camera images G51 and G52) of the captured images acquired in step S101 (step S102). For example, Figure 8 In display image D5, the image captured by right front camera 121D is displayed as single camera image G51, and the image captured by right rear camera 121C is displayed as single camera image G52. In the lower portion of display image D5, in the area corresponding to function key F6, icon G53 is displayed. Icon G53 represents the user menu display function. Icon G53 is pre-stored in storage 250 or main memory 230.

[0116] The display control unit 216 outputs a display signal for displaying the display image D5 generated in step S102 to the display 1431 (step S103). Thus, when the control device 143 is activated and the working machine 100 is in operation, the display image including the single camera image is displayed on the display 1431.

[0117] Next, if input unit 214 detects whether the user has pressed a function key, selection unit 215 determines whether the pressed function key is function key F6, which indicates the user menu display function (step S104). If function key F6 has not been pressed (step S104: NO), control device 143 returns the process to step S101 and continues generating and displaying display image D5. On the other hand, if function key F6 has been pressed (step S104: YES), selection unit 215 accepts selection of the user menu display function (step S105). In other words, selection unit 215 accepts the signal generated by pressing function key F6 as a signal to transition to the user menu display screen.

[0118] Next, the display image generation unit 213 generates a display image D6 (display image D6A, display image D6B) containing the single camera images G61 and G62 acquired in step S101 and the pre-stored user menu image G63 (step S106). Icons G64-G69 are displayed at the bottom of the display image D6, corresponding to the function keys F1-F6. Icons G64-G69 are pre-stored in the storage 250 or main memory 230.

[0119] The display control unit 216 outputs a display signal for displaying the display image D6 generated in step S106 to the display 1431 (step S107). The display image D6 including the single camera image is displayed on the display 1431. At this time, the working machine 100 is in operation.

[0120] Next, if the input unit 214 detects whether the user has pressed a function key, the selection unit 215 determines whether the pressed function key is the function key F1 or F2 for left and right movement, or the function key F3 or F4 for up and down movement, or the function key F5 for migration to the previous screen, or the function key F6 for decision (step S108).

[0121] When the function key F3 or F4 indicating up / down movement is pressed (step S108: up / down movement), the display control unit 216 changes the focused item based on the pressed function key, returns the process to step S106, and updates the display image D6.

[0122] When the function key F1 or F2 indicating left / right movement is pressed (step S108: left / right movement), the display image generation unit 213 obtains the user menu image G63 after the label switching from the storage 250 or the main memory 230 based on the pressed function key (step S109). Then, the control device 143 returns the process to step S106 and updates the display image D6. For example, when the Figure 8 When the function key F2 is pressed while the display image D6A is shown, the display image generation unit 213 generates the display image D6B.

[0123] When the function key F5 is pressed (step S108 : previous screen), the control device 143 returns the process to step S101 , and displays the display image D5 on the display 1431 .

[0124] When function key F6 is pressed (step S108: decision), selection section 215 receives selection of the function associated with the focused item among the plurality of functions (step S110). In other words, selection section 215 receives the signal generated by pressing function key F6 as a transition signal to the function image display screen.

[0125] Next, the function execution unit 217 acquires the function image G72 for realizing the function selected in step S110 from the storage 250 or the main memory 230 (step S111). Next, the display image generation unit 213 generates a display image D7 including the single camera images G71 and G72 acquired in step S101 and the function image G73 acquired in step S111 (step S112). Figure 6 In the example shown, icons G74 to G77 are respectively arranged at portions corresponding to the function keys F3 to F6 in the lower portion of the display image D7.

[0126] The display control unit 216 outputs a display signal for displaying the display image D7 generated in step S112 to the display 1431 (step S113). Next, when the input unit 214 detects whether the user has pressed a function key, the selection unit 215 determines whether the key is the function key F5 indicating a transition to the previous screen or the other function keys F3, F4, and F6 (step S114).

[0127] When any one of the function keys F3, F4, and F6 is pressed (step S114: Other), the function execution unit 217 executes the function selected in step S110 based on the pressed function key (step S115), and ends the process. Figure 7 The display control processing shown causes the display image D5 to be displayed on the display 1431 .

[0128] On the other hand, if function key F5 is pressed (step S114: previous screen), the process returns to step S106, and display image D6, which is the previous screen, is displayed on display 1431. That is, if the previous screen is display image D6A, display image D6A is displayed again, and if the previous screen is display image D6B, display image D6B is displayed again.

[0129] in addition, Figure 7 The flowchart shown is an example, and in other embodiments, not all steps may be executed. For example, in other embodiments, if a function can be selected without displaying image D6 using a shortcut button, etc., the control device 143 may not execute steps S105-109. Furthermore, if the display of display image D7 itself constitutes the execution of a function, the control device 143 may not execute step S115.

[0130] <Other Implementation Methods>

[0131] While one embodiment has been described in detail with reference to the accompanying drawings, the specific structure is not limited to the above, and various design changes can be made. That is, in other embodiments, the order of the above-mentioned processes can be appropriately changed. In addition, some processes can also be executed in parallel.

[0132] In the above embodiment, the working machine 100 is described as a hydraulic excavator, but in other embodiments, it can be applied to various working machines such as dump trucks and loaders. For example, it can be applied to display systems for dump trucks, loaders, and other working machines.

[0133] Furthermore, while the above-described embodiment describes displaying a user menu image in place of the single camera image G12, other embodiments may also display the user menu image along with the bird's-eye view image and the single camera image. Specifically, the bird's-eye view image generation unit 212 generates a reduced bird's-eye view image to display the user menu and the single camera image together. Furthermore, the display image generation unit 213 generates a reduced single camera image to display the user menu and the bird's-eye view image together. Specifically, upon receiving a transition signal, the display control unit 216 generates a display signal to display a display image including the surrounding image with a changed display size on the post-transition screen.

[0134] Furthermore, in the above embodiment, a single control device 143 is described as being installed on the work machine 100. However, in other embodiments, a portion of the components of the control device 143 may be installed on other control devices, thereby implementing a display system that includes two or more control devices. Furthermore, the single control device 143 shown in the above embodiment is also an example of a display system.

[0135] Further, the control device 143 according to the above-described embodiment is provided on the work machine 100, but in other embodiments, part or all of the control device 143 can be provided outside the work machine 100. For example, in other embodiments, the control device 143 can also control the work machine 100 involved in remote operation. In this case, the control device 143 performs the above-described display of the screen on a display device provided in a remote operation room.

[0136] Further, the control device 143 according to the above-described embodiment does not have a left front camera that captures the left front range Re, but in other embodiments, the control device 143 can also have a left front camera that captures the left front range Re. In this case, the control device 143 can generate an overhead image that captures the surroundings of the work machine 100.

[0137] The control device 143 according to the above-described embodiment has the display 1431 on which the display image is displayed, but in other embodiments, the control device 143 is not limited thereto. For example, the control device 143 according to other embodiments does not have the display 1431, but can transmit a signal that causes the display image to be displayed on a display 1431 separate from the control device 143. Alternatively, the display system can be implemented by two or more control devices that include a display 1431 separate from the control device 143 and each part of the control device 143 having the above-described structure.

[0138] The control device 143 according to the above-described embodiment generates a display signal for displaying a display image that does not include the overhead image and the single camera image when the work machine 100 becomes the non-working state by pressing the sub switch 1412 or the like, but in other embodiments, the control device 143 is not limited thereto. For example, the control device 143 can also generate a display signal for displaying a display image that includes the overhead image or the single camera image when the work machine 100 becomes the non-working state by pressing the sub switch 1412 or the like.

[0139] In the control device 143 according to the above-described embodiment, the display image including the overhead image or the single camera image is displayed on the display 1431 when displaying a function (for example, a user menu image, a function image, or the like) that is operated by an operator or the like, but in other embodiments, the control device 143 is not limited thereto. For example, in other embodiments, the control device 143 can also display a display image including the overhead image or the single camera image on the display 1431 when displaying a service menu or the like that is operated by a service person.

[0140] Figure 9This is a screen transition diagram illustrating the transition of screens displayed on the display according to another embodiment. After the control device 143 is activated, a default display image D1 including an overhead image G11 is displayed. A display image D8 including an overhead image, a single camera image, and other surrounding images of the work machine 100, as well as a service menu image G92, can be displayed. Display image D8 is displayed by a predetermined operation on the display 1431 when the control device 143 is activated and a display image is being displayed, such as the default display image D1. Display image D8 is displayed by a predetermined operation on the display 1431 when a menu image, such as the display image D2, is being displayed.

[0141] In this way, the control device 143 displays the display image D8 including the surrounding image of the working machine 100 and the service menu image G92, so that the service personnel can confirm the conditions around the working machine 100 while confirming and setting various histories of the working machine 100, and thus can operate the working machine 100 after confirming the safety of the surroundings.

[0142] Furthermore, in the above embodiment, the display 1431 is described as being installed on the work machine 100. However, in other embodiments, the display 1431 may be installed outside the work machine 100. For example, the display 1431 may be installed at a location remote from the work site, and the control device 143 may transmit a signal via a network such as the Internet to cause the display image to appear on the display 1431.

[0143] Industrial Applicability

[0144] According to the disclosure of the present invention, an operator can visually recognize the surrounding conditions of a work machine.

[0145] Description of labels

[0146] 100…Work machine 121…Camera 140…Cab 141…Driver's seat 1411…Engine key cylinder 1412…Auxiliary switch 143…Control device 211…Image acquisition unit 212…Overhead image generation unit 213…Display image generation unit 214…Input unit 215…Selection unit 216…Display control unit 217…Function execution unit 218…Vehicle status monitoring unit.

Claims

1. A display system for a working machine, comprising: an image acquisition unit that acquires images of the surroundings of the work machine from a plurality of cameras provided on the work machine; a display image generating unit that generates a surrounding image based on the captured image, a single camera image obtained from one of the plurality of cameras in the captured image, and a function image for realizing a plurality of functions possessed by the working machine; and The display control unit displays the surrounding image and the single camera image on the display unit and, when the working machine is in an operating state, receives a transition signal for causing the screen of the display unit to transition multiple times to display the functional image by a user's operation of transitioning multiple screens. When transitioning to each of the transitioned screens of the display unit, the display control unit generates a display signal for displaying a display image including the surrounding image while maintaining the display of the surrounding image. When transitioning to the screen of the display unit displaying the functional image, the display control unit generates a display signal for displaying a display image including the surrounding image and the functional image while maintaining the display of the surrounding image. In an emergency, when the working machine changes from the working state to the non-working state by pressing a switch for stopping the engine of the working machine, the display control unit generates a display signal for displaying a display screen that does not include the surrounding image from the display image including the surrounding image.

2. The display system for a working machine according to claim 1, wherein: During a period from when the working machine enters the operating state to when it enters the non-operating state, the display image displayed in response to the display signal includes the surrounding image.

3. The display system for a working machine according to claim 1 or claim 2, wherein: The display image generation unit further generates a service menu image, The display control unit, upon receiving a signal for transitioning the screen of the display unit to display the service menu image by an operation performed by the user, generates a display signal for displaying the display image including the service menu image and the surrounding image on the screen after the transition of the display unit.

4. The display system for a working machine according to claim 1 or claim 2, wherein: Upon receiving the transition signal, the display control unit generates a display signal for displaying a display image including the surrounding image with a changed display size on the screen of the display unit after the transition.

5. A display method for a working machine, comprising: a step of acquiring images of the surroundings of the work machine from a plurality of cameras provided at the work machine; generating a surrounding image based on the captured image, a single camera image obtained from one of the plurality of cameras in the captured image, and a functional image for realizing a plurality of functions of the working machine; and a step of: when the surrounding image and the single camera image are displayed on a display unit and the working machine is in an operating state, upon receiving a transition signal for causing the screen of the display unit to perform a plurality of transitions to display the functional image by a plurality of screen transition operations performed by a user, while maintaining the display of the surrounding image, generating a display signal for displaying a display image including the surrounding image when transitioning to each of the transitioned screens of the display unit; and generating a display signal for displaying a display image including the surrounding image and the functional image when transitioning to the screen of the display unit displaying the functional image, while maintaining the surrounding image; and In an emergency, when the working machine changes from the working state to the non-working state by pressing a switch for stopping the engine of the working machine, a step is performed to generate a display signal for displaying a display screen that does not include the surrounding image from a display image that includes the surrounding image.

Images