Work machine display system and work machine display method

Abstract

To allow an operator to visually recognize a peripheral situation of a work machine.SOLUTION: An image acquisition part acquires a captured image in which the peripheral area of a work machine is captured. A display control part generates a display signal for displaying a display image including a peripheral image based on the photographic image on a screen after screen transition of a display part when receiving a transition signal for transiting the screen when the peripheral image is displayed on the display part.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to a work machine display system and a work machine display method. [Background technology]

[0002] Patent Document 1 discloses a technique for allowing an operator to visually recognize the conditions around a work machine when calibrating the work machine. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 191853 Summary of the Invention [Problem to be solved by the invention]

[0004] According to Patent Document 1, display data showing the situation around the work machine is displayed on the display device, allowing the operator to visually confirm the situation around the work machine during calibration work. Meanwhile, the display device of the work machine can be switched to a screen that displays information related to the functions of the work machine, such as a menu screen. However, on the screen after switching, information related to the function is displayed on the entire display device, and display data showing the situation around the work machine is not displayed. When at least a portion of the work machine is in an operable state, there is a possibility that the operator will attempt to work on the work machine when information related to an arbitrary function is displayed on the display device. An object of the present disclosure is to provide a work machine display system and a work machine display method that allow an operator to visually recognize the conditions around the work machine. [Means for solving the problem]

[0005] According to a first aspect of the present invention, a display system for a work machine includes an image acquisition unit that acquires a captured image showing the surroundings of the work machine, and a display control unit that, when a transition signal for transitioning the screen is received while a surrounding image based on the captured image is being displayed on the display unit, generates a display signal for displaying a display image including the surrounding image on the screen after the transition of the display unit. [Effects of the Invention]

[0006] According to at least one of the above aspects, it is possible to allow the operator to visually recognize the situation around the work machine. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a schematic diagram showing the configuration of a work machine according to a first embodiment. [Figure 2] 2 is a diagram showing the imaging ranges of a plurality of cameras provided on the work machine according to the first embodiment. FIG. [Figure 3] FIG. 2 is a diagram showing the internal configuration of a driver's cab according to the first embodiment. [Figure 4] FIG. 1 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. [Figure 5] 5 is a flowchart showing a display control process of the control device according to the first embodiment. [Figure 6] FIG. 2 is a screen transition diagram showing transition of screens displayed on a display according to the first embodiment. [Figure 7] 10 is a flowchart showing a display control process of a control device according to a second embodiment. [Figure 8] FIG. 10 is a screen transition diagram showing transition of screens displayed on a display according to the second embodiment. [Figure 9] 10A and 10B are screen transition diagrams showing transitions of screens displayed on a display according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] First Embodiment Hereinafter, the embodiments will be described in detail with reference to the drawings.

[0009] <<Configuration of the work machine>> FIG. 1 is a schematic diagram showing the configuration of a work machine according to a first embodiment. The work machine 100 operates at a construction site and works on a construction target such as earth and sand. The work machine 100 according to the first embodiment is, for example, a hydraulic excavator. The work machine 100 includes a traveling body 110, a rotating body 120, a work implement 130, and a cab 140. The running body 110 supports the working machine 100 so that the working machine 100 can travel. The running body 110 is, for example, a pair of left and right caterpillars. The rotating body 120 is supported by the running body 110 so as to be able to rotate around a rotation center. The work implement 130 is hydraulically driven. The work implement 130 is supported on the front part of the rotating body 120 so that it can be driven in the vertical direction. The operator's cab 140 is a space where an operator sits and operates the work machine 100. The operator's cab 140 is provided on the left front part of the rotating body 120. Here, the portion of the revolving unit 120 to which the work implement 130 is attached is referred to as the front portion. Furthermore, with respect to the revolving unit 120, the portion opposite the front portion is referred to as the rear portion, the left portion as the left portion, and the right portion as the right portion.

[0010] <Configuration of rotating body> The revolving unit 120 is provided with a plurality of cameras 121 that capture images of the surroundings of the work machine 100. Fig. 2 is a diagram showing the imaging ranges of the multiple cameras provided on the work machine according to the first embodiment. Specifically, the revolving unit 120 is provided with a left rear camera 121A that images a left rear range Ra of the periphery of the revolving unit 120, a rear camera 121B that images a rear range Rb of the periphery of the revolving unit 120, a right rear camera 121C that images a right rear range Rc of the periphery of the revolving unit 120, and a right front camera 121D that images a right front range Rd of the periphery of the revolving unit 120. Note that the imaging ranges of the multiple cameras 121 may partially overlap each other. The imaging ranges of the multiple cameras 121 cover the entire circumference of the work machine 100 excluding the left front range Re that is visible from the cab 140. Note that, although the cameras 121 according to the first embodiment capture images of the left rear, rear, right rear, and right front of the revolving unit 120, other embodiments are not limited to this. For example, the number and imaging ranges of the cameras 121 according to other embodiments may differ from the examples shown in FIGS. 1 and 2.

[0011] Note that, as shown in the rear range Rb in Fig. 2, the left rear camera 121A images the left side area and the left rear area of ​​the revolving unit 120, but may image either one of these areas. Similarly, as shown in the right rear range Rc in Fig. 2, the right rear camera 121C images the right side area and the right rear area of ​​the revolving unit 120, but may image either one of these areas. Similarly, as shown in the right front range Rd in Fig. 2, the right front camera 121D images the right front area and the right side area of ​​the revolving unit 120, but may image either one of these areas.

[0012] <<Configuration of the work machine>> The work implement 130 includes a boom 131 , an arm 132 , a bucket 133 , a boom cylinder 134 , an arm cylinder 135 , and a bucket cylinder 136 .

[0013] The base end of the boom 131 is attached to the revolving body 120 via a boom pin P1. The arm 132 connects the boom 131 and the bucket 133. The base end of the arm 132 is attached to the tip of the boom 131 via an arm pin P2. The bucket 133 includes a blade for digging up earth and sand and a storage section for storing the excavated earth and sand. The base end of the bucket 133 is attached to the tip of the arm 132 via a bucket pin P3.

[0014] The boom cylinder 134 is a hydraulic cylinder for operating the boom 131. A base end of the boom cylinder 134 is attached to the revolving body 120. A tip end of the boom cylinder 134 is attached to the boom 131. The arm cylinder 135 is a hydraulic cylinder for driving the arm 132. A base end of the arm cylinder 135 is attached to the boom 131. A tip end of the arm cylinder 135 is attached to the arm 132. The bucket cylinder 136 is a hydraulic cylinder for driving the bucket 133. A base end of the bucket cylinder 136 is attached to the arm 132. A tip end of the bucket cylinder 136 is attached to a link member connected to the bucket 133.

[0015] <<Configuration of the driver's cab>> FIG. 3 is a diagram showing the internal configuration of the operator's cab according to the first embodiment. In the driver's cab 140, a driver's seat 141, an engine key cylinder 1411 (primary switch), a secondary switch 1412, an operating device 142, and a control device 143 are provided.

[0016] Engine key cylinder 1411 accepts the insertion of an engine key, and when the inserted engine key is rotated in a first direction, the engine (not shown) starts. Furthermore, when the engine is running, the engine key inserted into engine key cylinder 1411 is rotated in a second direction opposite to the first direction, the engine stops. The secondary switch 1412 is provided below the driver's seat 141. The secondary switch 1412 is a switch for stopping the engine in an emergency. When the secondary switch 1412 is pressed while the engine is running, the engine stops.

[0017] The operation device 142 is an interface for manually operating the traveling body 110, the rotating body 120, and the work machine 130 by an operator. The operation device 142 includes a left operation lever 1421, a right operation lever 1422, a left foot pedal 1423, a right foot pedal 1424, a left travel lever 1425, and a right travel lever 1426.

[0018] The left operating lever 1421 is provided on the left side of the driver's seat 141. The right operating lever 1422 is provided on the right side of the driver's seat 141.

[0019] The left operation lever 1421 is an operation mechanism for performing the rotation operation of the revolving unit 120 and the pulling / pushing operation of the arm 132. Specifically, when the operator of the work machine 100 tilts the left operation lever 1421 forward, the arm 132 performs a pushing operation. Furthermore, when the operator of the work machine 100 tilts the left operation lever 1421 rearward, the arm 132 performs a pulling operation. Furthermore, when the operator of the work machine 100 tilts the left operation lever 1421 to the right, the revolving unit 120 rotates to the right. Furthermore, when the operator of the work machine 100 tilts the left operation lever 1421 to the left, the revolving unit 120 rotates to the left. Note that in other embodiments, the revolving unit 120 may rotate to the right or left when the left operation lever 1421 is tilted forward or backward, and the arm 132 may perform a pulling or pushing operation when the left operation lever 1421 is tilted left or right.

[0020] The right operating lever 1422 is an operating mechanism for performing an excavation / dumping operation of the bucket 133 and a raising / lowering operation of the boom 131. Specifically, when the operator of the work machine 100 tilts the right operating lever 1422 forward, a lowering operation of the boom 131 is performed. Conversely, when the operator of the work machine 100 tilts the right operating lever 1422 rearward, a raising operation of the boom 131 is performed. Conversely, when the operator of the work machine 100 tilts the right operating lever 1422 to the right, a dumping operation of the bucket 133 is performed. Conversely, when the operator of the work machine 100 tilts the right operating lever 1422 to the left, an excavation operation of the bucket 133 is performed. Note that in other embodiments, when the right operating lever 1422 is tilted forward or backward, the bucket 133 may perform a dumping operation or an excavation operation, and when the right operating lever 1422 is tilted left or right, the boom 131 may perform a raising or lowering operation.

[0021] The left foot pedal 1423 is located on the left side of the floor in front of the driver's seat 141. The right foot pedal 1424 is located on the right side of the floor in front of the driver's seat 141. The left travel lever 1425 is pivotally supported by the left foot pedal 1423, and is configured so that tilting the left travel lever 1425 and pressing down the left foot pedal 1423 are linked. The right travel lever 1426 is pivotally supported by the right foot pedal 1424, and is configured so that tilting the right travel lever 1426 and pressing down the right foot pedal 1424 are linked.

[0022] The left foot pedal 1423 and left travel lever 1425 correspond to the rotational drive of the left track of the running body 110. Specifically, when the operator of the work machine 100 pushes the left foot pedal 1423 or the left travel lever 1425 forward, the left track rotates in the forward direction. Conversely, when the operator of the work machine 100 pushes the left foot pedal 1423 or the left travel lever 1425 backward, the left track rotates in the reverse direction.

[0023] The right foot pedal 1424 and the right travel lever 1426 correspond to the rotational drive of the right track of the running body 110. Specifically, when the operator of the work machine 100 pushes the right foot pedal 1424 or the right travel lever 1426 forward, the right track rotates in the forward direction. Conversely, when the operator of the work machine 100 pushes the right foot pedal 1424 or the right travel lever 1426 backward, the right track rotates in the reverse direction.

[0024] The control device 143 is an input / output device, and is equipped with a display 1431 that displays information related to multiple functions of the work machine 100. The control device 143 is an example of a display system. The display 1431 is an example of a display unit. The input means of the control device 143 according to the first embodiment is a hard key. Note that in other embodiments, a touch panel, a mouse, a keyboard, or the like may also be used as the input means. Also, although the control device 143 according to the first embodiment is provided integrally with the display 1431, in other embodiments, the display 1431 may be provided separately from the control device 143.

[0025] <Configuration of the control device> FIG. 4 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. The control device 143 is a computer that includes a processor 210 , a main memory 230 , a storage 250 , and an interface 270 .

[0026] The display 1431 is connected to the processor 210 via the interface 270 . The storage 250 is a non-transitory tangible storage medium. Examples of the storage 250 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), and a semiconductor memory. The storage 250 may be an internal medium directly connected to the bus of the control device 143, or an external medium connected to the control device 143 via the interface 270 or a communication line. The storage 250 stores a program for implementing monitoring of the surroundings of the work machine 100. The storage 250 also stores in advance a plurality of images including icons to be displayed on the display, user menu images, service menu images, and function images.

[0027] The program may be for realizing some of the functions to be performed by the control device 143. For example, the program may be combined with other programs already stored in the storage 250 or other programs implemented in other devices to perform the functions. In other embodiments, the control device 143 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor may be realized by the integrated circuit.

[0028] By executing the program, processor 210 functions as an image acquisition unit 211, an overhead image generation unit 212, a display image generation unit 213, an input unit 214, a selection unit 215, a display control unit 216, a function execution unit 217, and a vehicle state monitoring unit 218. In addition, processor 210 may read a plurality of images stored in storage 250 into main memory 230 by executing the program.

[0029] The image acquisition unit 211 acquires captured images from the multiple cameras 121 . The overhead image generating unit 212 generates an overhead image of the work site viewed from above in a plan view, with the work machine 100 at the center, by transforming and combining the multiple captured images acquired by the image acquiring unit 211. The overhead image generating unit 212 may also generate an overhead image by cutting out a portion of each of the transformed captured images and combining the cut-out captured images. An image of the work machine 100 viewed from above in a plan view is attached in advance to the center of the overhead image generated by the overhead image generating unit 212. The overhead image is an example of a surrounding image that shows the surroundings of the work machine 100.

[0030] The display image generation unit 213 generates a display image showing information for realizing each of the multiple functions possessed by the work machine 100. Examples of functions possessed by the work machine 100 include a function for displaying a user menu, a function for displaying vehicle information such as the remaining fuel level and coolant temperature, a function for adjusting the power mode of the work machine 100, and the like. The display images related to each function generated by the display image generating section 213 include the overhead image generated by the overhead image generating section 212 and a function image for realizing the function.

[0031] The input unit 214 receives an input signal generated when a user presses a hard key. The selection unit 215 identifies the function selected by the user from among the plurality of functions possessed by the work machine 100, based on the input signal received by the input unit 214. In other words, the selection unit 215 receives a selection of any one of the plurality of functions possessed by the work machine 100. The selection unit 215 is also an example of a transition reception unit that receives a transition signal for transitioning the screen displayed on the display 1431 of the work machine 100.

[0032] The display control unit 216 generates a display signal for displaying the display image related to the function specified by the selection unit 215 from among the display images generated by the display image generation unit 213, and outputs the signal to the display 1431. The function execution unit 217 executes the function specified by the selection unit 215 .

[0033] The vehicle state monitoring unit 218 monitors the vehicle state, including the operating state of the engine. For example, if the engine is stopped by pressing the secondary switch 1412, the vehicle state monitoring unit 218 detects that the engine has stopped.

[0034] <<Operation of the control device>> Fig. 5 is a flowchart showing a display control process of the control device according to the first embodiment Fig. 6 is a screen transition diagram showing transition of screens displayed on the display according to the first embodiment.

[0035] When the engine key is inserted into the engine key cylinder 1411 and turned in the first direction, the engine starts and the control device 143 starts up. When the control device 143 starts up, a display image D0 indicating that the work machine 100 is starting up is displayed on the display 1431, as shown in FIG. 6. While the display image D0 is being displayed, the control device 143 has not completed starting up and an overhead image cannot be generated. Therefore, the display image D0 does not include an overhead image. Note that the work machine 100 is in a non-operating state until the start-up of the control device 143 is complete.

[0036] When the activation of the control device 143 is completed, the display control process shown in Fig. 5 is executed. First, the image acquisition unit 211 acquires captured images from the multiple cameras 121 (step S1). Next, the overhead image generation unit 212 generates an overhead image by transforming and combining the multiple captured images acquired in step S1 (step S2). Next, the display image generation unit 213 generates a default display image D1 including one of the captured images acquired in step S1 (single camera image G12) and the overhead image G11 generated in step S3 (step S3). For example, in the display image D1 of Fig. 6, an image captured by the right rear camera 121C is arranged as the single camera image G12.

[0037] As shown in Fig. 6, the control device 143 has six hard function keys F1-F6 directly below the display 1431. An icon G13 is placed in the lower part of the display image D1 in a portion corresponding to the function key F6. The icon G13 indicates the display function of the user menu. In other words, the icon G13 indicates that the display function of the user menu is executed by pressing the function key F6. The icon G13 is stored in advance in the storage 250 or the main memory 230.

[0038] The display control unit 216 outputs a display signal for displaying the display image D1 generated in step S3 to the display 1431 (step S4). As a result, when the startup of the control device 143 is completed and the working machine 100 is in an operating state, a display image including an overhead image is displayed on the display 1431.

[0039] Next, when the input unit 214 detects whether or not the user has pressed a function key, the selection unit 215 determines whether or not the pressed function key is the function key F6, which indicates the display function of the user menu (step S5). If the function key F6 has not been pressed (step S5: NO), the control device 143 returns the process to step S1 and continues generating and displaying the display image D1. On the other hand, if the function key F6 has been pressed (step S5: YES), the selection unit 215 accepts the selection of the display function of the user menu (step S6). That is, the selection unit 215 accepts the signal generated by pressing the function key F6 as a transition signal to the display screen of the user menu.

[0040] Next, the display image generation unit 213 generates a display image D2 (display image D2A, display image D2B) that includes the overhead image G21 generated in step S3 and a pre-stored user menu image G22 (step S7). The user menu image G22 is a functional image for realizing the display function of the user menu. The user menu image G22 includes a list of multiple functions of the work machine 100, and one of the multiple items in the list is in a focused state. The focused item is set in a color different from the other items. The user menu image G22 displays multiple tabs that indicate the type of list. That is, as shown in the display images D2A and D2B, different lists are arranged in the user menu image G22 depending on the tab that has focus. Furthermore, icons G23-G28 are arranged in the portions of the lower part of the display image D2 that correspond to the function keys F1-F6, respectively. The icons G23-G28 are stored in advance in the storage 250 or the main memory 230.

[0041] Icon G23 indicates leftward movement, i.e., the icon G23 indicates that pressing the function key F1 moves the focus of the user menu tab to the left. Icon G24 indicates rightward movement, i.e., icon G24 indicates that pressing function key F2 moves the focus of the user menu tab to the right. Icon G25 indicates downward movement, i.e., icon G25 indicates that pressing function key F3 moves the focus of the user menu downward. Icon G26 indicates upward movement, i.e., icon G26 indicates that pressing function key F4 moves the focus of the user menu upward. Icon G27 indicates a transition to the previous screen. That is, icon G27 indicates that pressing function key F5 cancels the display function of the user menu and displays the previous screen. Icon G28 indicates that a function having focus in the user menu is selected by pressing function key F6, and a display screen relating to that function is displayed.

[0042] The display control unit 216 outputs a display signal for displaying the display image D2 generated in step S7 to the display 1431 (step S8). The display image D2 including the overhead image is displayed on the display 1431. At this time, the work machine 100 is in an operating state.

[0043] Next, when the input unit 214 detects whether the user has pressed a function key, the selection unit 215 determines whether the pressed function key is function key F1 or F2, which indicates left / right movement, function key F3 or F4, which indicates up / down movement, function key F5, which indicates transition to the previous screen, or function key F6, which indicates confirmation (step S9).

[0044] If the function key F3 or F4 indicating up / down movement is pressed (step S9: up / down movement), the display control unit 216 changes the item having the focus based on the pressed function key, returns the process to step S7, and updates the display image D2. When the function key F1 or F2 indicating left / right movement is pressed (step S9: left / right movement), the display image generation unit 213 acquires the user menu image G22 after the tab has been switched from the storage 250 or the main memory 230 based on the pressed function key (step S10). Then, the control device 143 returns the process to step S7 and updates the display image D2. For example, when the function key F2 is pressed while the display image D2A shown in FIG. 6 is being displayed, the display image generation unit 213 generates the display image D2B.

[0045] If the function key F5 is pressed (step S9: previous screen), the control device 143 returns the process to step S1 and causes the display 1431 to display the display image D1. When the function key F6 is pressed (step S9: Confirm), the selection unit 215 accepts the selection of the function related to the item having the focus among the multiple functions (step S11). That is, the selection unit 215 accepts the signal generated by pressing the function key F6 as a transition signal to the display screen of the functional image.

[0046] Next, the function executing unit 217 acquires a function image G32 for realizing the function selected in step S11 from the storage 250 or the main memory 230 (step S12). For example, if the selected function is a function for adjusting the power mode of the work machine 100, the function executing unit 217 acquires a function image for adjusting the power mode. Alternatively, for example, if the selected function is an attachment setting function for the work machine 100, the function executing unit 217 acquires a function image for setting the flow rate of oil used for the attachment provided on the work machine 100. This makes it possible to change the power of the attachment.

[0047] Next, display image generation unit 213 generates display image D3 including overhead image G31 generated in step S2 and function image G32 acquired in step S12 (step S13). In the example shown in Fig. 6, icons G33-G36 are arranged in portions corresponding to function keys F1-F6 at the bottom of display image D3. Icons G33-G36 are stored in advance in storage 250 or main memory 230.

[0048] Icon G33 is an icon indicating downward movement. Icon G34 is an icon indicating upward movement. Icon G35 is an icon indicating transition to the previous screen. Icon G36 is an icon indicating confirmation. Note that the type and number of icons arranged in display image D3 may differ depending on the selected function. On the other hand, at least an icon indicating transition to the previous screen is arranged in display image D3.

[0049] The display control unit 216 outputs a display signal for displaying the display image D3 generated in step S14 to the display 1431 (step S14). The display image D3 including the overhead image is displayed on the display 1431. At this time, the work machine 100 is in an operating state.

[0050] Next, when the input unit 214 detects whether the user has pressed a function key, the selection unit 215 determines whether the pressed function key is the function key F5, which indicates a transition to the previous screen, or any of the other function keys F1-F4 and F6 (step S15).

[0051] If any of the function keys F1-F4 and F6 is pressed (step S15: Other), the function execution unit 217 executes the function selected in step S11 based on the pressed function key (step S16) and ends the process. Thereafter, the control device 143 executes the display control process shown in Fig. 5 again and causes the display 1431 to display the display image D1.

[0052] On the other hand, if function key F5 is pressed (step S15: previous screen), the process returns to step S7, and the previous screen, display image D2, is displayed on display 1431. That is, if the previous screen was display image D2A, display image D2A is displayed again, and if the previous screen was display image D2B, display image D2B is displayed again.

[0053] 5 is an example, and in other embodiments, not all steps may necessarily be executed. For example, in other embodiments, if a function can be selected without using display image D2 by using a shortcut button or the like, control device 143 may not execute steps S6-10. Furthermore, if displaying display image D3 itself corresponds to executing a function, control device 143 may not execute step S16.

[0054] When the vehicle state monitoring unit 218 of the control device 143 detects that the engine has been stopped due to the pressing of the secondary switch 1412, the display control unit 216 outputs a display signal to the display 1431 by interrupt processing to display a stop display screen that prompts the user to turn off the engine key. This display signal does not include an overhead image. While the engine is stopped due to the pressing of the secondary switch 1412, the work machine 100 is in a non-operating state.

[0055] Actions and Effects In this way, when the control device 143 according to the first embodiment receives a transition signal while an overhead image is being displayed on the display 1431, it causes a display image including the overhead image to be displayed on the screen after the transition. This allows the control device 143 to keep the overhead image displayed on the display 1431 at all times, even if the screen is transitioned by an operation by the operator. This allows the control device 143 to allow the operator to visually recognize the situation around the work machine.

[0056] Furthermore, in the control device 143 according to the first embodiment, the display image displayed on the display 1431 in response to a display signal during the period from when the work machine 100 is in an operating state until when it is in a non-operating state includes a surrounding image, as shown in Fig. 6. This makes it possible to keep the overhead image displayed on the display 1431 at all times while the operator is able to operate at least a portion of the work machine 100.

[0057] Furthermore, the control device 143 according to the first embodiment generates a display signal for displaying a display image that does not include an overhead image when the work machine 100 is placed in a non-operating state by, for example, pressing the secondary switch 1412. This allows the control device 143 to indicate to the operator whether or not the work machine 100 is operable, based on the presence or absence of an overhead image. In other words, the operator can intuitively recognize whether or not the work machine 100 is operable, based on the presence or absence of an overhead image.

[0058] Furthermore, in the first embodiment, the multiple functions of the work machine 100 include a user menu display function, and the display image D2 includes an overhead image. This allows the control device 143 to keep the overhead image displayed on the display 1431 even when the display image D2 is being displayed.

[0059] Furthermore, in the first embodiment, the overhead image G11, overhead image G21, and overhead image G31 are all displayed in the same position. Therefore, the display position of the overhead image is constant regardless of the screen transition, so the operator can check the surroundings of the work machine 100 without feeling uncomfortable before and after the screen transition.

[0060] Furthermore, in the first embodiment, the display images including the user menu image G22 and the function image G32 do not display the service meters that display the engine water temperature, remaining fuel, and engine operating time that were displayed in the display image D1. In this way, by displaying only the information necessary for the operator after the screen transition, it is possible to prevent the display from becoming too cluttered.

[0061] Second Embodiment The control device 143 according to the first embodiment includes an overhead image in the display image while the work machine 100 is in operation. In contrast, the control device 143 according to the second embodiment includes a single camera image in the display image while the work machine 100 is in operation. The single camera image is an example of a surrounding image that shows the surroundings of the work machine 100. The control device 143 according to the second embodiment does not need to include the overhead image generating unit 212.

[0062] Fig. 7 is a flowchart showing a display control process of the control device according to the second embodiment Fig. 8 is a screen transition diagram showing transition of screens displayed on the display according to the second embodiment. When the engine key is inserted into the engine key cylinder 1411 and rotated in a first direction, the engine starts and the control device 143 starts up. When the control device 143 starts up, a display image D0 indicating that the work machine 100 is starting up is displayed on the display 1431, as shown in Fig. 6 .

[0063] When the startup of the control device 143 is completed, the display control process shown in FIG. 7 is executed. First, the image acquisition unit 211 acquires captured images from the multiple cameras 121 (step S101). Next, the display image generation unit 213 generates a default display image D5 including two of the captured images acquired in step S101 (single camera images G51 and G52) (step S102). For example, in the display image D5 of FIG. 8, an image captured by the right front camera 121D is arranged as the single camera image G51, and an image captured by the right rear camera 121C is arranged as the single camera image G52. An icon G53 is arranged in a portion of the lower part of the display image D5 corresponding to the function key F6. The icon G53 indicates a display function of the user menu. The icon G53 is stored in advance in the storage 250 or the main memory 230.

[0064] The display control section 216 outputs a display signal for displaying the display image D5 generated in step S102 to the display 1431 (step S103). As a result, when the startup of the control device 143 is completed and the work machine 100 enters an operating state, a display image including the single camera image is displayed on the display 1431.

[0065] Next, when the input unit 214 detects whether or not the user has pressed a function key, the selection unit 215 determines whether or not the pressed function key is the function key F6, which indicates the display function of the user menu (step S104). If the function key F6 has not been pressed (step S104: NO), the control device 143 returns the process to step S101 and continues generating and displaying the display image D5. On the other hand, if the function key F6 has been pressed (step S104: YES), the selection unit 215 accepts the selection of the display function of the user menu (step S105). That is, the selection unit 215 accepts the signal generated by pressing the function key F6 as a transition signal to the display screen of the user menu.

[0066] Next, display image generation unit 213 generates display image D6 (display image D6A, display image D6B) including single camera images G61 and G62 acquired in step S101 and a pre-stored user menu image G63 (step S106). Icons G64-G69 are displayed in the portions corresponding to function keys F1-F6 at the bottom of display image D6. Icons G64-G69 are stored in advance in storage 250 or main memory 230.

[0067] The display control unit 216 outputs a display signal for displaying the display image D6 generated in step S106 to the display 1431 (step S107). The display image D6 including the single camera image is displayed on the display 1431. At this time, the work machine 100 is in an operating state.

[0068] Next, when the input unit 214 detects whether the user has pressed a function key, the selection unit 215 determines whether the pressed function key is function key F1 or F2, which indicates left / right movement, function key F3 or F4, which indicates up / down movement, function key F5, which indicates transition to the previous screen, or function key F6, which indicates confirmation (step S108).

[0069] If the function key F3 or F4 indicating up / down movement is pressed (step S108: up / down movement), the display control unit 216 changes the item having the focus based on the pressed function key, returns the process to step S106, and updates the display image D6. When function key F1 or F2 indicating left / right movement is pressed (step S108: left / right movement), display image generation unit 213 acquires user menu image G63 after tab switching from storage 250 or main memory 230 based on the pressed function key (step S109). Then, control device 143 returns the process to step S106 and updates display image D6. For example, when function key F2 is pressed while display image D6A shown in FIG. 8 is displayed, display image generation unit 213 generates display image D6B.

[0070] If the function key F5 is pressed (step S108: previous screen), the control device 143 returns the process to step S101 and causes the display 1431 to display the display image D5. When the function key F6 is pressed (step S108: Confirm), the selection unit 215 accepts the selection of the function related to the item having the focus among the multiple functions (step S110). That is, the selection unit 215 accepts the signal generated by pressing the function key F6 as a transition signal to the display screen of the functional image.

[0071] Next, the function executing unit 217 acquires a functional image G72 for realizing the function selected in step S110 from the storage 250 or the main memory 230 (step S111). Next, the display image generating unit 213 generates a display image D7 including the single camera images G71 and G72 acquired in step S101 and the functional image G73 acquired in step S111 (step S112). In the example shown in Fig. 6, icons G74-G76 are arranged in the portions corresponding to the function keys F3-F6 at the bottom of the display image D7, respectively.

[0072] The display control unit 216 outputs a display signal for displaying the display image D7 generated in step S112 to the display 1431 (step S113). Next, when the input unit 214 detects whether the user has pressed a function key, the selection unit 215 determines whether the function key pressed is the function key F5 indicating a transition to the previous screen, or any of the other function keys F3, F4, and F6 (step S114).

[0073] If any of function keys F3, F4, and F6 is pressed (step S114: Other), function execution unit 217 executes the function selected in step S110 based on the pressed function key (step S115), and ends the process. Thereafter, control device 143 again executes the display control process shown in FIG. 7, and causes display image D5 to be displayed on display 1431.

[0074] On the other hand, if function key F5 is pressed (step S114: previous screen), the process returns to step S106, and the previous screen, display image D6, is displayed on display 1431. That is, if the previous screen was display image D6A, display image D6A is displayed again, and if the previous screen was display image D6B, display image D6B is displayed again.

[0075] 7 is an example, and in other embodiments, not all steps may necessarily be executed. For example, in other embodiments, if a function can be selected without using display image D6 by using a shortcut button or the like, control device 143 may not execute steps S105-109. Furthermore, if displaying display image D7 itself corresponds to executing a function, control device 143 may not execute step S115.

[0076] Other Embodiments Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design modifications are possible. That is, in other embodiments, the order of the above-described processes may be changed as appropriate. Furthermore, some processes may be executed in parallel.

[0077] In the above-described embodiment, the work machine 100 has been described as a hydraulic excavator, but in other embodiments, the work machine 100 can be applied to various work machines such as a dump truck, a wheel loader, etc. For example, the work machine 100 can be applied to a display system for a dump truck, a wheel loader, or other work machines.

[0078] Furthermore, in the above-described embodiment, a user menu image is displayed instead of the single camera image G12. However, in other embodiments, the user menu image may be displayed together with the overhead image and the single camera image. That is, the overhead image generation unit 212 generates a reduced overhead image so that it can be displayed together with the user menu and the single camera image. Furthermore, the display image generation unit 213 generates a reduced single camera image so that it can be displayed together with the user menu and the overhead image. That is, when the display control unit 216 receives a transition signal, it generates a display signal for displaying a display image including a surrounding image whose display size has been changed on the post-transition screen.

[0079] Furthermore, in the above-described embodiment, a single control device 143 is described as being installed on the work machine 100, but in other embodiments, a portion of the configuration of the control device 143 may be arranged in another control device, and the display system may be realized by two or more control devices. Note that the single control device 143 shown in the above-described embodiment is also an example of a display system.

[0080] Furthermore, the control device 143 according to the above-described embodiment has been described as being installed on the work machine 100, but in other embodiments, part or all of the configuration of the control device 143 may be installed outside the work machine 100. For example, in other embodiments, the control device 143 may control the work machine 100 in a remotely operated manner. In this case, the control device 143 causes a display device located in a remote operation room to display the above-described screen.

[0081] Furthermore, although the control device 143 according to the embodiment described above is not provided with a left front camera that captures images of the left front range Re, in other embodiments it may be provided with a left front camera that captures images of the left front range Re. In this case, the control device 143 can generate an overhead image that captures the entire periphery of the work machine 100.

[0082] The control device 143 according to the above-described embodiment has a display 1431 and displays a display image on the display 1431, but other embodiments are not limited to this. For example, the control device 143 according to other embodiments may not have the display 1431, but may transmit a signal to display a display image on the display 1431 separate from the control device 143. Note that the control device 143 may be realized as a display system including the display 1431 separate from the control device 143 and two or more control devices each having part of the configuration of the control device 143 described above.

[0083] The control device 143 according to the embodiment described above generates a display signal for displaying a display image that does not include an overhead image or a single camera image when the work machine 100 is placed in a non-operating state by, for example, pressing the secondary switch 1412, but this is not limited to this in other embodiments. For example, even when the work machine 100 is placed in a non-operating state by, for example, pressing the secondary switch 1412, a display signal for displaying a display image that includes an overhead image or a single camera image may be generated.

[0084] In the control device 143 according to the embodiment described above, when a function for accepting an operation by an operator or the like (for example, a user menu image or a function image) is displayed, a display image including an overhead image or a single camera image is displayed on the display 1431. However, in other embodiments, this is not limited to this. For example, in other embodiments, the control device 143 may display a display image including an overhead image or a single camera image on the display 1431 when a service menu for accepting an operation by a service technician or the like is displayed.

[0085] 9 is a screen transition diagram showing the transition of screens displayed on a display according to another embodiment. After the control device 143 is started up, a default display image D1 including an overhead image G11 may be displayed, and a display image D8 including an image of the surroundings of the work machine 100, such as an overhead image or a single camera image, and a service menu image G92 may be displayed. Display image D8 is displayed by a predetermined operation of the display 1431 when a display image is being displayed after the control device 143 is started up, for example, when the default display image D1 is being displayed. Display image D8 may also be displayed by a predetermined operation of the display 1431 when a menu image such as display image D2 is being displayed.

[0086] In this way, the control device 143 displays the display image D8 including an image of the surroundings of the work machine 100 and the service menu image G92, so that the service technician can check the situation around the work machine 100 while checking the various histories of the work machine 100 or performing settings, and can operate the work machine 100 after confirming that the surroundings are safe.

[0087] Furthermore, in the above-described embodiment, the display 1431 is described as being installed on the work machine 100, but in other embodiments, the display 1431 may be installed outside the work machine 100. For example, the display 1431 may be provided at a location away from the work site, and the control device 143 may transmit a signal to display an image on the display 1431 via a network such as the Internet. [Explanation of symbols]

[0088] 100... Work machine 121... Camera 140... Driver's cab 141... Driver's seat 1411... Engine key cylinder 1412... Secondary switch 143... Control device 211... Image acquisition unit 212... Bird's-eye view image generation unit 213... Display image generation unit 214... Input unit 215... Selection unit 216... Display control unit 217... Function execution unit 218... Vehicle state monitoring unit

Claims

1. A display system for a work machine, comprising: an image acquisition unit that acquires a captured image showing the surroundings of the work machine; a display control unit that, when a transition signal for transitioning to a second screen for accepting selection of one item from a plurality of menu items is received while a first screen including a surrounding image based on the captured image is being displayed on a display unit, generates a display signal for displaying a display image including the surrounding image on the second screen after the transition on the display unit while maintaining the display of the surrounding image, and causes the display to display a screen according to the display signal; Equipped with In a display area of ​​the display, which comprises a first area and a second area, the first screen is a screen in which a first peripheral image is arranged in the first area and a second peripheral image is arranged in the second area, the second screen is a screen in which the first surrounding image and the second surrounding image are arranged side by side in the first area, and a menu image representing the menu item is arranged in the second area. Work machine display system.

2. The display image displayed by the display signal includes the surrounding image from the time when the work machine is in an operating state with the engine running until the time when the work machine is in a non-operating state with the engine stopped. The display system for a work machine according to claim 1 .

3. 3. The work machine display system according to claim 1 or 2, wherein the display control unit generates a display signal for displaying a display image that does not include the surrounding image, instead of a display image that includes the surrounding image, when the work machine is in a non-operating state.

4. The display system of the work machine is a display system for displaying an image including a service menu image, When the display control unit receives a signal for transitioning to a screen displaying the service menu image, the display control unit generates a display signal for displaying the display image including the service menu image and the surrounding image on the transitioned screen of the display unit. The display system for a work machine according to any one of claims 1 to 3.

5. When the display control unit receives the transition signal, the display control unit generates a display signal for displaying a display image including the surrounding image with a changed display size on a post-transition screen of the display unit. The display system for a work machine according to any one of claims 1 to 4.

6. A method for displaying a work machine, comprising: acquiring a captured image showing the surroundings of the work machine; When a transition signal for transitioning to a second screen for accepting selection of one item from a plurality of menu items is received while a first screen including a surrounding image based on the captured image is being displayed on a display unit, a step of generating a display signal for displaying a display image including the surrounding image on the second screen after transition of the display unit while maintaining the display of the surrounding image; displaying a screen according to the display signal on a display; Equipped with In a display area of ​​the display, which comprises a first area and a second area, the first screen is a screen in which a first peripheral image is arranged in the first area and a second peripheral image is arranged in the second area, A display method for a work machine, wherein the second screen is a screen in which the first surrounding image and the second surrounding image are arranged side by side in the first area, and a menu image representing the menu item is arranged in the second area.

Images