Display control device and display control method

Abstract

This display control device displays, on a display screen, prescribed information pertaining to a work machine. The display screen includes a plurality of display regions for displaying the prescribed information. State information pertaining to the operation of the work machine is displayed in the display regions. When a prescribed input operation is performed, the state information pertaining to the operation of the work machine is displayed enlarged.

Description

Display control device and display control method

[0001] The present disclosure relates to a display control device and a display control method. This application claims priority based on Japanese Patent Application No. 2024-168224 filed in Japan on September 27, 2024, and incorporates its content herein by reference.

[0002] In Patent Document 1, on a display provided inside the operator's cab of a work machine, an overhead image representing the surroundings of the work machine generated based on captured images taken by a plurality of cameras, a single-camera image taken by one camera, and images representing an engine coolant temperature gauge, an operating oil temperature gauge, and a fuel gauge are displayed.

[0003] Japanese Unexamined Patent Application Publication No. 2021-42638

[0004] In the display images shown in Patent Document 1, compared with the overhead image and the single-camera image, the images representing each gauge are displayed smaller, so there is a problem that it is difficult to see each gauge.

[0005] The present disclosure has been made to solve the above problems, and an object thereof is to provide a display control device and a display control method capable of easily displaying an image representing predetermined information.

[0006] The display control device of the present disclosure is a display control device that displays predetermined information related to a work machine on a display screen. The display screen includes a plurality of display areas for displaying the predetermined information, and state information related to the operation of the work machine is displayed in the display area. When a predetermined input operation is performed, the state information related to the operation of the work machine is enlarged and displayed.

[0007] The display control method of the present disclosure is a display control method that displays predetermined information related to a work machine on a display screen. The display screen includes a plurality of display areas for displaying the predetermined information, and state information related to the operation of the work machine is displayed in the display area. When a predetermined input operation is performed, the state information related to the operation of the work machine is enlarged and displayed.

[0008] According to the display control device and the display control method of the present disclosure, an image representing predetermined information can be easily displayed.

[0009] This is a schematic diagram showing the configuration of a work machine according to an embodiment of this disclosure. This is a diagram showing the internal configuration of the operator's cab according to an embodiment of this disclosure. This is a schematic block diagram showing the hardware configuration of the control system according to an embodiment of this disclosure. This is a diagram showing an example of a display screen according to an embodiment of this disclosure. This is a diagram showing an example of a display screen according to an embodiment of this disclosure. This is a diagram showing an example of a display screen according to an embodiment of this disclosure. This is a flowchart showing an example of the operation of the display control unit according to an embodiment of this disclosure.

[0010] Embodiments of this disclosure will be described below with reference to the drawings. In each drawing, the same or corresponding components are given the same reference numerals, and their descriptions will be omitted as appropriate.

[0011] (Configuration of the work machine 100) Figure 1 is a schematic diagram showing the configuration of the work machine 100 according to an embodiment of the present disclosure. The work machine 100 operates at a construction site and performs construction work on materials such as soil and sand. The work machine 100 according to an embodiment of the present disclosure is a hydraulic excavator. However, the work machine of the present disclosure may be other work machines such as a dump truck, wheel loader, or motor grader. The work machine 100 comprises a traveling body 110, a rotating body 120, a work machine 130, and an operator's cab 140.

[0012] The traveling body 110 supports the work machine 100 so that it can move. The traveling body 110 is equipped with two tracks 111a and 111b on the left and right sides, and two travel motors 112 for driving each track 111a and 111b. The slewing body 120 is supported by the traveling body 110 so that it can slewing around a pivot point.

[0013] The work implement 130 is driven by hydraulics. The work implement 130 is supported on the front of the slewing body 120 so as to be movable in the vertical direction. The operator's cab 140 is a space for the operator to sit in and operate the work machine 100. The operator's cab 140 is located on the left front of the slewing body 120. In this embodiment, as shown in Figure 1, the vertical, horizontal, and front-to-back directions are defined with respect to the slewing body 120. The part of the slewing body 120 to which the work implement 130 is attached is called the front. Furthermore, with respect to the slewing body 120, the part opposite the front is called the rear, the left side is called the left side, and the right side is called the right side.

[0014] (Configuration of the slewing body 120) The slewing body 120 includes an engine 121, a hydraulic pump 122, a control valve 123, a slewing motor 124, a fuel injector 125, an exhaust gas purification device 127, etc. The engine 121 is the prime mover that drives the hydraulic pump 122. The engine 121 is an example of a power source. In this embodiment, the engine 121 is a diesel engine.

[0015] The hydraulic pump 122 is a variable displacement pump driven by the engine 121. The hydraulic pump 122 supplies hydraulic fluid to a plurality of actuators via a control valve 123. The plurality of actuators include a boom cylinder 131C, an arm cylinder 132C, a bucket cylinder 133C, a travel motor 112, and a slewing motor 124. The control valve 123 controls the flow rate of hydraulic fluid supplied from the hydraulic pump 122.

[0016] The slewing motor 124 is driven by hydraulic fluid supplied from the hydraulic pump 122 via the control valve 123, causing the slewing body 120 to rotate. The fuel injector 125 injects fuel into the engine 121. The exhaust gas purification device 127 is a device that purifies nitrogen oxides (NOx) and particulate matter (PM (Particulate Matter)) contained in the exhaust gas of the engine 121. The exhaust gas purification device 127 includes a DPF device, an SCR device, etc., installed in the exhaust passage of the engine 121. The DPF unit is equipped with a DOC (Diesel Oxidation Catalyst) and a DPF (Diesel Particulate Filter). PM is collected by the DPF, and the nitrogen dioxide converted by the DOC oxidizes the collected PM downstream to carbon dioxide, thus removing the PM. The SCR unit is equipped with an SCR (Selective Catalytic Rumination) and an injector, pump, and tank for supplying urea solution to the exhaust upstream of the SCR, converting nitrogen oxides (NOx) into nitrogen molecules (N2) and water (H2O).

[0017] (Configuration of the work machine 130) The work machine 130 comprises a boom 131, an arm 132, a bucket 133, a boom cylinder 131C, an arm cylinder 132C, and a bucket cylinder 133C.

[0018] The base end of the boom 131 is attached to the slewing body 120 via a boom pin P1. The arm 132 connects the boom 131 and the bucket 133. The base end of the arm 132 is attached to the tip of the boom 131 via an arm pin P2. The bucket 133 has a blade for excavating soil and sand and a storage section for collecting the excavated soil. The base end of the bucket 133 is attached to the tip of the arm 132 via a bucket pin P3.

[0019] The boom cylinder 131C is a hydraulic cylinder for driving the boom 131. The base end of the boom cylinder 131C is attached to the slewing body 120. The tip end of the boom cylinder 131C is attached to the boom 131. The arm cylinder 132C is a hydraulic cylinder for driving the arm 132. The base end of the arm cylinder 132C is attached to the boom 131. The tip end of the arm cylinder 132C is attached to the arm 132. The bucket cylinder 133C is a hydraulic cylinder for driving the bucket 133. The base end of the bucket cylinder 133C is attached to the arm 132. The tip end of the bucket cylinder 133C is attached to the link member connected to the bucket 133.

[0020] (Example of camera configuration) The rotating body 120 is equipped with multiple cameras 126 (front camera 126A, left side camera 126B, rear camera 126C, and right side camera 126D) that capture images of the area around the work machine 100. The front camera 126A may be installed inside the operator's cab 140 or outside the operator's cab 140. The front camera 126A, left side camera 126B, rear camera 126C, and right side camera 126D are collectively referred to as multiple cameras 126. Alternatively, instead of multiple cameras 126, or in conjunction with multiple cameras 126, a single omnidirectional camera may be used. The display unit 206, described later, displays an overhead view image and a single-camera image based on the images (single-camera images) captured by each camera 126.

[0021] (Configuration of the driver's cab 140) Figure 2 shows the internal configuration of the driver's cab 140 according to the embodiment of the present disclosure. The driver's cab 140 is equipped with a driver's seat 142, an operating device 143, a display unit 206, and the like.

[0022] The operating device 143 is a device for driving the traveling body 110, the slewing body 120, and the work equipment 130 by manual operation by the operator. The operating device 143 comprises a left operating lever 143LO, a right operating lever 143RO, a left foot pedal 143LF, a right foot pedal 143RF, a left traveling lever 143LT, and a right traveling lever 143RT. The left operating lever 143LO is located on the left side of the driver's seat 142. The right operating lever 143RO is located on the right side of the driver's seat 142.

[0023] The left operating lever 143LO is an operating mechanism for, for example, the rotational movement of the slewing body 120 and the digging / dumping movement of the arm 132. The right operating lever 143RO is an operating mechanism for, for example, the digging / dumping movement of the bucket 133 and the raising / lowering movement of the boom 131.

[0024] The left foot pedal 143LF is located on the left side of the floor in front of the driver's seat 142. The right foot pedal 143RF is located on the right side of the floor in front of the driver's seat 142. The left travel lever 143LT is pivotally supported by the left foot pedal 143LF, and the tilt of the left travel lever 143LT is linked to the downward movement of the left foot pedal 143LF. The right travel lever 143RT is pivotally supported by the right foot pedal 143RF, and the tilt of the right travel lever 143RT is linked to the downward movement of the right foot pedal 143RF.

[0025] The left foot pedal 143LF and the left travel lever 143LT correspond to the rotational drive of the left track 111a of the vehicle body 110. The right foot pedal 143RF and the right travel lever 143RT correspond to the rotational drive of the right track 111b of the vehicle body 110.

[0026] The display unit 206 is, for example, a touch panel. The display unit 206 is an input / output device that integrally comprises an input unit 206I (Figure 3) that accepts input operations by touching the display screen 206S with a fingertip or an input pen, and an output device that displays an image on the display screen 206S. The display unit 206 is controlled by a display control unit 2012, which will be described later, and displays predetermined information related to the work machine 100 on the display screen 206S. The predetermined information related to the work machine 100 that the display screen 206S displays is, for example, state information representing information related to the operation of the work machine 100 (engine water temperature, hydraulic oil temperature, fuel level, urea solution level, etc.), image information representing the surroundings of the work machine 100 based on moving images, still images, etc. captured by multiple cameras 126. In this embodiment, "information" refers to the content that is transmitted, and includes image information that represents the content as an image, numerical information that represents the content as a numerical value, etc. Also, in this embodiment, the vertical and horizontal directions of the display screen 206S are the directions shown in Figure 2. In this embodiment, the display screen 206S has a rectangular shape with its longitudinal direction being vertical. However, for example, the display screen 206S may have a rectangular shape with its longitudinal direction being horizontal. The display unit 206 also receives input operations for the display screen 206S and outputs information indicating the operation content to the display control unit 2012, which will be described later.

[0027] (Configuration of Control System 200) Figure 3 is a schematic block diagram showing the hardware configuration of the control system 200 of the work machine 100 according to an embodiment of the present disclosure. The control system 200 includes a controller 201. The controller 201 includes an input / output unit 2011, a display control unit 2012, and a work machine control unit 2013. The control system 200 also includes a plurality of cameras 126, a display unit 206, a plurality of sensors 208, a plurality of actuators 209, etc. In this embodiment, the video signal is also referred to as image information. Furthermore, the controller 201, the display control unit 2012, or the display image generation unit 2063 are examples of the configuration of the "display control device" of the present disclosure.

[0028] The controller 201 and the display control unit 2012 can be configured using a computer such as a microcontroller or CPU (Central Processing Unit), and hardware such as peripheral circuits and peripheral devices for the computer. The controller 201 and the display control unit 2012 may also be configured using a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device). Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), FPGA (Field Programmable Gate Array), etc. Furthermore, some or all of the functions realized by the processor may be realized by the integrated circuit. The controller 201 is a functional block composed of a combination of hardware such as a computer and software such as a program executed by the computer, and includes an input / output unit 2011, a display control unit 2012, and a work machine control unit 2013. Note that the controller 201 may be configured as a single controller, or it may be configured as multiple controllers connected and cooperating via a predetermined communication line, for example.

[0029] The input / output unit 2011 inputs and outputs predetermined signals (input or output or input and output) to and from the multiple cameras 126, multiple sensors 208, and multiple actuators 209.

[0030] The work machine control unit 2013 acquires various data related to the hydraulic equipment that controls the operation of the work machine 130 using multiple sensors 208, and outputs multiple control signals to multiple actuators 209 related to the work machine 130 to control the hydraulic equipment according to the operation of the operating device 143. In other words, the work machine control unit 2013 controls the drive of the boom cylinder 131C, arm cylinder 132C, bucket cylinder 133C, travel motor 112, slewing motor 124, etc. The work machine control unit 2013 is an example of a vehicle body control unit that outputs control signals to drive the vehicle body of the work machine 100 using power supplied by a power source. The work machine control unit 2013 also transmits status information related to the operation of the work machine 100 to the display control unit 2012.

[0031] In this embodiment, the status information related to the operation of the work machine 100 includes information representing measurement results from multiple sensors 208 and numerical information obtained based on control commands to multiple actuators 209. In this embodiment, the status information related to the operation of the work machine 100 also includes numerical information (information expressed numerically or information representing numerical values), information representing the result of a relative comparison of numerical values ​​(for example, information such as large, medium, small, high, medium, low, strong, medium, weak, etc.). In this embodiment, the status information related to the operation of the work machine 100 transmitted by the work machine control unit 2013 to the display control unit 2012 includes information representing the hydraulic oil temperature. The multiple sensors 208 include a sensor for detecting hydraulic oil temperature, a sensor for detecting hydraulic pressure, a sensor for detecting the operating state of the operating device 143, a sensor for detecting the posture of the work machine 130, etc. The multiple actuators 209 include a hydraulic pump 122, a control valve 123, a boom cylinder 131C, an arm cylinder 132C, a bucket cylinder 133C, a travel motor 112, a slewing motor 124, etc.

[0032] Furthermore, the work machine control unit 2013 acquires various data related to the engine 121, etc., using a plurality of sensors 208, and outputs a plurality of control signals for controlling the engine 121, etc., to a plurality of actuators 209 related to the engine 121. The work machine control unit 2013 also transmits status information related to the operation of the work machine 100 to the display control unit 2012. In this embodiment, the status information related to the operation of the work machine 100 that the work machine control unit 2013 transmits to the display control unit 2012 includes information representing the engine coolant temperature, fuel level, and urea solution level. The plurality of sensors 208 include a sensor for detecting engine speed, a sensor for detecting engine coolant temperature, a sensor for detecting fuel level, a sensor for detecting urea solution level, etc. The plurality of actuators 209 include a fuel injector 125, a urea solution injector, a pump, etc.

[0033] Note that the connection configuration between the multiple sensors 208 and actuators 209 shown in Figure 3 and the work machine control unit 2013 is just one example, and the connection configuration is not limited to the example shown in Figure 3. Also, for example, status information related to the operation of the work machine 100, such as hydraulic oil temperature, engine coolant temperature, fuel level, and urea solution level, may be transmitted to the display control unit 2012 from the work machine control unit 2013, or it may be transmitted from the sensor 208 to the display control unit 2012 without going through the input / output unit 2011 or the work machine control unit 2013.

[0034] The display control unit 2012 controls the image display on the display screen 206S of the display unit 206. The display control unit 2012 also notifies the work machine control unit 2013, etc., of touch operations on the input unit 206I of the display unit 206. For example, the work machine control unit 2013 may be notified directly of touch operations on the input unit 206I of the display unit 206. In this embodiment, the display control unit 2012 comprises an information acquisition unit 2061, an input unit 2062, a display image generation unit 2063, and a display screen control unit 2064 as a functional block composed of a combination of hardware such as a computer and software such as a program executed by the computer.

[0035] The information acquisition unit 2061 acquires image information and also acquires status information related to the operation of the work machine 100 from the work machine control unit 2013. In this embodiment, the information acquisition unit 2061 acquires status information related to the operation of the work machine 100, including information representing engine coolant temperature, hydraulic oil temperature, fuel level, and urea solution level.

[0036] The input unit 2062 receives signals from the display unit 206 that represent input operations to the input unit 206I of the display unit 206, outputs them to the display image generation unit 2063, etc., and notifies the work machine control unit 2013, etc., of a signal indicating that a predetermined input operation has been performed. In other words, the input unit 2062 receives input from the operator to the display unit 206.

[0037] The display image generation unit 2063 generates an image to be displayed on the display screen 206S based on a predetermined set of display modes, using information acquired by the information acquisition unit 2061, signals input by the input unit 2062, etc. In this embodiment, "display screen" (display screen 206S) refers to the surface on which the image is projected. "Display mode" refers to the layout and format when projecting the image onto the display screen 206S, for example, what kind of information to display, where on the display screen 206S, and how to display it. The image projected onto the display screen 206S is determined by the display mode and the content of the information to be displayed. The display image generation unit 2063 also changes the display mode based on information acquired by the information acquisition unit 2061, signals input by the input unit 2062 (such as signals indicating input operations to the input unit 206I), etc.

[0038] The display screen control unit 2064 displays the image generated by the display image generation unit 2063 on the display screen 206S.

[0039] Here, with reference to Figures 4 to 7, an example of the display of an image generated by the display image generation unit 2063 on the display screen 206S will be described. Figures 4 and 5 show an example of the display using the same display mode D1. Display mode D1 is the display mode that is displayed on the display screen 206S by default when the work machine 100 is in an operational state. Note that Figure 4 shows an actual display example, and Figure 5 shows multiple display areas DA1 to DA3 included in the display screen 206S with thick dashed or chained lines added for explanatory purposes. In display mode D1, the display screen 206S includes display area DA1, display area DA2, and display area DA3. In display mode D1, display area DA1 and display area DA2 are adjacent, and display area DA2 and display area DA3 are adjacent, arranged from top to bottom in the order of display area DA1, display area DA2, and display area DA3.

[0040] As shown in Figures 4 and 5, in display mode D1, an overhead view image IM1 is displayed in display area DA1, a single-camera image IM2 is displayed in display area DA2, and status information QI1 related to the operation of the work machine 100 is displayed in display area DA3. The status information QI1 includes an engine coolant temperature gauge QI11, a hydraulic oil temperature gauge QI12, a fuel level gauge QI13, and a urea solution level gauge QI14. The engine coolant temperature gauge QI11 is image information representing the engine coolant temperature. The hydraulic oil temperature gauge QI12 is image information representing the hydraulic oil temperature. The fuel level gauge QI13 is image information representing the fuel level. The urea solution level gauge QI14 is image information representing the urea solution level. Gauges QI11 to QI14 represent each state by changing the length, display color, or presence or absence of display of a continuous or discrete rod shape according to the temperature or remaining amount. The overhead image IM1, single-camera image IM2, and state information QI1 (and state information QI2 shown in Figures 6 and 7) correspond to the "predetermined information relating to the work machine" in this disclosure. Furthermore, state information QI1 and state information QI2 correspond to the "state information relating to the operation of the work machine" in this disclosure. In addition, the overhead image IM1 corresponds to the "first information" in this disclosure, and the single-camera image IM2 corresponds to the "second information" in this disclosure. Furthermore, engine water temperature, hydraulic oil temperature, fuel level, and urea solution level (and image information representing them) are also information that quantitatively represents the state relating to the operation.

[0041] Figures 6 and 7 show an example of display using display mode D2, which is different from display mode D1. Display mode D2 is a display mode that is displayed on the display screen 206S (input unit 206I of the display unit 206) when a predetermined input operation is performed on the display screen 206S (input unit 206I of the display unit 206) of display mode D1. In this case, the predetermined input operation is a swipe, where a finger or the like is slid in the direction of the thick arrow S1, starting from point T1 within the display area DA3 on the display screen 206S of display mode D1, or a long tap (long press) within the display area DA3, as shown in Figure 5. A swipe is an input operation in which a finger or the like is brought into contact with the display screen 206S and slid while maintaining contact. In this case, the input operation to change from display mode D1 to display mode D2 is an input operation that instructs the display of enlarged status information QI1. In other words, when the predetermined input operation is performed, a portion of the status information QI1 related to the operation of the work machine 100 displayed in the display area DA3 is enlarged and displayed. Furthermore, when a predetermined input operation is performed, the display mode of the status information QI1 related to the operation of the work machine 100 may be changed and enlarged. In this case, the input operation may be either a swipe or a long tap. The starting point T1 can be anywhere within the display area DA3 that shows the status information QI1 related to the operation of the work machine 100, and the status information QI1 is enlarged by swiping the screen upward from the starting point on the display screen 206S. However, the starting point T1 is preferably near the center of the display area DA3.

[0042] Figure 6 shows an actual display example, and Figure 7 shows the multiple display areas DA1 and DA4 included in the display screen 206S, with thick dashed or chained lines added for explanatory purposes. In display mode D2, the display screen 206S includes display area DA1 and display area DA4. Display area DA1 is the same area as in display mode D1 (same position and size). For example, display area DA1 has the same position and size as displayed in display mode D1 and display mode D2, but is not limited to that. For example, display area DA1 in display mode D2 may be displayed enlarged or reduced compared to display area DA1 in display mode D1. Display area DA4 is an area that includes at least display area DA2 (all or part of it) and display area DA3 (all or part of it) from display mode D1. In this example, display area DA4 is the combined area of ​​display area DA2 and display area DA3 (an area including all of display area DA2 and all of display area DA3). Display areas DA1, DA2, DA3, and DA4 correspond to the "first area," "second area," "third area," and "fourth area" of this disclosure, respectively. For example, a portion of the status information QI1 related to the operation of the work machine 100 displayed in display area DA3 may be displayed in display area DA2. In that case, the portion of the status information QI1 that is not enlarged and displayed in display area DA2 is enlarged and displayed in display area DA3. That is, the engine coolant temperature gauge QI11 and the hydraulic oil temperature gauge QI12 displayed in display area DA3 are displayed in place of the single camera image IM2 displayed in display area DA2. At this time, the engine coolant temperature gauge QI11 and the hydraulic oil temperature gauge QI12 are displayed in the same manner as the engine coolant temperature gauge QI21 and the hydraulic oil temperature gauge QI22, which will be described later. In addition, the fuel level gauge QI13 and the urea solution level gauge QI14 displayed in the display area DA3 are displayed enlarged within the display area DA3.

[0043] As shown in FIGS. 6 and 7, in display mode D2, an aerial image IM1 is displayed in display area DA1, and state information QI2 related to the operation of the working machine 100 is displayed in display area DA4. The state information QI2 includes a gauge QI21 for the engine coolant water temperature, a gauge QI22 for the hydraulic oil temperature, a gauge QI23 for the remaining fuel amount, and a gauge QI24 for the remaining urea water amount. Further, the display area DA4 includes a button HB1.

[0044] The sizes (the area of the display region of the pixels representing the gauges, the outer circumference, etc.) of the gauges QI21 to QI24 are larger than those of the gauges QI11 to QI14 in display mode D1. Also, as shown in FIG. 7, the gauge QI21 for the engine coolant water temperature and the gauge QI22 for the hydraulic oil temperature are in a display mode using a needle M1 and an arc-shaped scale M2 to represent each quantity. On the other hand, the display modes of the gauge QI23 for the remaining fuel amount and the gauge QI24 for the remaining urea water amount are the same as those of the gauge QI13 for the remaining fuel amount and the gauge QI14 for the remaining urea water amount in display mode D1. However, the gauge QI23 for the remaining fuel amount and the gauge QI24 for the remaining urea water amount in display mode D2 are displayed in an enlarged manner compared to the gauge QI13 for the remaining fuel amount and the gauge QI14 for the remaining urea water amount in display mode D1. When the state information QI1 includes a plurality of types of information, the state information QI2 may include some of the types of information included in the state information QI1 without enlargement.

[0045] Also, when a predetermined input operation is performed on the display screen 206S (input unit 206I of the display unit 206) in display mode D2, the display screen 206S in display mode D1 shown in FIGS. 4 and 5 is displayed. The predetermined input operation in this case is, as shown in FIG. 7, a swipe in which a finger or the like is slid in the direction of the thick arrow S2 starting from the point T2 within the display area DA4 on the display screen 206S in display mode D2, or a tap on the button HB1. In this case, the input operation for changing display mode D2 to display mode D1 is an input operation instructing to return the display of the state information QI2 to the display of the state information QI1. That is, it is an input operation to return from the enlarged display of the state information QI2 to the normal display of the state information QI1. The predetermined input operation may be either a swipe or a tap on the button HB1.

[0046] (Example of operation of display control unit 2012) Referring to the flowchart in Figure 8, an example of the processing flow when switching between display screen 206S of display mode D1 shown in Figures 4 and 5 and display screen 206S of display mode D2 shown in Figures 6 and 7 will be explained. The processing shown in Figure 8 is repeatedly executed by the display control unit 2012 at predetermined time intervals.

[0047] When the process shown in Figure 8 is started, first, the information acquisition unit 2061 of the display control unit 2012 acquires status information related to the operation of the work machine 100 from the work machine control unit 2013, etc. (step S101). In this embodiment, the status information related to the operation of the work machine 100 includes engine coolant temperature, hydraulic oil temperature, fuel level, and urea solution level. The information acquisition unit 2061 also acquires image information (video signal) from the camera 126 (step S102). Next, the display image generation unit 2063 updates the display content of the display area DA1 (step S103). In step S103, the latest overhead image IM1 is displayed in the display area DA1.

[0048] Next, the display image generation unit 2063 determines whether the current display mode is the display mode D1 (step S104). If the current display mode is the display mode D1 (step S104: YES), the display image generation unit 2063 determines whether there has been an input operation instructing an enlarged display of the state information QI1 (step S105). If there has been an input operation instructing an enlarged display of the state information QI1 (step S105: YES), the display image generation unit 2063 switches the display mode from the display mode D1 to the display mode D2 (step S106). Note that when switching the display mode in step S106, for example, the display mode may be switched so that the display mode gradually switches using an animation. Next, the display image generation unit 2063 updates the display content of the display area DA4 (step S107), and ends the process shown in FIG. 8. In step S107, the latest state information QI2 is displayed in the display area DA4. That is, in step S107, the engine coolant temperature gauge QI11 and the operating oil temperature gauge QI12 in the display mode D1 are displayed as the engine coolant temperature gauge QI21 and the operating oil temperature gauge QI22 with different enlarged displays and display modes. Also, in the determination in step S105, if there has been no input operation instructing an enlarged display of the state information QI1 (step S105: NO), the display image generation unit 2063 updates the display content of the display area DA2 (step S110), and then updates the display content of the display area DA3 (step S111), and ends the process shown in FIG. 8. In step S110, the latest single-camera image IM2 is displayed in the display area DA2, and in step S111, the latest state information QI1 is displayed in the display area DA3.

[0049] Furthermore, if the determination in step S104 indicates that the current display mode is not display mode D1 (step S104: NO), the display image generation unit 2063 determines whether or not there has been an input operation to return the enlarged display, which is the display of state information QI2, to the normal display, which is the display of state information QI1 (step S108). In other words, it determines whether or not there has been an input operation to return the display of the enlarged state information QI2 to the normal display, which is the display of a portion of the gauge of state information QI1, in display mode D2. If there has been an input operation to return the enlarged display of state information QI2 to the normal display of state information QI1 (step S108: YES), the display image generation unit 2063 switches the display mode from display mode D2 to display mode D1 (step S109). Next, the display image generation unit 2063 updates the display content of display area DA2 (step S110), then updates the display content of display area DA3 (step S111), and then terminates the process shown in Figure 8. In step S110, the latest single-camera image IM2 is displayed in display area DA2, and in step S111, the latest status information QI1 is displayed in display area DA3. Also, if, in the determination in step S108, there is no input operation to return the enlarged display of status information QI2 to the normal display of information QI1 (step S108: NO), the display image generation unit 2063 updates the display content of display area DA4 (step S107) and terminates the process shown in Figure 8. In step S107, the latest status information QI2 is displayed in display area DA4.

[0050] For example, through the above processing, in this embodiment, the display screen 206S of display mode D1 shown in Figures 4 and 5 and the display screen 206S of display mode D2 shown in Figures 6 and 7 are switched and displayed. Note that the processing flow shown in Figure 8 is just one example, and the processing flow is not limited to the case shown in Figure 8. For example, processing such as updating the display content may be executed in parallel on an object-by-object basis.

[0051] (Function and Effects) As described above, the display control unit 2012 (display image generation unit 2063) (display control device) of this embodiment displays predetermined information related to the work machine 100 on the display screen 206S. The display screen 206S includes a plurality of display areas DA1 to DA3 for displaying predetermined information, and displays status information QI1 related to the operation of the work machine 100 in the display area. When a predetermined input operation is performed, the status information related to the operation of the work machine 100 is enlarged and displayed. With this configuration, images representing predetermined information can be displayed in an easy-to-view manner.

[0052] Furthermore, in the configuration of this embodiment, the display screen 206S is a touch panel, and a predetermined input operation includes at least one of swiping the display portion of the status information QI1 related to the operation of the work machine 100 and long-tapping the display portion of the status information QI related to the operation of the work machine 100. With this configuration, the input operation that instructs the enlarged display of the status information QI1 can be an intuitive input operation related to the area to be enlarged.

[0053] Furthermore, in the configuration of this embodiment, the state information QI1 related to the operation of the work machine 100 includes image information QI11 representing the engine coolant temperature of the work machine 100, and image information QI12 representing the hydraulic oil temperature. This configuration makes it easier to check the engine coolant temperature and hydraulic oil temperature.

[0054] Furthermore, in the configuration of this embodiment, when a predetermined input operation is performed, the display mode of the status information related to the operation of the work machine 100 is changed and enlarged. With this configuration, the display mode can be appropriately changed according to the size of the image.

[0055] Furthermore, in the configuration of this embodiment, when a predetermined input operation is performed, status information related to the operation of multiple work machines is displayed in an enlarged view. This configuration allows for the display of multiple status information in an enlarged view.

[0056] Furthermore, in the configuration of this embodiment, when a predetermined input operation is performed, for example, state information QI2 related to the operation of the work machine 100 is displayed in the lower half of the display screen 206S. That is, in this embodiment, state information QI2, which is an enlarged version of state information QI1 before the predetermined input operation is performed, is displayed in an area other than the upper half of the display screen 206S. With this configuration, the information displayed in the upper half of the display screen 206S can be displayed in the same state, for example, before and after the predetermined input operation.

[0057] Furthermore, in the configuration of this embodiment, the display screen 206S is a display area for predetermined information related to the work machine 100, and includes display area DA1 (first area), display area DA2 (second area), and display area DA3 (third area). The display control unit 2012 displays an overhead image IM1 (first information) in display area DA1 (first area), a single-camera image IM2 (second information) in display area DA2 (second area), and status information QI1 related to the operation of the work machine 100 in display area DA3 (third area). When a predetermined input operation (an input operation to instruct enlarged display of the status information) is performed, the display control unit 2012 displays at least a part of the status information enlarged in display area DA4 (fourth area) of the display screen 206S, which includes at least display area DA2 (second area) among display areas DA2 (second area) and display area DA3 (third area). With this configuration, images representing predetermined information can be displayed in an easy-to-view manner. Furthermore, even when displaying the status information QI1 in an enlarged view, the overhead image IM1 (first information) can always be displayed in the display area DA1 (first area). For example, when an input operation to instruct enlarged display is performed, the single-camera image IM2 displayed in the display area DA2 is replaced with a part of the status information QI1 displayed in the display area DA3. At that time, the status information QI1 displayed in the display area DA2 is displayed in an enlarged view with a changed display mode.

[0058] Furthermore, display area DA2 (second area) is adjacent to display area DA1 (first area), and display area DA3 (third area) is adjacent to display area DA2 (second area). With this configuration, by continuously changing the line of sight, the overhead image IM1 (first information), single-camera image IM2 (second information), and status information QI1 can be viewed.

[0059] Furthermore, the display screen 206S is a display unit 206, and a predetermined input operation (an input operation to instruct the enlarged display of state information QI1) includes at least one of a swipe originating from within the display area DA3 (third area) and a long tap within the display area DA3 (third area). With this configuration, the input operation to instruct the enlarged display of state information QI1 can be an intuitive input operation related to the area to be enlarged.

[0060] Furthermore, display area DA4 (the fourth area) is the combined area of ​​display area DA2 (the second area) and display area DA3 (the third area) (the area corresponding to the entirety of display area DA2 and display area DA3). With this configuration, the size of display area DA4 (the fourth area) can be maximized when display area DA4 (the fourth area) is made to include only display area DA2 (the second area) and display area DA3 (the third area).

[0061] In this embodiment, the state information QI1 and QI2 include image information representing the engine coolant temperature of the work machine 100, image information representing the hydraulic oil temperature, image information representing the fuel level, and image information representing the urea solution level.

[0062] Furthermore, the display area DA4 (fourth area) includes a button HB1 for returning the display screen 206S to the display mode D1 before the state information QI1 was enlarged and displayed. With this configuration, for example, by making the image (icon) contained in button HB1 an image related to the display mode D1 before it was enlarged and displayed, the meaning of button HB1 can be intuitively grasped. In this embodiment, display mode D1 is the default home screen, and the image contained in button HB1 is an image representing "Home".

[0063] Furthermore, when the status information QI2 is displayed in an enlarged state, the display image generation unit 2063 returns the display screen 206S to the display mode D1 before the status information was enlarged when the button HB1 is tapped or when a swipe is made starting from the display area DA4 (within the fourth area). With this configuration, the input operation to instruct the normal display of the status information can be made into an intuitive input operation.

[0064] Furthermore, at least a portion of the display modes of the status information QI1 and QI2 differs depending on whether they are displayed in display area DA3 (third area) or display area DA4 (fourth area). This configuration allows the display mode of the status information QI2 to be appropriately changed according to the size of the image.

[0065] Furthermore, the display control method of this embodiment is a display control method that displays predetermined information relating to the work machine 100 on a predetermined display screen 206S, the display screen 206S including a display area DA1 (first area), a display area DA2 (second area), and a display area DA3 (third area), which are display areas for predetermined information, the first information is displayed in the display area DA1 (first area), the second information different from the first information is displayed in the display area DA2 (second area), and state information QI1 relating to the operation of the work machine 100 is displayed in the display area DA3 (third area), and when a predetermined input operation is performed, at least a part of the state information relating to the operation of the work machine 100 is displayed enlarged in the display area DA4 (fourth area) of the display screen 206S which includes at least the display area DA2 (second area) of the display area DA2 (second area) and the display area DA3 (third area).

[0066] (Modifications, etc.) Although embodiments of this invention have been described above with reference to the drawings, the specific configuration is not limited to the above embodiments, and design changes, etc., that do not depart from the gist of this invention are also included.

[0067] For example, the display unit 206 may use an input / output device equipped with a monitor without touch input functionality, such as an LCD (Liquid Crystal Display) or an organic electroluminescent display, and physical buttons, instead of a touch panel. Alternatively, the display screen 206S may be horizontally elongated, with display areas DA1 to DA3 arranged horizontally. Display area DA4 (fourth area) may be a pop-up window superimposed on display area DA2 (second area) and display area DA3 (third area), or on display area DA2 (second area). The prime mover may be an electric motor instead of an engine 121, or a combination of an engine and an electric motor. The operating device 143, display unit 206, etc., may be located in a remote control room for remotely operating the work machine 100, instead of (or in addition to) the operator's cab 140 of the work machine 100. Furthermore, the operating device 143, display unit 206, etc., are not limited to being mounted on the work machine 100, but may be provided separately from the work machine 100, for example. In this case, the operating device 143, display unit 206, etc., are configured to communicate with the work machine 100, enabling remote operation of the work machine 100. In the above embodiment, the overhead image IM1 is displayed in display area DA1 and the single-camera image IM2 is displayed in display area DA2, but the information to be displayed in each area is not limited to this example. For example, one enlarged overhead image IM1 may be displayed in the areas corresponding to display areas DA1 and DA2, and when displaying status information QI2 in display area DA4, the normal-sized overhead image IM1 may be displayed in display area DA1. Also, status information QI2 may be displayed in display area DA2 instead of the single-camera image IM2.

[0068] Furthermore, some or all of the program executed by the computer in the above embodiment can be distributed via a computer-readable recording medium or communication line.

[0069] (Note) The display control unit 2012 (display control device) described in the embodiment can be understood as follows.

[0070] (1) A display control device according to a first aspect of the present disclosure is a display control device that displays predetermined information relating to a work machine on a display screen, wherein the display screen includes a plurality of display areas for displaying the predetermined information, and the display areas display state information relating to the operation of the work machine, and when a predetermined input operation is performed, the state information relating to the operation of the work machine is displayed in an enlarged view.

[0071] (2) A display control device according to a second aspect of the present disclosure is the display control device of (1), wherein the display screen is a touch panel, and the predetermined input operation includes at least one of swiping the display portion of the status information relating to the operation of the work machine and long-tapping the display portion of the status information relating to the operation of the work machine.

[0072] (3) A display control device according to a third aspect of the present disclosure is the display control device according to (1) to (2), wherein the state information relating to the operation of the work machine includes image information representing the engine coolant temperature of the work machine and image information representing the hydraulic oil temperature.

[0073] (4) The display control device according to the fourth aspect of the present disclosure is the display control device according to (1) to (3), wherein when the predetermined input operation is performed, the display mode of the status information relating to the operation of the work machine is changed and enlarged.

[0074] (5) A display control device according to a fifth aspect of the present disclosure is the display control device according to (1) to (4), wherein the display screen includes a first area, a second area and a third area which are display areas for predetermined information, the first area displays first information, the second area displays second information different from the first information, the third area displays state information relating to the operation of the work machine, and when a predetermined input operation is performed, at least a part of the state information is displayed enlarged in a fourth area of ​​the display screen which includes at least the second area among the second and third areas.

[0075] (6) A display control device according to a sixth aspect of the present disclosure is the display control device according to (1) to (5), wherein the second region is adjacent to the first region and the third region is adjacent to the second region.

[0076] (7) A display control device according to a seventh aspect of the present disclosure is the display control device according to (1) to (6), wherein the display screen is a touch panel, and the predetermined input operation includes at least one of a swipe originating from within the third area and a long tap within the third area.

[0077] (8) The eighth aspect of the present disclosure is the display control device according to (1) to (7), wherein the fourth region is the region comprising the second region and the third region.

[0078] (9) A display control device according to the ninth aspect of the present disclosure is the display control device according to (1) to (8), wherein the state information includes image information representing the engine coolant temperature of the work machine, image information representing the hydraulic oil temperature, image information representing the fuel level, and image information representing the urea solution level.

[0079] (10) A display control device according to a tenth aspect of the present disclosure is the display control device according to (1) to (9), wherein the fourth area includes a button for returning the display screen to the display state before the state information was enlarged and displayed.

[0080] (11) A display control device according to the eleventh aspect of the present disclosure is the display control device according to (1) to (10), wherein when the state information is displayed in an enlarged state, the display screen returns to the display state before the state information was displayed in an enlarged state when the button is tapped.

[0081] (12) A display control device according to a twelfth aspect of the present disclosure is a display control device according to (1) to (11), wherein at least a part of the display mode of the state information differs depending on whether it is displayed in the third area or the fourth area.

[0082] (13) A display control device according to a thirteenth aspect of the present disclosure is a display control device according to (1) to (12), wherein when the predetermined input operation is performed, the display enlarges and displays status information relating to the operation of the plurality of work machines.

[0083] (14) A display control device according to a fourteenth aspect of the present disclosure is a display control device according to (1) to (13), wherein when the predetermined input operation is performed, status information relating to the operation of the work machine is displayed on the lower half of the display screen.

[0084] According to the display control device and display control method of this disclosure, an image representing predetermined information can be displayed in an easily viewable manner.

[0085] 100...Work machine 110...Traveling body 120...Slewing body 126, 126A-D...Camera 130...Work machine 140...Operator's cab 200...Control system 201...Controller 2013...Work machine control unit 2012...Display control unit 206...Display unit 206S...Display screen DA1-4...Display area

Claims

1. A display control device for displaying predetermined information relating to a work machine on a display screen, wherein the display screen includes a plurality of display areas for displaying the predetermined information, the display areas for displaying status information relating to the operation of the work machine, and when a predetermined input operation is performed, the display control device enlarges and displays the status information relating to the operation of the work machine.

2. The display control device according to claim 1, wherein the display screen is a touch panel, and the predetermined input operation includes at least one of swiping the display portion of the status information relating to the operation of the work machine and long-tapping the display portion of the status information relating to the operation of the work machine.

3. The display control device according to claim 2, wherein the status information relating to the operation of the work machine includes image information representing the engine coolant temperature of the work machine and image information representing the hydraulic oil temperature.

4. The display control device according to claim 1 or 2, wherein when the predetermined input operation is performed, the display mode of the status information related to the operation of the work machine is changed and enlarged for display.

5. The display control device according to claim 1, wherein the display screen includes a first area, a second area, and a third area, which are display areas for the predetermined information, the first area displays the first information, the second area displays second information different from the first information, the third area displays status information related to the operation of the work machine, and when the predetermined input operation is performed, at least a portion of the status information is enlarged and displayed in a fourth area of ​​the display screen, which includes at least the second area among the second and third areas.

6. The display control device according to claim 5, wherein the second region is adjacent to the first region, and the third region is adjacent to the second region.

7. The display control device according to claim 6, wherein the predetermined input operation includes at least one of a swipe originating within the third area and a long tap within the third area.

8. The display control device according to claim 7, wherein the fourth region is the combined region of the second region and the third region.

9. The display control device according to claim 8, wherein the status information includes image information representing the engine coolant temperature of the work machine, image information representing the hydraulic oil temperature, image information representing the fuel level, and image information representing the urea solution level.

10. The display control device according to claim 9, wherein the fourth area includes a button for returning the display screen to the display configuration before the state information was enlarged and displayed.

11. The display control device according to claim 10, wherein when the state information is displayed in an enlarged state, the display screen returns to the display state before the state information was displayed in an enlarged state when the button is tapped.

12. The display control device according to claim 5 or claim 11, wherein at least a portion of the display mode of the state information differs depending on whether it is displayed in the third area or in the fourth area.

13. The display control device according to claim 1, which displays enlarged status information relating to the operation of a plurality of work machines when the predetermined input operation is performed.

14. The display control device according to claim 1, wherein when the predetermined input operation is performed, status information relating to the operation of the work machine is displayed on the lower half of the display screen.

15. A display control method for displaying predetermined information relating to a work machine on a display screen, wherein the display screen includes a plurality of display areas for displaying the predetermined information, the display areas for displaying status information relating to the operation of the work machine, and when a predetermined input operation is performed, the display control method for displaying the status information relating to the operation of the work machine in an enlarged view.

Images