Control device, work machine, and control method

Abstract

To improve the operability of a control device.SOLUTION: A control device for a touch panel monitor, including a display signal generation unit configured to generate a display signal for display on the touch panel monitor including a plurality of display screens, a determination unit configured to accept a touch operation for each of the plurality of display screens and determine whether the accepted touch operation is an acceptable touch operation defined for each display screen type, and a control unit configured to control the display screen according to the accepted touch operation when the accepted touch operation is the acceptable touch operation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a control device, a work machine, and a control method. [Background technology]

[0002] Patent Document 1 discloses an input control method for a touch panel monitor for a work machine that enables a monitor screen display while preventing erroneous input operations on the touch panel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-202841 A Summary of the Invention [Problem to be solved by the invention]

[0004] There are known control devices that are attached to the driver's seat of a work machine or the like and that can assist the operator in driving by displaying the status of the work machine. Some control devices are capable of simultaneously displaying multiple types of display screens on a single monitor. Types of display screens include, for example, a side view that shows the body of the work machine as seen from the side, a top view that shows the body of the work machine as seen from above, a front view that shows the body of the work machine as seen from the front, as well as 3D screens.

[0005] There is a need to improve the operability of such control devices.

[0006] In view of the above-mentioned problems, an object of the present disclosure is to improve the operability of the above-mentioned control device. [Means for solving the problem]

[0007] According to one aspect of the present disclosure, the control device is a control device for a touch panel monitor and includes: a display signal generation unit that generates a display signal to be displayed on the touch panel monitor, which includes a plurality of display screens; a determination unit that receives a touch operation for each of the plurality of display screens and determines whether the received touch operation is an acceptable touch operation defined for each type of display screen; and a control unit that, when the received touch operation is an acceptable touch operation, performs control on the display screen in accordance with the received touch operation. [Effects of the Invention]

[0008] According to the above aspect, the operability of the control device can be improved. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing the overall configuration of a work machine according to a first embodiment. [Figure 2] 1 is a diagram showing the configuration of a cab of a work machine according to a first embodiment. [Figure 3] FIG. 2 is a diagram illustrating a functional configuration of a control device according to the first embodiment. [Figure 4] FIG. 3 is a diagram showing a processing flow of the control device according to the first embodiment. [Figure 5] FIG. 4 is a diagram showing an example of control by a control unit according to the first embodiment, and is a diagram showing an example of pinch-out in a side view. [Figure 6] FIG. 4 is a diagram showing an example of control by a control unit according to the first embodiment, and is a diagram showing an example of pinch-out in a top view. [Figure 7] FIG. 4 is a diagram showing an example of control by a control unit according to the first embodiment, and is a diagram showing an example of pinch-in on a side view. [Figure 8] FIG. 4 is a diagram showing an example of control by a control unit according to the first embodiment, and is a diagram showing an example of rotation in a top view. [Figure 9] FIG. 10 is a diagram showing an example of control by a control unit according to the first embodiment, and is a diagram showing an example of a one-finger swipe in a side view. [Figure 10] FIG. 10 is a diagram showing an example of control by a control unit according to the first embodiment, and is a diagram showing an example of a one-finger swipe in a top view. [Figure 11] 10A and 10B are diagrams showing an example of control by a control unit according to the first embodiment, and are diagrams showing an example of simultaneous operation in a side view and a top view. [Figure 12] FIG. 10 is a diagram showing an example of control by a control unit according to the first embodiment, and is a diagram showing an example of a two-finger swipe on a 3D screen. [Figure 13] FIG. 2 is a diagram showing an example of control by a control unit according to the first embodiment, and is a diagram for explaining in detail examples of one-finger swipes and two-finger swipes on a 3D screen. DETAILED DESCRIPTION OF THE INVENTION

[0010] First Embodiment

[0011] (Structure of Work Machinery) FIG. 1 is a diagram showing the structure of a work machine according to a first embodiment. The work machine 1, which is a hydraulic excavator, excavates earth and sand and levels the ground at a work site or the like. As shown in Fig. 1, the work machine 1, which is a hydraulic excavator, comprises a lower traveling body 11 for traveling, and an upper rotating body 12 that is installed on top of the lower traveling body 11 and can rotate around a vertical axis. The upper rotating body 12 is also provided with a driver's cab 12A, a work implement 12B, and two GNSS antennas N1 and N2.

[0012] The undercarriage 11 has a left crawler track CL and a right crawler track CR. The work machine 1 moves forward, turns, and reverses by rotation of the left crawler track CL and the right crawler track CR.

[0013] The operator's cab 12A is where the operator of the work machine 1 gets in and operates and pilots the machine. The operator's cab 12A is installed, for example, on the left side of the front end of the upper rotating body 12. A control device 2 is mounted in the operator's cab 12A of the work machine 1.

[0014] The work implement 12B consists of a boom BM, an arm AR, and a bucket BK. The boom BM is attached to the front end of the upper rotating body 12. An arm AR is attached to the boom BM. A bucket BK is attached to the arm AR. A boom cylinder SL1 is attached between the upper rotating body 12 and the boom BM. By driving the boom cylinder SL1, the boom BM can be moved relative to the upper rotating body 12. An arm cylinder SL2 is attached between the boom BM and the arm AR. By driving the arm cylinder SL2, the arm AR can be moved relative to the boom BM. A bucket cylinder SL3 is attached between the arm AR and the bucket BK. By driving the bucket cylinder SL3, the bucket BK can be moved relative to the arm AR. The above-described upper rotating body 12, boom BM, arm AR, and bucket BK that the work machine 1, which is a hydraulic excavator, is equipped with are one example of the movable parts of the work machine 1.

[0015] Although the work machine 1 according to this embodiment has been described as having the above-described configuration, in other embodiments, the work machine 1 does not necessarily have to have all of the above-described configuration. For example, the work machine 1 according to other embodiments does not have to be equipped with GNSS antennas N1, N2.

[0016] (Driver's cab configuration) FIG. 2 is a diagram showing the configuration of the operator's cab of the work machine according to the first embodiment.

[0017] As shown in FIG. 2, the operator's cab 12A is provided with operation levers L1 and L2, foot pedals F1 and F2, and travel levers R1 and R2. The operating levers L1 and L2 are disposed on the left and right sides of the seat ST in the operator's cab 12A. The foot pedals F1 and F2 are disposed on the floor in front of the seat ST in the operator's cab 12A.

[0018] The operating lever L1 located on the left side facing the front of the cab is an operating mechanism for performing the swing operation of the upper rotating body 12 and the excavation / dumping operation of the arm AR. The operating lever L2 located on the right side facing the front of the cab is an operating mechanism for performing the excavation / dumping operation of the bucket BK and the raising / lowering operation of the boom BM.

[0019] Additionally, the travel levers R1, R2 are operating mechanisms for controlling the operation of the undercarriage 11, i.e., controlling the travel of the work machine 1. The travel lever R1, located on the left side as one faces the front of the cab, corresponds to the rotational drive of the left crawler CL of the undercarriage 11. The travel lever R2, located on the right side as one faces the front of the cab, corresponds to the rotational drive of the right crawler CR of the undercarriage 11. Note that the foot pedals F1, F2 are linked to the travel levers R1, R2, respectively, and travel control can also be performed using the foot pedals F1, F2.

[0020] The control device 2 is installed on the front right side when facing the front of the cab. The functions of the control device 2 will be described in detail below. Note that in other embodiments, the control device 2 may be installed on the front left side when facing the front of the cab, etc.

[0021] (Functional configuration of the control device) FIG. 3 is a diagram illustrating a functional configuration of the control device according to the first embodiment. The control device 2 is a control device for a touch panel monitor, and as shown in Fig. 3, includes a CPU 20, a memory 21, a monitor 22, a touch sensor 23, a communication interface 24, and a storage 25. The CPU 20 may be any type of device such as an FPGA, a GPU, or the like.

[0022] The CPU 20 is a processor that controls the overall operation of the control device 2. The various functions of the CPU 20 will be described later.

[0023] The memory 21 is a so-called main storage device, and stores instructions and data necessary for the CPU 20 to operate based on a program.

[0024] The monitor 22 is a display panel capable of visually displaying information, and is, for example, a liquid crystal display or an organic EL display.

[0025] The touch sensor 23 is formed integrally with the monitor 22 and is an input device that can specify a position on an image displayed on the monitor 22.

[0026] The communication interface 24 is a communication interface for communicating with an external server.

[0027] The storage 25 is a so-called auxiliary storage device, such as a hard disk drive (HDD) or a solid state drive (SSD).

[0028] Next, a detailed description will be given of the functions of the CPU 20. The CPU 20 operates based on a predetermined program to fulfill the functions of a display signal generation unit 200, a determination unit 201, a control unit 202, a display screen identification unit 203, and a touch operation type identification unit 204. The predetermined program may be for realizing some of the functions to be performed by the control device 2. For example, the program may be combined with other programs already stored in the storage 25 or other programs implemented in other devices to perform the functions. In other embodiments, the control device 2 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor may be realized by the integrated circuit.

[0029] The display signal generation unit 200 generates display signals for displaying multiple display screens on a single monitor 22. The multiple display screens each display the status of the work machine 1 in a different manner as described below. The monitor 22 displays the display screen based on the signal generated by the display signal generation unit 200. The status of the work machine 1 is, for example, the operating state of the movable parts of the work implement 12B and the positional relationship between the work machine 1 and the surrounding terrain (design surface). As the status of the work machine 1 changes, the display screen is updated and changed accordingly. The operator of the work machine 1 operates and pilots the work machine 1 while viewing the various display screens displayed on the monitor 22.

[0030] The types of display screens that the display signal generating unit 200 according to this embodiment displays are as follows. (1) Side view: A display screen showing the body of the work machine 1 and the surrounding terrain as seen from the side. (2) Top view: A display screen showing the body of the work machine 1 and the surrounding terrain as viewed from above. (3) Front view: This is a display screen showing the bucket BK of the work machine 1 and the surrounding terrain as viewed from the front. (4) 3D screen: This is a display screen that displays the body of the work machine 1 and the surrounding terrain as a three-dimensional image.

[0031] The determination unit 201 receives a touch operation for each of the plurality of display screens, and determines whether or not the received touch operation is an acceptable touch operation defined for each type of display screen.

[0032] The types of touch operations that can be accepted by the control device according to this embodiment are as follows. (a) One-finger swipe: An operation of moving one finger on the surface of the monitor 22. (b) Two-finger swipe: An operation of moving two fingers together on the surface of the monitor 22. (c) Pinch-in: An operation of moving two fingers closer together on the surface of the monitor 22. (d) Pinch out: This is an operation of moving two lines on the surface of the monitor 22 so as to widen the space between them. (e) Rotate: This is an operation of rotating two fingers on the surface of the monitor 22 while keeping the distance between them constant. The two-finger swipe is an example of a swipe operation based on touching at least two points. In other embodiments, any operation similar to the two-finger swipe may be used, including, for example, using a touch pen. The same applies to other touch operations.

[0033] In this embodiment, the acceptable touch operations defined for each type of display screen are as follows. (a) Acceptable touch operations on the side view: (a) one-finger swipe, (c) pinch in, (d) pinch out (a) Acceptable touch operations on the top view: (a) one-finger swipe, (c) pinch in, (d) pinch out, (e) rotate (c) Acceptable touch operations on the front view: (a) one-finger swipe, (c) pinch in, (d) pinch out (d) Acceptable touch operations on the 3D screen: (a) one-finger swipe, (b) two-finger swipe, (c) pinch in, (d) pinch out The correspondence between the types of display screens and the touch sensors that can accept input, as defined in the above (A) to (D), is recorded in advance in, for example, the memory 21 or the storage 25.

[0034] When a touch operation received on a display area of ​​a certain display screen is an acceptable touch operation for that display screen, the control unit 202 performs control according to the received touch operation on that display screen.

[0035] In this embodiment, the control in response to a touch operation is, for example, as follows: (A) Control when a one-finger swipe is received: The image displayed on the display screen is slid. In a 3D screen, the camera position in three-dimensional space is slid on a predetermined plane. (B) Control when a two-finger swipe is accepted: When a two-finger swipe is accepted on a 3D screen, the camera position in three-dimensional space is rotated around a specified point. (C) Control when pinch-in is received: The image displayed on the display screen is reduced in size at the center of the screen. (D) Control when pinch out is received: Enlarge the image displayed on the display screen at the center of the screen. (E) Control when rotation is accepted: Rotate the image (top view) displayed on the display screen around the center of the vehicle body.

[0036] When the display screen identification unit 203 receives a touch operation from the operator via the touch sensor 23, it identifies the type of the display screen in the display area where the touch operation is received.

[0037] The touch operation type identification unit 204 identifies the type of touch operation received via the touch sensor 23 .

[0038] (Processing flow of the control device) FIG. 4 is a diagram showing a processing flow of the control device according to the first embodiment. The processing flow shown in Fig. 4 is executed after the operator turns on the power to the control device 2. Note that the control device 2 may be configured to start up automatically when the power to the work machine 1 is turned on.

[0039] First, the CPU 20 acquires information indicating the position of the work machine 1 via the GNSS antennas N1, N2, the communication interface 24, etc. (step S01). Here, the information indicating the position of the work machine 1 is three-dimensional coordinate information shown in the global coordinate system. The CPU 20 also calculates the orientation of the rotating bed of the work machine 1 based on the acquired positions of the GNSS antennas N1 and N2. Note that a design surface, which is terrain data acquired as three-dimensional data, is pre-recorded in the storage 25 of the control device 2. The CPU 20 displays the surrounding terrain on each display screen based on this terrain data and the positioning information of the work machine 1 obtained from the GNSS antennas N1 and N2.

[0040] Next, CPU 20 calculates the cutting edge position of bucket BK (step S02). Here, CPU 20 calculates the operating state of each movable part of work implement 12B, i.e., the angle of boom BM relative to upper structure 12, the angle of arm AR relative to boom BM, and the angle of bucket BK relative to arm AR, through sensors attached to each of cylinders SL1, SL2, and SL3 or an angle sensor such as an IMU. The storage 25 stores in advance the shape, connecting position, size, etc. of each movable part of the work implement 12B. Hereinafter, the shape, connecting position, size, etc. of each movable part of the work implement 12B will also be referred to as specification information. The CPU 20 combines this specification information with the calculation results of the operating state of each movable part and the terrain data to calculate the distance from the cutting edge of the bucket BK to the design surface. In other embodiments, the distance may be the distance from a part of the bucket BK other than the cutting edge to the design surface.

[0041] Next, the CPU 20 displays and updates each display screen based on the position information acquired in step S01 and the calculation results of the operating state of the movable parts of the work machine 12B performed in step S02 (step S03), thereby reflecting the actual operating state of the work machine 1 on each display screen.

[0042] Next, the CPU 20 determines whether or not a touch operation has been received from the operator (step S04). If a touch operation has not been received from the operator (step S04; NO), the CPU 20 ends the processing flow without performing any particular processing, and starts a new processing flow from the above-mentioned step S01. On the other hand, if a touch operation is received from the operator (step S04; YES), the CPU 20 performs the following process.

[0043] The display screen identification unit 203 of the CPU 20 identifies the type of display screen at the position where the touch operation was received (step S05). Specifically, the display screen identification unit 203 acquires touch coordinates indicating the position on the monitor 22 where the touch was sensed via the touch sensor 23. Then, the display screen identification unit 203 identifies to which of the display areas of the above-mentioned display screens (1) to (4) the touch coordinates belong. Specific processing by the display screen identification unit 203 is as follows. First, when the display screen identification unit 203 identifies the touch position, the display screen identification unit 203 acquires coordinate information (X, Y) indicating the position. At the time when the display screen identification unit 203 acquires the coordinate information (X, Y), the display screen identification unit 203 acquires which display screen is displayed at which position on the monitor 22 at that time. Then, the display screen identification unit 203 identifies whether the display screen displayed at the acquired coordinate information (X, Y) is a side view, a top view, a front view, a 3D screen, or another screen. In this way, the display screen identification unit 203 identifies the display screen currently being displayed on the monitor 22 at the touched position.

[0044] Next, the touch operation type identification unit 204 of the CPU 20 identifies the type of touch operation received via the touch sensor 23 (step S06). Specifically, the touch operation type identification unit 204 traces the number and movements of touch positions received from the operator, and determines which of the above-mentioned (a) to (e) the type of touch operation corresponds to. For example, when the touch sensor 23 senses the coordinates of at least one point on the surface of the monitor 22 and senses that the sensed coordinates move in a predetermined direction, it may be determined that a one-finger swipe has occurred. Also, for example, if the touch sensor 23 senses the coordinates of at least two points on the surface of the monitor 22 and senses that the sensed coordinates of at least two points are moving in the same predetermined direction, it may be determined to be a two-finger swipe. Also, for example, when the touch sensor 23 senses the coordinates of at least two points on the surface of the monitor 22 and senses that the sensed coordinates of at least two points are moving in a direction that brings them closer to each other, it may be determined that a pinch-in has occurred. Also, for example, when the touch sensor 23 senses the coordinates of at least two points on the surface of the monitor 22 and senses that the sensed coordinates of at least two points are moving away from each other, it may be determined that a pinch out has occurred. Also, for example, when the touch sensor 23 senses the coordinates of at least two points on the surface of the monitor 22 and senses that the sensed coordinates of at least two points move in a rotating direction while maintaining a predetermined distance, it may be determined that a rotation has occurred. In addition, when there are multiple sensed coordinates, if some of the multiple coordinates are close to each other, the coordinates may be determined as one point. Although the above describes operations using a finger, in other embodiments, operations may be performed using other means for sensing a touch panel monitor, such as a touch pen. Furthermore, the methods for determining (a) to (e) above are not limited to the above modes, and determination may be performed in other commonly used modes.

[0045] Next, the determination unit 201 of the CPU 20 determines whether or not the type of touch operation received in step S04 is an acceptable touch operation defined for each type of display screen (step S07). Specifically, the determination unit 201 determines whether or not the type of touch operation identified in step S06 corresponds to an acceptable touch operation defined for the type of display screen identified in step S05, while referring to the correspondence information defined in the above (A) to (D).

[0046] If the type of touch operation accepted in step S04 is not an acceptable touch operation defined for each type of display screen (step S07; NO), the CPU 20 ends the processing flow without performing any special processing, and starts a new processing flow from the above-mentioned step S01. On the other hand, if the type of touch operation received in step S04 is an acceptable touch operation defined for each type of display screen (step S07; YES), the control unit 202 of the CPU 20 performs control on the display screen on which the touch operation is received in accordance with the received touch operation (step S08). Specifically, the control unit 202 applies control according to the type of touch operation defined as in (A) to (E) above to the one display screen on which the touch operation is received.

[0047] It should be noted that steps S01 and S02 in the process flows described with reference to FIG. 4 are not essential components of the control device 2, and other embodiments may not include such steps.

[0048] (Example of control by the control unit) 5 to 13 are diagrams illustrating an example of control by the control unit according to the first embodiment. The control contents of the control unit 202 will be described in detail below with reference to FIGS.

[0049] First, an example of the initial state of the image displayed on the monitor 22 will be described. As shown in Fig. 5 to Fig. 13, the information displayed on the monitor 22 includes a guidance image G0 and three display screens. The three display screens can display a side view G1, a top view G2, a front view G3, and a 3D screen G4 (see Fig. 12) at the operator's discretion.

[0050] The side view G1, top view G2, front view G3 and 3D screen G4 are all diagrams showing the current operating state of the work machine 1, for example, the posture of the boom BM, arm AR and bucket BK that make up the work implement 12B, and their positional relationship with the surrounding terrain, and each shows the state as seen from a different perspective.

[0051] The guidance image G0 will be briefly described with reference to FIG. The guidance image G0 is a diagram that schematically shows the distance between the cutting edge of the bucket BK and the design surface. 5, the guidance image G0 is composed of multiple index images G40 arranged vertically. The index images G40 are displayed in color or colorless, and the bottommost index image G40 among the colored index images G40 corresponds to the cutting edge position of the bucket BK. The index image G40 with the reference position image G41 attached corresponds to the design surface. The index image G40 above the index image G40 with the reference position image G41 attached corresponds to a position higher than the design surface. The index image G40 below the index image G40 with the reference position image G41 attached corresponds to a position lower than the design surface. The distance between the lowest index image G40 displayed in color and the index image G40 with the reference position image G41 attached corresponds to the distance between the cutting edge and the design surface of the bucket BK. That is, the index image G40 to be displayed in color is determined based on the calculation result of the distance between the cutting edge and the design surface of the bucket BK described above. When the cutting edge of the bucket BK is located below the design surface, the index image G40 below the index image G40 to which the reference position image G41 is attached is displayed in color. The color of the indicator image G40 when displayed in color varies depending on the distance between the cutting edge of the bucket BK and the design surface.

[0052] (Pinch out (side view)) Fig. 5 shows an example of control when a pinch-out operation is received in the display area of ​​side view G1. As shown in Fig. 5 <before operation>, assume that the operator performs a pinch-out touch operation in the display area of ​​side view G1. In this case, as shown in Fig. 5 <after operation>, the control unit 202 enlarges the display image of side view G1 for which the touch operation was received, using the screen center C1 of side view G1 as a reference. Note that the hands shown in the figures are a schematic representation of the movements of the operator's own hands and fingers performing the touch operation, and are not images that are actually displayed on the display screen. The same applies to the following Figures 6 to 12. Also, the marks indicating the screen center C1, screen center C2, and work machine center C2' shown in Figures 5, 6, 7, 8, and 11 indicate positions on the display screen for the purpose of explanation, and are not images that are actually displayed on the display screen. In this case, the control unit 202 does not perform any control over the display areas of the guidance image G0, the top view G2, and the front view G3.

[0053] (Pinch out (top view)) Fig. 6 shows an example of control when a pinch-out operation is received in the display area of ​​the top view G2. As shown in Fig. 6 <before operation>, assume that the operator performs a pinch-out touch operation in the display area of ​​the top view G2. In this case, as shown in Fig. 6 <after operation>, the control unit 202 enlarges the display image of the top view G2 on which the touch operation was received, using the screen center C2 of the top view G2 as a reference. In this case, the control unit 202 does not perform any control over the display areas of the guidance image G0, the side view G1, and the front view G3.

[0054] (Pinch in (side view)) Fig. 7 shows an example of control when a pinch-in operation is received in the display area of ​​side view G1. As shown in Fig. 7 <before operation>, assume that the operator performs a pinch-in touch operation in the display area of ​​side view G1. In this case, as shown in Fig. 7 <after operation>, control unit 202 reduces the display image of side view G1 for which the touch operation was received, using screen center C1 of side view G1 as a reference. In this case, the control unit 202 does not perform any control over the display areas of the guidance image G0, the top view G2, and the front view G3.

[0055] (Rotate (top view)) Fig. 8 shows an example of control when a rotate operation is received in the display area of ​​the top view G2. As shown in Fig. 8 <Before Operation>, assume that the operator performs a rotate touch operation in the display area of ​​the top view G2. In this case, as shown in Fig. 8 <After Operation>, the control unit 202 rotates the display image of the top view G2 for which the touch operation was received about the vehicle body center, that is, with the work machine center C2' in the top view G2 as the reference. Note that the display image of the top view G2 rotates with the work machine center C2' as the reference, regardless of in which area of ​​the top view G2 the rotate touch operation is performed. In this case, the control unit 202 does not perform any control over the display areas of the guidance image G0, the side view G1, and the front view G3. Furthermore, for example, when a touch operation of the reset button RES superimposed on the top view G2 of FIG. 8 is received, the control unit 202 may perform control to return the display state of the top view G2 to the state before the rotation touch operation was performed.

[0056] (One finger swipe (side view)) Fig. 9 shows an example of control when a one-finger swipe operation is received in the display area of ​​the side view G1. As shown in Fig. 9 <before operation>, it is assumed that the operator performs a one-finger swipe touch operation in the display area of ​​the side view G1. In this case, as shown in Fig. 9 <after operation>, the control unit 202 slides the display image of the side view G1 for which the touch operation was received. In this case, the control unit 202 does not perform any control over the display areas of the guidance image G0, the top view G2, and the front view G3.

[0057] (One finger swipe (top view)) Fig. 10 shows an example of control when a one-finger swipe operation is received in the display area of ​​the top view G2. As shown in Fig. 10 <before operation>, assume that the operator performs a one-finger swipe touch operation in the display area of ​​the top view G2. In this case, as shown in Fig. 10 <after operation>, the control unit 202 slides the display image of the top view G2 on which the touch operation is received. In this case, the control unit 202 does not perform any control over the display areas of the guidance image G0, the side view G1, and the front view G3.

[0058] (Example of simultaneous operations) Fig. 11 shows an example of control when a pinch out operation is received in the display area of ​​the side view G1, and at the same time, a rotate operation is received in the display area of ​​the top view G2. As shown in Fig. 11 <before operation>, assume that the operator performs a pinch out touch operation in the display area of ​​the side view G1, and at the same time, performs a rotate touch operation in the display area of ​​the top view G2. In this case, as shown in Fig. 11 <after operation>, the control unit 202 enlarges the display image of the side view G1, for which the pinch out touch operation has been received, based on the screen center C1, and rotates the display image of the top view G2, for which the rotate touch operation has been received, based on the work machine center C2'. In this way, when the control unit 202 receives touch operations simultaneously on each of a plurality of display screens, the control unit 202 simultaneously performs control according to each touch operation on each of the display screens on which the touch operations have been received.

[0059] Note that the combination of display screens on which simultaneous touch operations are performed and the combination of types of touch operations are not limited to the example of FIG. 11 and can be any combination. For example, it is possible to accept one of a one-finger swipe, pinch-in, or pinch-out operation on the side view G1, while accepting one of a one-finger swipe, pinch-in, or pinch-out operation on the front view G3. Similarly, it is possible to accept one of a one-finger swipe, pinch-in, or pinch-out operation on the top view G2, while accepting one of a one-finger swipe, pinch-in, or pinch-out operation on the front view G3. Furthermore, it is possible to accept one of a one-finger swipe, pinch-in, or pinch-out operation on the side view G1, while accepting one of a one-finger swipe, two-finger swipe, pinch-in, or pinch-out operation on a 3D screen G4 (described later).

[0060] (Two-finger swipe (3D screen)) Fig. 12 shows an example of control when a two-finger swipe operation is received in the display area of ​​3D screen G4. As shown in Fig. 12 <before operation>, assume that the operator performs a two-finger swipe touch operation in the display area of ​​3D screen G4. In this case, as shown in Fig. 12 <after operation>, the control unit 202 rotates the position of the camera in three-dimensional space that captures the 3D screen G4 on which the touch operation was received, around a predetermined point. In this case, the control unit 202 does not perform any control over the display areas of the guidance image G0, the side view G1, and the front view G3.

[0061] The relationship between the two-finger swipe and one-finger swipe touch operations on the 3D screen G4 and the movement of the camera position in the three-dimensional space V will be described with reference to FIG. When a two-finger swipe touch operation is received in the display area of ​​the 3D screen G4, the control unit 202 moves the camera position in the three-dimensional space V along a spherical surface SP centered on the central position O so that the orientation of the camera R capturing the three-dimensional image always faces a predetermined central position O. This allows the operator to change the viewing angle as desired while keeping the central position O at the center of the screen. When a one-finger swipe touch operation is received in the display area of ​​3D screen G4, control unit 202 moves the camera position in three-dimensional space V along plane PL without changing the orientation of camera R. In this way, it is possible to change the area to be displayed in three-dimensional space V as desired. In this case, the position of the center position also moves parallel from O to O' in conjunction with the movement of the camera position from R to R'. 13 shows an example in which the operator swipes horizontally on the monitor 22, whether it is a one-finger swipe touch operation or a two-finger swipe touch operation, and as a result, the camera position moves horizontally in the three-dimensional space V. However, when the operator swipes vertically or diagonally on the monitor 22, the camera position also moves vertically in the three-dimensional space V, regardless of whether it is a one-finger swipe touch operation or a two-finger swipe touch operation.

[0062] (Other display screens) Note that even if the control unit 202 receives any of the above touch operations (a) to (e) in the display area of ​​the guidance image G0, it does not perform control in response to the touch operation. Moreover, even if the control unit 202 receives any of the above touch operations (a) to (e) in an area where, in addition to the guidance image G0, for example, an icon, a pop-up menu window, or the like is displayed, it does not perform control in response to the touch operation. In order to realize the above control, step S05 in FIG. 4 may be executed in more detail as follows. That is, when a touch operation by the operator is received (step S04; YES), the display screen identification unit 203 first determines whether or not the position on the monitor 22 where the touch operation is received is a display area that does not belong to any of the above (1) to (4). If the display area is one that does not belong to any of the above (1) to (4), that is, a display area for a guidance image G0, an icon, a menu window, or the like, the control unit 202 ends the process without performing any particular control. That is, the control unit 202 does not perform any control in response to the received touch operation. On the other hand, if the position on the monitor 22 where the touch operation is received is a display area that belongs to any of the above (1) to (4), the display screen identification unit 203 identifies the type of the display screen. By doing this, when an operator performs a touch operation with the intention of inputting to the guidance image G0 or the menu window, it is possible to prevent the touch operation from being mistaken for an operation (a) to (e) on each display screen (1) to (4).

[0063] Note that the enlargement and reduction of each display screen may be performed by pinching in and out, or by touching dedicated buttons superimposed on each display screen. For example, the control device 2 may enlarge or reduce the display screen by accepting touch operations of "+" and "-" buttons superimposed on each display screen shown in Figures 5 to 12.

[0064] The images displayed on the monitor 22 are not limited to those shown in FIGS. 5 to 12 and may be displayed in various screen configurations. For example, the images shown in FIGS. 5 to 12 are each divided into three sections, each including a side view G1, a top view G2, a front view G3, and a 3D screen G4. However, other embodiments are not limited to this. For example, the control device 2 according to other embodiments may be configured such that the screen is divided into two sections, each including only two of the side view G1, the top view G2, the front view G3, and the 3D screen G4. Furthermore, the control device 2 according to other embodiments may be configured such that the screen is not divided into two sections, each including only one of the side view G1, the top view G2, the front view G3, and the 3D screen G4. Furthermore, the number of screen divisions and the type of display screen to be displayed in each divided display area may be freely customized by the operator.

[0065] In the above-described embodiment, the monitor 22 and the touch sensor 23 are described as being integrally formed with the housing of the control device 2, which is a so-called tablet-type terminal device, but other embodiments are not limited to this configuration. The control device 2 according to another embodiment may not include the monitor 22, but may transmit a signal to display a display image on a monitor separate from the control device 2. Similarly, the control device 2 according to another embodiment may not include the touch sensor 23, but may receive a signal related to a touch operation from a touch sensor separate from the control device 2. In addition, the control device 2 of another embodiment may be realized as a system consisting of a monitor and touch sensor separate from the control device 2, and two or more control devices each having part of the configuration of the control device 2 of the first embodiment.

[0066] The various processes of the control device 2 described above are stored in the form of a program on a computer-readable recording medium, and the various processes are performed by a computer reading and executing the program. The computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, the computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program.

[0067] The program may be one that realizes part of the above-mentioned functions, or may be one that realizes the above-mentioned functions in combination with a program already recorded in the computer system, such as a so-called differential file or differential program.

[0068] Although several embodiments of the present disclosure have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the inventions described in the claims and their equivalents, as well as in the scope and spirit of the inventions.

[0069] In the above-described embodiment, the work machine 1 has been described as a hydraulic excavator, but in other embodiments, the work machine 1 can be applied to various work machines such as a dump truck, a wheel loader, and a bulldozer.

[0070] Furthermore, in the above-described embodiment, it has been described that one control device 2 is installed on the work machine 1, but in other embodiments, a part of the configuration of the control device 2 may be arranged in another control device, and the control system may be realized by two or more control devices. Note that the control device 2 according to the above-described embodiment is also an example of a control system.

[0071] Furthermore, although the control device 2 in the above-described embodiment has been described as being installed on the work machine, in other embodiments, part or all of the configuration of the control device 2 may be installed outside the work machine.

[0072] Furthermore, in the above-described embodiment, the monitor is described as being installed on the work machine, but in other embodiments, the monitor may be installed outside the work machine. For example, the monitor may be installed at a location away from the work site, and the control device 2 may send a signal to display a display screen on the monitor via a network such as the Internet or wireless communication. [Explanation of symbols]

[0073] 1 Work machine, 2 Control device, 20 CPU, 200 Display signal generation unit, 201 Determination unit, 202 Control unit, 203 Display screen identification unit, 204 Touch operation type identification unit 21 Memory, 22 Monitor, 23 Touch sensor, 24 Communication interface, 25 Storage

Claims

1. A control device for a touch panel monitor for a work machine, comprising: a display signal generating unit that generates a display signal to be displayed on the touch panel monitor, which includes a plurality of display screens each including a side view of the work machine and the surrounding terrain; a display screen identification unit that identifies to which of the plurality of display screens a display area that has received a touch operation belongs; a control unit that determines a type of the touch operation from a mode of the received touch operation, and, when the determined type of the touch operation is an acceptable touch operation determined for each type of the specified display screen, performs control on the one display screen in accordance with the received touch operation; A control device comprising:

2. The acceptable touch operations on a top view, which is one type of the display screen, include at least a rotation operation. The control device according to claim 1 .

3. When the control unit receives the rotation operation on the display screen of the top view, the control unit rotates the top view around the center of the vehicle body. The control device according to claim 2 .

4. The acceptable touch operation on a 3D screen, which is one of the types of the display screen, includes a swipe operation based on touches at least at two or more points. The control device according to any one of claims 2 and 3.

5. When the control unit receives a swipe operation based on the touch of two or more points on the display screen of the 3D screen, the control unit rotates and moves a camera position in three-dimensional space around a predetermined point. The control device according to claim 4.

6. When touch operations are simultaneously received on each of the plurality of display screens, the control unit simultaneously performs control on each of the display screens on which the touch operations are received in accordance with the respective touch operations. The control device according to any one of claims 1 to 5.

7. The control unit prohibits control according to the touch operation in an area where a predetermined display screen is displayed. The control device according to any one of claims 1 to 6.

8. A first operation is included in the acceptable touch operations of a first display screen that is one of the plurality of display screens, and a second display screen that is one of the plurality of display screens and is different from the first display screen, and the first operation is not included in the acceptable touch operations of a second display screen that is one of the plurality of display screens and is different from the first display screen. The control device according to any one of claims 1 to 7.

9. A control device according to any one of claims 1 to 8 is provided. Work machinery.

10. A method for controlling a touch panel monitor for a work machine, comprising: generating a display signal for display on the touch panel monitor including a plurality of display screens including a side view of the work machine and surrounding terrain; identifying to which of the plurality of display screens a display area on which a touch operation has been received belongs; determining a type of the touch operation from a mode of the received touch operation, and if the determined type of the touch operation is an acceptable touch operation determined for each type of the specified display screen, performing control on the one display screen according to the received touch operation; A control method comprising:

Images