Shovel and display device for shovel

Abstract

To improve safety.SOLUTION: A shovel according to one aspect of the embodiment includes a control unit configured to display a first screen based on image information captured by an imaging device on a display device, and then switch from the first screen to a second screen based on the image information in accordance with a first operation from a first operating device, and to perform control to return to the first screen when a second operation from a second operating device is accepted.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a shovel and a display device for a shovel. [Background technology]

[0002] Conventionally, a display device that displays various information has been provided on a shovel. In recent years, a technology has been proposed that displays the surrounding situation using image information captured by an imaging device in order to check the surroundings of the shovel (see Patent Document 1). This allows the operator of the shovel to recognize the surroundings of the shovel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-183638 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 does not take into consideration switching of the display of the surrounding situation. In other words, when the surrounding situation is displayed using image information, the location that the operator should check varies depending on the situation. For this reason, it is preferable to switch the display based on image information depending on the situation.

[0005] In view of the above, by switching the display based on image information from the imaging device depending on the status of the shovel, it is possible to make it easier for the operator to grasp the surroundings of the shovel, thereby improving safety. [Means for solving the problem]

[0006] A shovel according to one embodiment of the present invention includes a control unit configured to display a first screen based on image information captured by an imaging device on a display device, and then switch from the first screen to a second screen based on the image information in accordance with a first operation from a first operating device, and to perform control to return to the first screen when a second operation from a second operating device is accepted. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to make it easier for an operator to grasp the surroundings of a shovel, thereby improving safety. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a side view showing a shovel (excavator) according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of a drive control system of the shovel according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating functional blocks in a shovel controller of the shovel according to the first embodiment. [Figure 4] FIG. 4 illustrates an example of a screen displayed on the display device by the display control unit according to the first embodiment. [Diagram 5] FIG. 5 is a diagram showing a change in the rear image BM displayed in the image display area by the display control unit according to the first embodiment. [Figure 6] FIG. 6 is a flowchart showing the control of the display screen in the shovel controller according to this modified example. [Figure 7] FIG. 7 is a side view of a shovel as a work machine according to the second embodiment. [Figure 8] FIG. 8 is a diagram illustrating functional blocks in a shovel controller of a shovel according to the second embodiment. [Figure 9] FIG. 9 is a conceptual diagram showing the processing by the three-dimensional space processing section according to the second embodiment. [Figure 10]FIG. 10 illustrates an example of a screen displayed on a display device by a display control unit according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing a change in an overhead image displayed in an overhead image display area by a display control unit according to the second embodiment. [Figure 12] FIG. 12 is a schematic diagram illustrating an example of a remote control system according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is an example, not a limitation of the invention, and all features and combinations described in the embodiment are not necessarily essential to the invention. In addition, the same or corresponding components in each drawing are denoted by the same or corresponding reference numerals, and the description may be omitted.

[0010] (First embodiment) First, an overview of a shovel 100 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a side view of a shovel 100 as a work machine according to a first embodiment. An upper rotating body 3 is rotatably mounted on a lower traveling body 1 of the shovel 100 via a rotating mechanism 2. A boom 4 is attached to the upper rotating body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6 is attached to the tip of the arm 5 as an end attachment. The end attachment may be a slope bucket, a dredging bucket, or the like.

[0011] The boom 4, arm 5, and bucket 6 constitute an excavation attachment AT, which is an example of an attachment, and are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.

[0012] A boom angle sensor S1 is attached to the boom 4, an arm angle sensor S2 is attached to the arm 5, and a bucket angle sensor S3 is attached to the bucket 6. The excavation attachment may be provided with a bucket tilt mechanism.

[0013] The boom angle sensor S1 detects the rotation angle of the boom 4. In this embodiment, the boom angle sensor S1 is an acceleration sensor, and can detect the boom angle, which is the rotation angle of the boom 4 relative to the upper rotating body 3. For example, the boom angle is at its minimum when the boom 4 is lowered to the lowest, and increases as the boom 4 is raised.

[0014] The arm angle sensor S2 detects the rotation angle of the arm 5. In this embodiment, the arm angle sensor S2 is an acceleration sensor, and can detect the arm angle, which is the rotation angle of the arm 5 with respect to the boom 4. For example, the arm angle is at its minimum when the arm 5 is fully closed, and increases as the arm 5 is opened.

[0015] The bucket angle sensor S3 detects the rotation angle of the bucket 6. In this embodiment, the bucket angle sensor S3 is an acceleration sensor, and can detect the bucket angle, which is the rotation angle of the bucket 6 with respect to the arm 5. For example, the bucket angle is at its minimum when the bucket 6 is fully closed, and increases as the bucket 6 is opened.

[0016] The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 may be a potentiometer using a variable resistor, a stroke sensor that detects the stroke amount of the corresponding hydraulic cylinder, or a rotary encoder that detects the rotation angle around the connecting pin, etc. The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 constitute a posture sensor that detects the posture of the excavation attachment.

[0017] The upper rotating body 3 is provided with a cabin 10 as a driver's cab, and is equipped with a driving source such as an engine 11. In addition, a hydraulic motor for rotation, a shovel controller 30, an imaging device S6, etc. are attached to the upper rotating body 3. The hydraulic motor for rotation may be a motor-generator for rotation.

[0018] The engine 11 is a driving source of the excavator 100. In this embodiment, the engine 11 is a diesel engine. An output shaft of the engine 11 is coupled to input shafts of a main pump 14 (see FIG. 2) and a pilot pump 15 (see FIG. 2).

[0019] The drive source is not limited to the engine 11, and may be a motor. The shovel 100 may be equipped with an engine and a motor as the drive source. The shovel 100 may be driven only by the engine, may be a hybrid type driven by the engine and the motor, or may be an electric shovel (electrical work machine) driven only by the motor.

[0020] The upper rotating body 3 is equipped with a cabin 10 serving as a driver's cab, a vehicle tilt sensor S4, a rotation angular velocity sensor S5, an imaging device S6, a positioning device S7, a communication device T1, and the like.

[0021] The machine body inclination sensor S4 is configured to detect the inclination of the upper rotating body 3 with respect to a predetermined plane. In this embodiment, the machine body inclination sensor S4 is an acceleration sensor that detects the inclination angle about the front-rear axis and the left-right axis of the upper rotating body 3 with respect to the horizontal plane. The front-rear axis and the left-right axis of the upper rotating body 3 are, for example, perpendicular to each other and pass through the shovel center point, which is a point on the rotation axis of the shovel 100.

[0022] The rotation angular velocity sensor S5 is configured to detect the rotation angular velocity of the upper rotating body 3. In this embodiment, the rotation angular velocity sensor S5 is a gyro sensor. The rotation angular velocity sensor S5 may be a resolver or a rotary encoder, etc. The rotation angular velocity sensor S5 may detect a rotation speed. The rotation speed may be calculated from the rotation angular velocity.

[0023] The imaging device S6 is configured to capture images of the surroundings of the excavator 100. The imaging device S6 is, for example, a monocular camera, a stereo camera, a distance imaging camera, an infrared camera, a LiDAR, etc. The imaging device S6 includes a rear camera S6B attached to the rear end of the upper surface of the upper rotating body 3, a left camera S6L attached to the left end of the upper surface of the upper rotating body 3, and a right camera S6R attached to the right end of the upper surface of the upper rotating body 3.

[0024] The rear camera S6B, the left camera S6L, and the right camera S6R are attached to the upper rotating body 3 so that their optical axes face diagonally downward and so that a part of the upper rotating body 3 is included in the imaging range. The imaging range of each of the rear camera S6B, the left camera S6L, and the right camera S6R has a viewing angle of about 180 degrees in a plan view, for example.

[0025] The positioning device S7 is configured to acquire information related to the position of the shovel 100. In this embodiment, the positioning device S7 is configured to measure the position and orientation of the shovel 100 in a reference coordinate system. Specifically, the positioning device S7 is a GNSS receiver incorporating an electronic compass, and measures the latitude, longitude, and altitude of the current position of the shovel 100, and measures the orientation of the shovel 100. The reference coordinate system according to this embodiment is, for example, the World Geodetic System. The World Geodetic System is a three-dimensional orthogonal XYZ coordinate system with the origin at the center of gravity of the earth, the X axis in the direction of the intersection of the Greenwich meridian and the equator, the Y axis in the direction of 90 degrees east longitude, and the Z axis in the direction of the North Pole.

[0026] A shovel controller 30 is installed in the cabin 10. Also installed in the cabin 10 are a driver's seat, a display device DI, an operating device 26, and the like.

[0027] A touch panel sensor S8 is provided on the surface of the display device DI. The touch panel sensor S8 transmits a signal indicating an operation performed on the display device DI by the operator to the shovel controller 30.

[0028] The shovel controller 30 is a calculation device that executes various calculations. The shovel controller 30 is provided, for example, in the cabin 10 and controls the drive of the shovel 100. The functions of the shovel controller 30 may be realized by any hardware, software, or a combination thereof. For example, the shovel controller 30 is mainly configured with a microcomputer including a central processing unit (CPU), a memory device such as a random access memory (RAM), a non-volatile auxiliary storage device such as a read only memory (ROM), and an interface device for various inputs and outputs. The shovel controller 30 executes various programs installed in the non-volatile auxiliary storage device on the CPU, for example, to realize various functions.

[0029] The communication device T1 is configured to control communication with devices external to the shovel 100. In this embodiment, the communication device T1 is configured to control communication between the communication device T1 and devices external to the shovel 100 via a communication line (including a wireless communication network). The communication device T1 includes, for example, a mobile communication module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), a satellite communication module for connecting to a satellite communication network, and the like.

[0030] Furthermore, the communication device T1 controls wireless communication between the excavator 100 and an external GNSS (Global Navigation Satellite System) surveying system, for example.

[0031] [Excavator drive control system] Fig. 2 is a diagram showing an example of the configuration of a drive control system of the excavator 100 in Fig. 1. In Fig. 2, a mechanical power transmission system is indicated by a double line, a hydraulic oil line by a thick solid line, a pilot line by a dashed line, and an electric drive and control system by a dotted line.

[0032] The drive system of the excavator 100 according to this embodiment includes the engine 11, the regulator 13, the main pump 14, the pilot pump 15, and the control valve unit 17. Moreover, the hydraulic drive system of the excavator 100 according to this embodiment includes hydraulic actuators such as the traveling hydraulic motors 1L, 1R, the swing hydraulic motor 2A, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9, which hydraulically drive the lower traveling structure 1, the upper rotating structure 3, the boom 4, the arm 5, and the bucket 6, respectively, as described above.

[0033] The engine 11 is a main power source in the hydraulic drive system, and is mounted, for example, on the rear of the upper rotating body 3. Specifically, the engine 11 rotates at a constant speed at a preset target speed under direct or indirect control by a shovel controller 30 (described later), and drives the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses light oil as fuel.

[0034] The engine 11 is connected to an engine control unit (hereinafter referred to as ECU) 74. The ECU 74 controls the fuel injection amount, fuel injection timing, boost pressure, etc. of the engine 11. Furthermore, the ECU 74 transmits various data indicating the state of the engine 11 to the shovel controller 30.

[0035] The regulator 13 controls the discharge amount of the main pump 14. For example, the regulator 13 adjusts the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the shovel controller 30. The regulator 13 controls the discharge amount (displacement volume) of the main pump 14 per rotation.

[0036] The main pump 14 is mounted on the rear of the upper rotating body 3, for example, like the engine 11, and supplies hydraulic oil to the control valve unit 17 through a high-pressure hydraulic line. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the shovel controller 30, the tilt angle of the swash plate is adjusted by the regulator 13 to adjust the stroke length of the piston, thereby controlling the discharge flow rate (discharge pressure).

[0037] The pilot pump 15 is an example of a pilot pressure generating device, and is configured to be able to supply hydraulic oil to the hydraulic control device via a pilot line 25. In this embodiment, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pressure generating device may be realized by the main pump 14. That is, the main pump 14 may have a function of supplying hydraulic oil to various hydraulic control devices via the pilot line 25, in addition to a function of supplying hydraulic oil to the control valve unit 17 via the hydraulic oil line. In this case, the pilot pump 15 may be omitted.

[0038] The control valve unit 17 is configured to selectively supply hydraulic oil received from the main pump 14 to one or more hydraulic actuators. In this embodiment, the control valve unit 17 includes a plurality of control valves corresponding to the plurality of hydraulic actuators. The control valve unit 17 is configured to selectively supply hydraulic oil discharged from the main pump 14 to one or more hydraulic actuators. The hydraulic actuators include, for example, the boom cylinder 7, the arm cylinder 8, the bucket cylinder 9, the left traveling hydraulic motor 1L, the right traveling hydraulic motor 1R, and the swing hydraulic motor 2A.

[0039] The pilot pump 15 is connected to the operating device 26 via a pilot line 25 and a gate lock valve 50. The operating device 26 includes, for example, a left operating lever, a right operating lever, and a traveling operating device. The traveling operating device includes, for example, a traveling lever and a traveling pedal. Each of the operating devices 26 is a hydraulic operating device, and is connected to a pilot port of a corresponding spool valve in the control valve unit 17 from the pilot line 25 via the gate lock valve 50. However, the operating device 26 may be an electric operating device. When an electric operating device is applied, the excavator controller 30 can operate a hydraulic actuator corresponding to a specific operating device 26 even when no operation is being performed on that specific operating device 26. This makes it possible to realize, for example, remote operation of the excavator 100.

[0040] The operation sensor 29 detects the operation content of the operation device 26 as a pressure. The operation sensor 29 outputs a detected value to the shovel controller 30. The operation content of the operation device 26 may be detected electrically.

[0041] The gate lock lever DL is configured to be able to switch between an enabled state and an disabled state of the operating device 26. The enabled state of the operating device 26 indicates a state in which, when the operator operates the operating device 26, the hydraulic actuator corresponding to that operation operates. The disabled state of the operating device 26 indicates a state in which, even when the operator operates the operating device 26, the hydraulic actuator corresponding to that operation does not operate.

[0042] The gate lock lever DL is provided at the front end on the left side of the driver's seat. The operator can activate the operation device 26 by pulling up the gate lock lever DL to an unlocked state DLU (state shown by a solid line). The operator can also deactivate the operation device 26 by pushing down the gate lock lever DL to a locked state DLL (state shown by a dotted line).

[0043] The gate lock switch S9 is a device that outputs a signal to operate the gate lock valve 50. In this embodiment, the gate lock switch S9 is configured to switch its state according to the gate lock lever DL. For example, the gate lock switch S9 is configured to output an unlock signal when the gate lock lever DL is in an unlocked state DLU, and to output a lock signal when the gate lock lever DL is in a locked state DLL. The unlock signal and the lock signal may be a current signal or a voltage signal. The unlock signal and the lock signal may also be output by the excavator controller 30.

[0044] The gate lock valve 50 is a solenoid valve that switches between communication and cut-off of the pilot line 25 that connects the operating device 26 and the pilot pump 15. In this embodiment, the gate lock valve 50 opens the pilot line 25 when an unlock signal is received, and cuts off the pilot line 25 when an unlock signal is not received. It may also be configured to cut off the pilot line 25 when a lock signal is received.

[0045] The gate lock relay 51 switches between communication and interruption of an electric path E1 that connects the gate lock switch S9 and the gate lock valve 50. The gate lock relay 51 is an electromagnetic relay that is composed of, for example, a contact, a spring, a coil, etc. The gate lock relay 51 may be composed of a semiconductor switching element such as a MOSFET, a transistor, a thyristor, etc.

[0046] The gate lock relay 51 is connected to the gate lock switch S9 via an electric line E1a. Furthermore, the shovel controller 30 is also connected to the gate lock switch S9 via an electric line E1b. In other words, the shovel controller 30 can also receive the unlock signal or lock signal from the gate lock switch S9. Furthermore, the gate lock relay 51 is connected to the shovel controller 30 via an electric line E2.

[0047] The gate lock relay 51 switches the electric circuit E1 between a conductive state and a cut-off state depending on whether or not a current flows from the shovel controller 30 through the electric circuit E2.

[0048] In this embodiment, the shovel controller 30 may cause a current to flow through the electrical circuit E2 to bring the electrical circuit E1 (including the electrical circuits E1a and E1b) into a conductive state, for example, when it determines that the seat belt is fastened based on a signal from a seat belt switch (not shown), and when it determines that the operator is seated based on a seated signal from a seat occupied switch (not shown).

[0049] When the electric path E1 is in a conductive state, the gate lock relay 51 can transmit a signal (for example, an unlock signal, a lock signal, etc.) from the gate lock switch S9 or the shovel controller 30 to the gate lock valve 50.

[0050] <<Excavator functional blocks>> Each functional block in the shovel controller 30 of the shovel 100 will be described. FIG. 3 is a diagram illustrating an example of a functional block in the shovel controller 30 of the shovel 100. Note that each functional block in the shovel controller 30 is conceptual, and does not necessarily have to be physically configured as shown in the figure. It is possible to configure all or a part of each functional block by distributing and integrating it functionally or physically in any unit. All or any part of each processing function performed in each functional block is realized by a program executed by a CPU. Alternatively, each functional block may be realized as hardware by wired logic. As shown in FIG. 3, the shovel controller 30 is provided with an acquisition unit 301, an operation processing unit 302, and a display control unit 303 by implementing a program.

[0051] The acquisition unit 301 acquires signals from various detection devices provided in the excavator 100. For example, the acquisition unit 301 acquires detection results from a boom angle sensor S1, an arm angle sensor S2, and a bucket angle sensor S3. The acquisition unit 301 also acquires image information and position information.

[0052] The image information is information captured by the imaging device S6. The position information is information indicating the position and orientation of the shovel 100 in the reference coordinate system measured by the positioning device S7.

[0053] Moreover, the acquisition unit 301 acquires information indicating an operation performed by the operating device 26 from the operation sensor 29. Furthermore, the acquisition unit 301 acquires an unlock signal or a lock signal from the gate lock switch S9.

[0054] Moreover, the acquiring unit 301 acquires, from the touch panel sensor S8, an operation signal indicating an operation performed on the display device DI.

[0055] When the acquisition unit 301 acquires an operation signal indicating the detected values ​​of the operation direction and operation amount of the operation device 26, the operation processing unit 302 specifies the drive control corresponding to the operation signal.

[0056] Furthermore, when the operation processing unit 302 acquires an operation signal from the touch panel sensor S8, it identifies a process corresponding to the acquired operation signal. For example, when a pinch-in operation, a pinch-out operation, a double tap operation, a tap operation, a swipe operation, or the like is received from the touch panel sensor S8, it identifies a process for the operation. The process corresponding to the operation is, for example, a screen enlargement process, a screen reduction process, or the like.

[0057] The display control unit 303 controls the display device DI to display a screen.

[0058] Fig. 4 explains an example of a screen displayed on the display device DI by the display control unit 303. Fig. 4 is a diagram showing an example of a screen displayed on the display device DI.

[0059] The display screen 41 of the display device DI includes an upper display area 41A, a main display area 41B, and a lower display area 41C. The upper display area 41A is disposed in an upper part of the display screen 41. The lower display area 41C is disposed in a lower part of the display screen 41. The main display area 41B is disposed in the center of the display screen 41. The main display area 41B is disposed between the upper display area 41A and the lower display area 41C. The area of ​​the main display area 41B is, for example, larger than the combined area of ​​the upper display area 41A and the lower display area 41C.

[0060] The upper display area 41A includes a date and time display area 41a, a driving mode display area 41b, an attachment display area 41c, a fuel efficiency display area 41d, an engine control status display area 41e, a coolant temperature display area 41g, a remaining fuel amount display area 41h, a rotation speed mode display area 41i, a urea water remaining amount display area 41j, and a hydraulic oil temperature display area 41k.

[0061] The main display area 41B includes an image display area 41n. The image display area 41n includes an overhead image display area 41n1, a rear image display area 41n2, and a right image display area 41n3. These image display areas 41n1, 41n2, and 41n3 occupy most of the display screen 41. The image display areas 41n1, 41n2, and 41n3 are disposed below the upper display area 41A.

[0062] The lower display area 41C includes an engine operating time display area 41f, an air conditioner operating state display area 41m, and a menu display area 41p.

[0063] The date and time display area 41a is an area for displaying the current date and time. The driving mode display area 41b is an area for displaying the current driving mode. The attachment display area 41c is an area for displaying an image representing the currently attached attachment. The fuel efficiency display area 41d includes an average fuel efficiency display area 41d1 for displaying the lifetime average fuel efficiency or the section average fuel efficiency, and an instantaneous fuel efficiency display area 41d2 for displaying the instantaneous fuel efficiency.

[0064] The engine operating time display area 41f is an area that displays the accumulated operating time of the engine 11. The coolant temperature display area 41g is an area that displays the current temperature state of the engine coolant. The remaining fuel amount display area 41h is an area that displays the remaining amount of fuel stored in the fuel tank. The urea water remaining amount display area 41j is an area that displays the remaining amount of urea water stored in the urea water tank as an image. The hydraulic oil temperature display area 41k is an area that displays the temperature state of the hydraulic oil in the hydraulic oil tank.

[0065] The air conditioner operation status display area 41m includes an air outlet display area 41m1 that displays the current position of the air outlet, an operation mode display area 41m2 that displays the current operation mode of the air conditioner, a temperature display area 41m3 that displays the current set temperature, and an air volume display area 41m4 that displays the current set air volume.

[0066] The image display area 41n is an area for displaying an image captured by the imaging device S6. The image display area 41n displays an overhead image FV, a rear image BM, and a right image RM. The overhead image FV is a virtual viewpoint image generated by the display control unit 303, and is generated based on images captured by the rear camera S6B, the left camera S6L, and the right camera S6R. In addition, a shovel figure GE corresponding to the shovel 100 is arranged in the center of the overhead image FV. This is to allow the operator to intuitively grasp the positional relationship between the shovel 100 and objects existing around the shovel 100.

[0067] The rear image BM is an image that shows the space behind the shovel 100, and includes an image GC of the counterweight. The rear image BM is a real viewpoint image generated by the display control unit 303, and is generated based on an image acquired by the rear camera S6B.

[0068] The right image RM is an image that shows the space to the right of the shovel 100, and includes the image RF of the right side frame of the shovel 100. The right image RM is a real viewpoint image generated by the display control unit 303, and is generated based on an image acquired by the right camera S6R.

[0069] The display device DI displays an overhead image FV in the overhead image display area 41n1, a rear image BM in the rear image display area 41n2, and a right image RM in the right image display area 41n3.

[0070] The menu display area 41p has tabs 41p1 to 41p7. In the example of Fig. 4, the tabs 41p1 to 41p7 are arranged at intervals from each other on the left and right at the bottom of the display screen 41. Icons for displaying various information are displayed on the tabs 41p1 to 41p7.

[0071] In tab 41p1, menu detail item icons for displaying menu detail items are displayed. When tab 41p1 is selected by the operator, the icons displayed in tabs 41p2 to 41p7 are switched to icons associated with the menu detail items.

[0072] An icon for displaying information related to the digital level is displayed on the tab 41p4. When the operator selects the tab 41p4, the rear image BM and the right image RM are switched to a screen showing information related to the digital level. However, the screen showing information related to the digital level may be displayed by superimposing the rear image BM and the right image RM or by reducing the rear image BM. Also, the overhead image FV may be switched to a screen showing information related to the digital level, or the screen showing information related to the digital level may be displayed by superimposing the overhead image FV or by reducing the overhead image FV.

[0073] An icon for displaying information on information-based construction is displayed on the tab 41p6. When the operator selects the tab 41p6, the rear image BM and the right image RM are switched to a screen showing information on information-based construction. However, the screen showing information on information-based construction may be displayed by superimposing the rear image BM and the right image RM or by reducing the rear image BM and the right image RM. The overhead image FV may be switched to a screen showing information on information-based construction, or may be superimposed on the overhead image FV or by reducing the overhead image FV to display a screen showing information on information-based construction. Information-based construction is a technology for performing construction according to the current situation by communicating with an external device during construction by the excavator 100. At that time, the excavator 100 may perform semi-automatic control or autonomous control.

[0074] An icon for displaying information related to the crane mode is displayed on tab 41p7. When tab 41p7 is selected by the operator, the rear image BM is switched to a screen showing information related to the crane mode. However, a screen showing information related to the crane mode may be displayed by superimposing on the rear image BM or by reducing the rear image BM. Also, the overhead image FV may be switched to a screen showing information related to the crane mode, or a screen showing information related to the crane mode may be displayed by superimposing on the overhead image FV or by reducing the overhead image FV.

[0075] No icon is displayed on the tabs 41p2, 41p3, and 41p5, so even if the operator operates the tabs 41p2, 41p3, and 41p5, the image displayed on the display screen 41 does not change.

[0076] It should be noted that the icons displayed on the tabs 41p1 to 41p7 are not limited to the above examples, and icons for displaying other information may be displayed.

[0077] The touch panel sensor S8 detects a gesture operation on the display screen 41. Then, the operation processing unit 302 identifies a process corresponding to the detected gesture operation. Then, the display control unit 303 updates the display screen in accordance with the identified process.

[0078] For example, the display control unit 303 can perform enlargement and reduction processing on the rear image BM or the right image RM in response to a gesture operation on the rear image BM or the right image RM.

[0079] Fig. 5 is a diagram showing a change in the rear image BM displayed in the image display area 41n2 by the display control unit 303 according to this embodiment. The display screen shown in Fig. 5 is the same as Fig. 4 except for the image display area 41n2.

[0080] 5(A), a rear image BM1 is displayed in an image display area 41n2 of a display screen (an example of a first screen) displayed by the display control unit 303. When a pinch-out operation 1511 is received on the image display area 41n2, the operation processing unit 302 determines that the pinch-out operation 1511 is an enlargement process on the rear image BM1 displayed in the image display area 41n2.

[0081] Then, the display control unit 303 switches and displays the screen from the rear image BM1 shown in Fig. 5(A) to the rear image BM2 shown in Fig. 5(B) in accordance with a pinch out operation (an example of a first operation) 1511 from the touch panel sensor S8 (an example of a first operation device). Note that in this embodiment, a case will be described in which the gesture operation (an example of a first operation) for switching from the screen including the rear image BM1 (an example of a first screen) to the screen including the rear image BM2 (an example of a second screen) is a pinch out operation, but the operation for switching the display mode is not limited to the pinch out operation, and may be any gesture operation set in advance.

[0082] In this way, when the operator judges that the rear image BM or the right image RM is difficult to see after referring to it, the operator performs a gesture operation to enlarge the rear image BM or the right image RM. In accordance with this operation, a screen in which the rear image BM or the right image RM is enlarged is displayed. This makes it easier for the operator to detect a person or object that exists in the area captured in the rear image BM or the right image RM.

[0083] 5(B), when a pinch-in operation (an example of a first gesture operation) 1512 is received in the image display area 41n2 while the rear image BM2 is displayed, the operation processing unit 302 identifies that the pinch-in operation 1512 is a reduction process for the rear image BM1 displayed in the image display area 41n2. Then, as shown in FIG. 5(A), the display control unit 303 can display the original rear image BM1 in the image display area 41n2.

[0084] The rear image BM2 shown in Fig. 5(B) is an image in which an area specified by the operator is enlarged, and is not an image showing a wide area behind the shovel 100. In other words, even if the operator cannot recognize the behavior of the person 1521 shown in the rear image BM1, the operator can recognize the details of the person 1522 by referring to the rear image BM2. In other words, since the operator can recognize the behavior of the person 1521, the operator can determine whether or not it is acceptable to operate the shovel 100. After determining that it is acceptable to operate the shovel 100, there is a possibility that the operator will perform an operation to travel the shovel 100 with the enlarged rear image BM2 shown on the display screen.

[0085] Incidentally, when the shovel 100 is operating, it is preferable that people present within a predetermined radius (for example, 12 m) from the shovel 100 are displayed at a predetermined size (for example, 10 mm) or more. Furthermore, in order to recognize the surrounding situation behind the shovel 100, it is preferable that a range of a predetermined angle (45 degrees) or more behind the shovel 100 is displayed on the screen.

[0086] In other words, if the operator operates the shovel 100 while the rear image BM2 shown in FIG. 5(B) is displayed, there is a possibility that the operator will not be able to grasp the situation behind the shovel 100.

[0087] Therefore, in this embodiment, when an operation 1513 is performed on the operation device 26 (an example of a second operation device), if the operation processing unit 302 determines that the operation is an operation to drive the shovel 100, the display control unit 303 performs control to return to a screen that allows the operator to recognize the surroundings of the shovel 100.

[0088] Specifically, when the display control unit 303 receives an operation to drive the shovel 100 from the operation device 26 while the image displayed in the image display area 41n2 is being enlarged or reduced, the display control unit 303 performs control to return the display screen to one including the rear image BM1 shown in Figure 5 (A).

[0089] The operation (an example of a second operation) 1513 of the operating device 26 (an example of a second operating device) may be any operation for driving the shovel 100, and may be, for example, an operation for lowering or raising the boom 4 by an operating lever, or an operation for opening or closing the arm 5 or the bucket 6. It may also be an operation for rotating the upper rotating body 3 of the shovel 100 by an operating lever. Furthermore, the operation (an example of a second operation) 1513 of the operating device 26 (an example of a second operating device) may be an operation for starting the traveling of the shovel 100 by a traveling operating device. Furthermore, the operation (an example of a second operation) 1513 of the operating device 26 (an example of a second operating device) may be an operation of an attachment or the like connected via a spare valve provided on the shovel 100.

[0090] In other words, it is sufficient to operate any one of the left operating lever, right operating lever, travel lever, and travel pedal that configure the operating device 26. In this embodiment, when any one of the left operating lever, right operating lever, travel lever, and travel pedal is operated, control is performed to return to the display screen including the rear image BM1, so that the operator can grasp the surrounding situation in accordance with the operation of the excavator 100. Therefore, improved safety can be achieved.

[0091] That is, even when a preset operation (an example of a second operation) different from an operation (e.g., a gesture operation) for returning from a state in which the display screen including the rear image BM2 shown in FIG. 5(B) is displayed to a display screen including the rear image BM1 shown in FIG. 5(A) is received by the display control unit 303 according to the present embodiment, the display control unit 303 performs control for returning to the display screen including the rear image BM1 shown in FIG. 5(A). In this way, when a preset operation is performed, control for returning to the display screen including the rear image BM1 is performed even for an operation other than the operation for returning to the display screen including the rear image BM1. That is, when operating the excavator 100, the operator does not need to perform an operation for returning to the display screen including the rear image BM1 shown in FIG. 5(A), and therefore the operation time can be shortened.

[0092] In the present embodiment, a pinch-in operation is performed to return the display screen including the rear image BM1 shown in Fig. 5(A) from the display screen including the rear image BM2 shown in Fig. 5(B) to the display screen including the rear image BM1 shown in Fig. 5(A). However, the operation to return the display screen is not limited to the pinch-in operation on the touch panel sensor S8, and may include, for example, a double tap operation or any other predetermined gesture operation.

[0093] 5, a case has been described in which the rear image BM is enlarged in accordance with an operation received by the touch panel sensor S8. However, the present embodiment is not limited to an example in which control to return the display screen is performed only when the image is enlarged. For example, when the image is reduced, control to return the display screen may be performed when an operation to drive the shovel 100 is received.

[0094] In this embodiment, the case where the operation device for accepting operations on the display screen is the touch panel sensor S8 has been described. However, in this embodiment, the operation device for enlarging or reducing the image displayed on the display screen is not limited to the touch panel sensor S8. For example, an operation device provided with buttons or the like for operating the display device DI may be used.

[0095] In this embodiment, an example has been described in which the shovel controller 30 displays on the screen an image that has been enlarged or reduced from the image information captured by the imaging device S6. However, this embodiment is not limited to a method of enlarging or reducing the image when changing the image to be displayed. For example, by providing each imaging device S6 with a zoom function and a rotation mechanism, the shovel controller 30 may control the zoom function and rotation function of the imaging device S6 in response to an operation by an operator, so that an image corresponding to the operation is captured.

[0096] In the shovel controller 30 according to this embodiment, when an operation is performed from the operating device 26, control is performed to return the display screen, making it easier to check the surrounding situation, thereby achieving improved safety.

[0097] (Modification 1 of the first embodiment) In the above-described embodiment, an example has been described in which the display control unit 303 performs control to return the display screen when an operation is performed on the operation device 26 (an example of a second operation device). However, the above-described embodiment does not limit the control to return the display screen to when an operation is performed on the operation device 26 (an example of a second operation device). Therefore, in the modified example, a case will be described in which the presence or absence of locking by the gate lock lever DL is controlled.

[0098] Fig. 6 is a flowchart showing the control of a display screen in the shovel controller 30 according to this modified example. In the processing procedure shown in Fig. 6, it is assumed that the operator of the shovel 100 gets on the shovel and pulls up the gate lock lever DL to set it to the unlocked state DLU (the state shown by the solid line), but the shovel 100 is not operating.

[0099] As shown in FIG. 6, the display control unit 303 displays a screen including a rear image BM and a right image RM (S1601).

[0100] The operation processing unit 302 specifies that the process instructed by the gesture operation received on the rear image BM or the right image RM (via the touch panel sensor S8) is an enlargement process or a reduction process (S1602).

[0101] When the operation processing unit 302 identifies that the operation is an enlargement process or a reduction process, it outputs a lock signal to the gate lock valve 50 (S1603). As a result, the pilot line 25 is blocked, so that the operation of the shovel 100 is suppressed even if an operation is performed on the operating device 26.

[0102] The display control unit 303 displays a screen including the enlarged or reduced image (the rear image BM or the right image RM) (S1604). The operator can easily check the surroundings of the excavator 100 by referring to the image processed according to the operation.

[0103] Then, the operation processing unit 302 determines, based on the signal acquired by the acquisition unit 301, whether or not an operation to return the gate lock lever DL to the unlocked state DLU has been accepted (S1605).

[0104] When the operation processing unit 302 determines that an operation to set the state to the unlocked state DLU has not been received (S1605: NO), the operation processing unit 302 determines whether or not an operation of the operation device 26 has been received (S1606) based on the signal acquired by the acquisition unit 301. When it determines that an operation of the operation device 26 has not been received (S1606: NO), the process is performed again from S1605.

[0105] On the other hand, when the operation processing unit 302 determines that the operation of the operation device 26 has been accepted (S1606: NO), the display control unit 303 displays a message prompting the user to reset the gate lock lever DL to the unlocked state DLU (S1607). The message may be, for example, a pop-up display.

[0106] In S1605, when the operation processing unit 302 determines that an operation to change to the unlocked state DLU has been received (S1605: YES), the operation processing unit 302 outputs an unlock signal to the gate lock valve 50 (S1608).

[0107] Furthermore, the display control unit 303 displays the rear image BM and the right image RM restored to their original magnification ratios (S1609).

[0108] In this modification, when the rearward image BM or rightward image RM is enlarged or reduced, the operation of the shovel 100 is suppressed by locking and interrupting the pilot line 25. Then, when the lock is released, the rearward image BM and rightward image RM are returned to their original state, so that the operator can operate the shovel 100 in a state in which the rearward image BM and rightward image RM have been returned to their original state, in other words, in a state in which the operator can grasp the surroundings of the shovel 100.

[0109] In this modification, an example has been described in which, when the rear image BM or the right image RM is enlarged or reduced, the operation of the excavator 100 is suppressed by locking and interrupting the pilot line 25, and when the lock is released, the rear image BM and the right image RM are returned to their original state. However, the operation of the gate lock lever DL is not limited to this control. For example, after being locked when reduced or enlarged, the locked state may be maintained even if the gate lock lever DL is operated until the display of the rear image BM or the right image RM is returned to its original state by the operation of the operator. Then, after the display of the rear image BM or the right image RM is returned to its original state by the operation of the operator, the excavator controller 30 controls the gate lock switch S9 to transmit an unlock signal to the gate lock valve 50. As an example of this control, the excavator controller 30 realizes it by controlling the electric circuit E1 to be switched between a conductive state and a cut-off state. Furthermore, when the operation processing unit 302 accepts an operation of the operation device 26 after locking, the display control unit 303 may display a message prompting an operation to return the viewpoint to its original state.

[0110] In this modified example, the method of unlocking is not limited to a method in which the operator operates the gate lock lever DL again to release the lock. For example, a release switch (not shown) may be provided in addition to the gate lock lever DL. When the operator presses the release switch, the shovel controller 30 may be controlled to transmit an unlock signal to the gate lock valve 50. By performing the above-mentioned control in this modified example, it is possible to achieve the same effects as the first embodiment.

[0111] (Modification 2 of the first embodiment) In the above-described embodiment and modified examples, an example has been described in which the display control unit 303 performs control to return the display screen when an operation is performed on the operation device 26 or the gate lock lever DL. However, the above-described embodiment does not limit the control to return the display screen to when an operation is performed on the operation device 26 or the gate lock lever DL. Therefore, in modified example 2, a case in which control is performed when a person is detected will be described.

[0112] In this modification, first, the display control unit 303 displays a screen including an image (rearward image BM or rightward image RM) that has been enlarged or reduced. Then, when the shovel controller 30 detects a person from the image information captured by the imaging device S6, the shovel controller 30 displays the rearward image BM and rightward image RM restored to their original magnification ratio.

[0113] In this modified example, by performing the above-described control, the operator can operate the shovel 100 after recognizing people present around the shovel 100, thereby achieving improved safety.

[0114] Second embodiment In the above-described embodiment or modification, an example has been described in which an enlarged or reduced image (rear image BM or right image RM) is returned to its original state when a shovel is operated. However, the above-described embodiment or modification does not limit the image to be returned to its original state to the rear image BM or right image RM. In the second embodiment, an example will be described in which an image displayed from an arbitrary viewpoint in a virtual three-dimensional space is returned to its original state when a shovel is operated.

[0115] First, an overview of a shovel 100 (100A) according to this embodiment will be described with reference to FIG. 7. FIG. 7 is a side view of a shovel 100A as a work machine according to a second embodiment. The shovel 100A shown in FIG. 7 is provided with four imaging devices S6. Specifically, in addition to the rear camera S6B, left camera S6L, and right camera S6R shown in the first embodiment, a front camera S6F is provided. The shovel 100A is similar to the shovel 100 shown in the first embodiment except for the provision of the front camera S6F, and therefore description thereof will be omitted.

[0116] The front camera S6F is attached, for example, to the front end of the upper surface of the cabin 10. The front camera S6F is attached so that the optical axis faces diagonally downward and so that a part of the cabin 10 is included in the imaging range. The imaging range of the front camera S6F has, for example, a viewing angle of about 180 degrees in a plan view.

[0117] <<Excavator functional blocks>> Each functional block in the shovel controller 30A of the shovel 100A will be described. FIG. 8 is a diagram illustrating an example of a functional block in the shovel controller 30A of the shovel 100A. Note that each functional block in the shovel controller 30A is conceptual, and does not necessarily have to be physically configured as shown in the figure. It is possible to configure all or a part of each functional block by distributing and integrating it functionally or physically in any unit. All or any part of each processing function performed in each functional block is realized by a program executed by a CPU. Alternatively, each functional block may be realized as hardware by wired logic. As shown in FIG. 8, the shovel controller 30A includes an acquisition unit 301, an operation processing unit 302, a three-dimensional space processing unit 801, and a display control unit 802 by realizing a program. Compared to the shovel controller 30 shown in the first embodiment, the shovel controller 30A of this embodiment is provided with a three-dimensional space processing unit 801 added thereto, and a display control unit 802 whose processing is different from that of the display control unit 303.

[0118] The three-dimensional space processing unit 801 performs processing relating to a virtual three-dimensional space for display on the display device DI. Next, the processing performed by the three-dimensional space processing unit 801 will be described.

[0119] Fig. 9 is a conceptual diagram showing processing by the three-dimensional space processing unit 801 according to this embodiment. As shown in Fig. 9, the three-dimensional space processing unit 801 holds a virtual projection plane 1900. The virtual projection plane 1900 is data stored in a storage unit (not shown). A point in a three-dimensional space having the virtual projection plane 1900 is determined by coordinates (X, Y, Z).

[0120] The virtual projection plane 1900 has, for example, a bottom surface 1901 along the ground and a side surface 1902 rising from the bottom surface 1901. The bottom surface 1901 corresponds to the ground (installation surface) on which the shovel 100A is installed. The side surface 1902 is a curved surface that rises in the Z-axis direction (height direction) from the outer periphery of the bottom surface 1901 and is configured to widen in the outer periphery. In the example shown in FIG. 9, a three-dimensional model 1910 representing the shovel 100A is installed near the center position of the bottom surface 1901. In the example shown in FIG. 9, the side surface 1902 is provided so as to surround the periphery of the three-dimensional model 1910. The three-dimensional model 1910 representing the shovel 100A is a model showing the current shape of the shovel 100A. Note that this embodiment shows an example of the virtual projection plane, and the shape of the virtual projection plane is not limited to the shape shown in FIG. 9. For example, the virtual projection plane may be cylindrical.

[0121] Then, the three-dimensional spatial processing unit 801 projects image information 1921 captured by the front camera S6F, image information 1922 captured by the right camera S6R, image information 1923 captured by the rear camera S6B, and image information 1924 captured by the left camera S6L onto the side 1902 of the virtual projection plane 1900.

[0122] Then, the three-dimensional space processing unit 801 generates an overhead image viewed from a viewpoint 1951 for a three-dimensional space including the virtual projection surface 1950 onto which the image information 1921-1924 is projected and the three-dimensional model 1910. As shown in Fig. 9, the viewpoint 1951 is located above (in the +Z axis direction) the center of the three-dimensional model 1910. The viewpoint direction from the viewpoint 1951 is downward (in the -Z axis direction). In this way, the overhead image is an image viewed from the viewpoint 1951 of the image information 1921-1924 projected into the virtual three-dimensional space.

[0123] In this embodiment, the viewpoint and the viewpoint direction can be switched in response to an operation by an operator. When the viewpoint 1951 or the viewpoint direction is switched, the three-dimensional space processing unit 801 generates an image according to the switched viewpoint and viewpoint direction. A specific image will be described later.

[0124] The display control unit 802 controls the display device DI to display a screen. The display control unit 802 according to this embodiment uses an image generated by the three-dimensional space processing unit 801 when displaying a screen.

[0125] Next, a screen displaying the overhead image generated by the three-dimensional space processing unit 801 will be described. Fig. 10 is a diagram showing an example of a screen displayed on the display device DI according to this embodiment. In the screen shown in Fig. 10, the same reference numerals are assigned to the same components as those in the screen shown in Fig. 4, and the description thereof will be omitted.

[0126] 10, the main display area 41B includes an image display area 41n'. The image display area 41n' includes an overhead image display area 41n1', a rear image display area 41n2, and a right image display area 41n3.

[0127] The overhead image display region 41n1' displays an overhead image FV'. The overhead image FV' is a virtual viewpoint image generated by the three-dimensional space processing unit 801 as described above. Furthermore, a top view diagram of a three-dimensional model 1910 is disposed in the center of the overhead image FV'.

[0128] The touch panel sensor S8 according to this embodiment can detect a gesture operation for changing the display mode (e.g., viewpoint) of the overhead image FV' displayed in the overhead image display area 41n1'. Then, the operation processing unit 302 identifies a process corresponding to the detected gesture operation. Then, the display control unit 802 updates the display screen according to the identified process. In this embodiment, the process corresponding to the gesture operation includes a process for changing the viewpoint in a three-dimensional space including the virtual projection plane 1950 and the three-dimensional model 1910.

[0129] In this embodiment, the overhead image FV' can be reduced by moving the viewpoint away from the virtual projection plane 1950, and the overhead image FV' can be enlarged by moving the viewpoint closer to the virtual projection plane 1950. Furthermore, the viewpoint direction may be changed along with the change in the viewpoint position.

[0130] Fig. 11 is a diagram showing a change in an overhead image FV' displayed in an overhead image display region 41n1' by a display control unit 802 according to this embodiment. The display screen shown in Fig. 11 is the same as Fig. 10 except for the overhead image display region 41n1'.

[0131] As shown in FIG. 11(A), an overhead image FV1' is displayed in an overhead image display area 41n1' of a display screen (an example of a first screen) displayed by the display control unit 802. When a swipe operation (an example of a first operation) 2111 for changing the viewpoint on the overhead image display area 41n1' is received, the operation processing unit 302 identifies the swipe operation 2111 as a change of the viewpoint on the overhead image FV1' displayed in the overhead image display area 41n1'. In this embodiment, an example is given in which the swipe operation 2111 on the overhead image display area 41n1' is used as a gesture operation for changing the viewpoint. Note that this embodiment does not limit the gesture operation for changing the viewpoint (an example of a second gesture operation) to a swipe operation, and may be, for example, a pinch-in operation or a pinch-out operation.

[0132] Then, as shown in FIG. 11(B), the display control unit 802 switches the display from a screen including an overhead image FV1' shown in FIG. 11(A) to a screen including an overhead image FV2' in accordance with a swipe operation 2111 on the touch panel sensor S8 (an example of a first operating device).

[0133] 11(B), when an overhead image FV2' is displayed, if a swipe operation 2111 is received on the image display area 41n1, the operation processing unit 302 identifies this as a change in viewpoint for the overhead image FV2' displayed in the image display area 41n1. Then, as shown in FIG. 11(A), the display control unit 802 may display the original overhead image FV1' in the image display area 41n1 as a result of the change in viewpoint.

[0134] Since the overhead image FV2' shown in FIG. 11(B) is an image in which the viewpoint has been changed in accordance with an operation by the operator, there is a possibility that a blind spot will occur in the peripheral display of the excavator 100A.

[0135] Therefore, in this embodiment, when an operation 2112 is performed on the operation device 26 (an example of a second operation device), the operation processing unit 302 specifies the drive control corresponding to the operation.

[0136] When an image from a viewpoint different from the viewpoint defined as the overhead image is displayed in the image display area 41n1, and an operation to drive the shovel 100A is received from the operation device 26, the display control unit 802 performs control to return the display screen to one including the overhead image FV1' shown in Figure 11 (A).

[0137] In this embodiment, the case where the operation device for receiving an operation on the display screen is the touch panel sensor S8 has been described. However, in this embodiment, the operation device for performing an operation to change the viewpoint is not limited to the touch panel sensor S8. For example, an operation device provided with buttons or the like for operating the display device DI may be used.

[0138] In this embodiment, when the image information projected in the virtual three-dimensional space is displayed as image information referenced from a viewpoint, the viewpoint can be changed according to the operation of the operator, so that the operator can easily check the surrounding situation, thereby improving convenience. Furthermore, when an operation is received from the operation device 26, control is performed to return to the display screen including the overhead image FV1', so that the same effect as the above-mentioned embodiment can be achieved.

[0139] (Modification of the second embodiment) In the above-described second embodiment, an example has been described in which the display control unit 802 performs control to return to a screen including the overhead image FV1' when an operation is performed on the operation device 26 (an example of a second operation device). However, the above-described embodiment does not limit the control to return to a screen including the overhead image FV1' when an operation is performed on the operation device 26 (an example of a second operation device). For example, the shovel controller 30 may control to set the overhead image FV1' to a locked state when an operation to change the viewpoint is performed on the overhead image FV1', and to set to an unlocked state when an operation to reset the unlocked state DLU is performed by the gate lock lever DL. The specific procedure is the same as that of the first modification of the first embodiment, and a description thereof will be omitted.

[0140] (Third embodiment) In the above-described embodiment, the control when a gesture operation is received on a screen displayed by a display device provided in a cabin of a shovel has been described. However, the above-described embodiment and modified example are not limited to the case where a gesture operation is received on a screen displayed by a display device provided in a cabin of a shovel. Therefore, in the third embodiment, a case where the present invention is applied to a remote control system for operating a shovel will be described.

[0141] An overview of the remote control system SYS according to the third embodiment will be described with reference to Fig. 12. Fig. 12 is a schematic diagram showing an example of the remote control system SYS according to the third embodiment.

[0142] <Devices that make up the remote control system> 12, the remote operation system SYS according to the third embodiment includes a shovel 100B and a remote operation room RC. The configuration of the shovel 100B is similar to that of the shovel 100A according to the second embodiment, and therefore a description thereof will be omitted.

[0143] The excavator 100B and the remote control room RC are connected via a communication line NT so as to be able to send and receive data.

[0144] The shovel 100B is capable of wireless communication. The shovel 100B is then capable of transmitting and receiving data to and from a device (for example, a remote control room RC) connected to the communication line NT.

[0145] The shovel 100B can then transmit information about the work site to the remote control room RC. This allows the remote control room RC to check the work site according to the information from the shovel 100B. Note that in this embodiment, the device that measures the work site is not limited to the shovel 100B, and may be another type of device, such as a drone that flies over the work site, a fixed camera, or an imaging device that can be carried by a user.

[0146] For example, the shovel 100B is provided with an imaging device S6. The shovel 100B transmits imaging information indicating the imaging results of the work site captured by the imaging device S6 to the remote control room RC.

[0147] The remote operation system SYS may include one or more shovels 100B. This allows the remote operation system SYS to provide information about the work site to the remote operation room RC through the multiple shovels 100B.

[0148] <Example of remote control room configuration> The remote control room RC is an example of a display device for a shovel that displays various information to allow an operator to operate the shovel 100B, and is equipped with a communication device T2, a remote controller R30, an operation device R26, an operation sensor R29, and a display device RDI. Also, an operation seat DS is installed in the remote control room RC, where an operator OP who remotely operates the shovel 100B sits.

[0149] The communication device T2 is configured to control communication with the communication device T1 attached to the excavator 100B.

[0150] The remote controller R30 is a calculation device that executes various calculations. In this embodiment, the remote controller R30 is configured with a microcomputer including a CPU and a memory. The various functions of the remote controller R30 are realized by the CPU executing a program stored in the memory.

[0151] The display device RDI displays a screen based on information transmitted from the shovel 100B so that the operator OP in the remote control room RC can visually confirm the surroundings of the shovel 100B. The display device RDI enables the operator to confirm the status of the work site including the surroundings of the shovel 100B even though he is in the remote control room RC.

[0152] The display device RDI may also be provided with a touch panel sensor RS8, which can detect gesture operations on the display device RDI.

[0153] An operation sensor R29 is connected to the operation device R26 (an example of an operation unit). The operation sensor R29 detects the operation content of the operation device R26. The operation sensor R29 is, for example, an inclination sensor that detects the inclination angle of the operation lever, or an angle sensor that detects the swing angle of the operation lever around the swing axis. The operation sensor R29 may be composed of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor R29 outputs information on the detected operation content of the operation device R26 to the remote controller R30. The remote controller R30 generates an operation signal based on the received information and transmits the generated operation signal to the excavator 100B. The operation sensor R29 may be configured to generate an operation signal. In this case, the operation sensor R29 may output the operation signal to the communication device T2 without passing through the remote controller R30. This makes it possible to remotely operate the excavator 100B from the remote operation room RC.

[0154] The display device RDI displays a display screen similar to that of the above-described embodiment. The remote controller R30 then accepts a gesture operation on the screen displayed on the display device RDI via the touch panel sensor RS8. This allows the screen displayed on the display device RDI to be changed. This allows the remote controller R30 to realize, for example, the switching of the screens shown in FIG. 5 or FIG. 11.

[0155] Then, when the display content of the screen displayed on the display device RDI is changed by a gesture operation, if the remote controller R30 receives an operation signal from the operation sensor R29 indicating that the shovel 100B is to be driven, the remote controller R30 performs control to return the screen displayed on the display device RDI to the original state. Examples of the restored screen are shown in Figs. 5 and 11, and therefore description thereof will be omitted.

[0156] In this embodiment, the display device of the shovel 100B is the remote control room RC. However, this embodiment does not limit the display device of the shovel 100B to the remote control room RC. For example, the display device of the shovel 100B may be a mobile communication terminal such as a PC or a tablet terminal.

[0157] <effect> In the above-described embodiment and modification, when an operation is received from an operating device or a gate lock lever in the shovel or in the remote operation room of the remote operation system, a control is performed to return the screen displayed on the display device to its original state, so that when the shovel 100 is operated, the screen before the operator operates the shovel 100 can be displayed. The screen before the operator operates the shovel 100 is a screen set so that the surroundings of the shovel 100 can be confirmed when the shovel 100 is operated. Therefore, when the shovel 100 is operated, a screen that satisfies a predetermined visibility requirement can be displayed. The visibility requirement is, for example, that people present within a predetermined radius (e.g., 12 m) from the shovel 100 are displayed at a predetermined size (e.g., 10 mm) or more, and all people at a predetermined angle (45 degrees) behind the shovel 100 are displayed on the screen. This allows the operator to properly recognize the surroundings of the shovel 100 when operating the shovel 100, thereby improving safety.

[0158] In the above-described embodiment and modified examples, a case has been described in which a shovel is used as an example of a work machine. However, the configurations shown in the embodiment and modified examples are not limited to being applied to a shovel as a work machine, and may be applied to, for example, a crane, a forklift, etc.

[0159] Although an embodiment showing an example of a shovel and a display device for a shovel has been described above, the present invention is not limited to the above embodiment. Various changes, modifications, substitutions, additions, deletions, and combinations are possible within the scope of the claims. Naturally, these also fall within the technical scope of the present invention. [Explanation of symbols]

[0160] 100 Shovel 1 Undercarriage 2 Swivel mechanism 3. Upper rotating body 4. Boom 5 Arm 6 Bucket 26 Operating device 29 Operation Sensor S6 imaging device S7 Positioning Device S8 Touch Panel Sensor S9 Gate lock switch T1 Communications Equipment DI display device DL Gate lock lever 30 Excavator Controller 301 Acquisition Department 302 Operation Processing Unit 303, 802 Display control unit 801 Three-dimensional spatial processing section RC Remote Control Room T2 Communication Equipment RDI display device R26 operating device R29 Operation Sensor R30 Remote Controller

Claims

1. After displaying a first screen based on image information captured by the imaging device on the display device, switching from the first screen to a second screen based on the image information in accordance with a first operation from a first operation device; a control unit configured to perform control to return to the first screen when a second operation is received from a second operation device, Shovel ready.

2. the second operation is different from an operation for returning the second screen to the first screen; The shovel according to claim 1.

3. the first operation device is a touch panel sensor provided on the display device, the operation for returning the second screen to the first screen is a first gesture operation on the display device. The shovel according to claim 2.

4. the first operation is a second gesture operation on the display device; The shovel according to claim 3.

5. The second operating device is an operating lever for operating the shovel, a traveling operating device for traveling the shovel, or a gate lock lever for switching whether or not the operation of the shovel is locked. The shovel according to claim 1.

6. The control unit performs control to lock the operation of the shovel when the first operation is received from the first operating device, receiving an operation for releasing the lock by the gate lock lever as the second operation; The shovel according to claim 5.

7. The second operation is an operation for operating an attachment or an upper rotating body of the shovel by the operating lever, or an operation for starting traveling of the shovel by the traveling operation device. The shovel according to claim 5.

8. the first operation is an operation for enlarging or reducing the image information displayed on the first screen, or an operation for changing a viewpoint of the image information displayed on the first screen and projected into a virtual three-dimensional space; A shovel according to any one of claims 1 to 7.

9. a first screen displays an overhead image by referring to the image information projected into the virtual three-dimensional space from a viewpoint located above the virtual three-dimensional space; The shovel according to claim 8.

10. After displaying a first screen based on image information captured by an imaging device provided on the shovel, switching from the first screen to a second screen based on the image information in accordance with a first operation from a first operation device; a control unit configured to perform control to return to the first screen when a second operation is received from a second operation device, The shovel is equipped with a display device.

Images