Shovel, display device, and shovel control system

Abstract

The present invention addresses the problem of ascertaining surrounding conditions to improve safety. A shovel according to one embodiment of the present invention is provided with: a lower traveling body; an upper rotating body capable of rotating relative to the lower traveling body; an object detection device which is provided on the upper rotating body and detects an object present around the shovel; an angle detection device for detecting the rotation angle of the upper rotation body with respect to the lower traveling body; a control device configured to acquire the rotation angle, a detection result of the object detection device, and position information of a person present around the shovel; and a display device for simultaneously displaying first display information and second display information in a predetermined display area so as to indicate the positional relationship between the shovel and the person. The first display information indicates the shovel in such a manner that the angle between the upper revolving body and the lower traveling body can be recognized on the basis of the revolving angle, and the second display information indicates the person present at the position indicated by the position information.

Description

Technical Field

[0001] This application claims priority based on Japanese Patent Application No. 2023-187090 filed on October 31, 2023. The entire contents of this Japanese application are incorporated herein by reference.

[0002] The invention relates to an excavator, a display device and a control system of the excavator. Background Art

[0003] Conventionally, there have been proposed technologies for displaying surrounding conditions when working with a shovel. For example, there is a technology in which a display device of the shovel displays the direction in which the shovel is traveling together with an icon representing the shovel (for example, Patent Document 1).

[0004] Patent Document 1: Japanese Patent Application Publication No. 2017-252214

[0005] However, although the conventional technology can recognize the direction in which the shovel is traveling, it is difficult to grasp the positional relationship with people existing around the shovel. Summary of the invention

[0006] One aspect of the present invention is to display the positional relationship between the shovel and a person together with the direction in which the shovel is traveling, thereby understanding the surrounding situation and improving safety.

[0007] An excavator according to one embodiment of the present invention comprises: a lower traveling body; an upper rotating body capable of rotating relative to the lower traveling body; an object detection device, which is arranged on the upper rotating body and detects objects existing around the excavator; an angle detection device, which detects the rotation angle of the upper rotating body relative to the lower traveling body; a control device, which is configured to obtain the rotation angle, the detection result of the object detection device, and position information of a person existing around the excavator; and a display device, which simultaneously displays first display information and second display information in a prescribed display area in a manner indicating the positional relationship between the excavator and the person, the first display information indicating the excavator in a manner capable of identifying the angle between the upper rotating body and the lower traveling body according to the rotation angle, and the second display information indicating the person existing at the position indicated by the position information.

[0008] Effects of the Invention

[0009] According to one aspect of the present invention, surrounding conditions are understood and safety is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a side view of the shovel which concerns on 1st Embodiment.

[0011] Figure 2 It is a top view of the shovel which concerns on 1st Embodiment.

[0012] Figure 3 It is a diagram showing a configuration example of a basic system mounted on the shovel according to the first embodiment.

[0013] Figure 4 This is a diagram showing an example of a display screen displayed by the display device according to the first embodiment.

[0014] Figure 5 It is a diagram showing changes in the shovel icon and the direction display icon displayed on the image display unit according to the first embodiment.

[0015] Figure 6 It is a diagram showing a change in the display form of the shovel state display area displayed by the display device according to the first modification of the first embodiment.

[0016] Figure 7 This is a flowchart showing a processing procedure for displaying a display screen by the controller and the display device according to the first modification of the first embodiment.

[0017] Figure 8 This is a diagram showing an example of a display screen displayed by a display device according to Modification 2 of the first embodiment.

[0018] Fig. 9 This is a diagram showing an example of a display screen displayed by a display device according to Modification 3 of the first embodiment.

[0019] Fig.10 It is a schematic diagram showing a configuration example of a remote support system for a shovel according to a second embodiment.

[0020] In the figure: 100-excavator, 1-lower walking body, 2-slewing mechanism, 3-upper slewing body, 4-boom, 5-arm, 6-bucket, S6-camera, T1-communication device, 30-controller, 30a-acquisition unit, 30b-detection unit, 30c-position estimation unit, 30d-output control unit, 40-display device, 40a-control unit, 42

[0021] -Image display unit, 43-operation unit, RC-remote operation room, R30-remote controller, T2-communication device, DR-display device. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, the embodiments described below do not limit the invention but are illustrative, and all features and combinations thereof described in the embodiments are not necessarily the essence of the invention. In addition, in each of the drawings, the same or corresponding structures are marked with the same or corresponding symbols, and the description is sometimes omitted.

[0023] In the following, in the embodiment of the present invention, an example using an excavator as an example of a construction machine is described, but the present invention is not limited to an excavator and can also be applied to construction machinery, standard machines, application machines, forestry machinery, or transport machinery based on a hydraulic excavator.

[0024] (First embodiment)

[0025] First, refer to Figure 1 , Figure 2 , an overview of the shovel 100 according to the present embodiment will be described. Figure 1 , Figure 2 They are respectively a plan view and a side view of the shovel 100 according to the present embodiment.

[0026] like Figure 1 , Figure 2 As shown in FIG. 1 , the shovel 100 according to the present embodiment includes a lower traveling body 1, an upper swing body 3 that is rotatably mounted on the lower traveling body 1 via a swing mechanism 2, an attachment AT for performing various operations, and a cab 10. Hereinafter, when the shovel 100 is viewed from directly above along the swing axis of the upper swing body 3 (from above), the front of the shovel 100 (upper swing body 3) corresponds to the direction in which the attachment extends relative to the upper swing body 3. Furthermore, the left and right of the shovel 100 (upper swing body 3) correspond to the left and right, respectively, as viewed from the operator sitting on the operating seat in the cab 10.

[0027] The lower traveling body 1 includes, for example, a pair of left and right crawler tracks 1C. Specifically, the crawler tracks 1C include a left crawler track 1CL and a right crawler track 1CR. The lower traveling body 1 causes the shovel 100 to travel by hydraulically driving the left crawler track 1CL and the right crawler track 1CR by the left traveling hydraulic motor 2ML and the right traveling hydraulic motor 2MR, respectively.

[0028] The upper swing body 3 swings relative to the lower traveling body 1 when the swing mechanism 2 is hydraulically driven by the swing hydraulic motor 2A. That is, the swing hydraulic motor 2A is a swing driving unit that drives the upper swing body 3 as a driven unit and can change the direction of the upper swing body 3.

[0029] The attachment AT (an example of the attachment) includes a boom 4 , an arm 5 , and a bucket 6 .

[0030] The boom 4 is installed at the front center of the upper rotating body 3 so as to be able to pitch. The arm 5 is installed at the front end of the boom 4 so as to be able to rotate up and down. The bucket 6 is installed at the front end of the arm 5 so as to be able to rotate up and down.

[0031] The bucket 6 is an example of a working tool. The bucket 6 is used, for example, for excavation work. In addition, other working tools may be installed at the front end of the boom 5 instead of the bucket 6, depending on the work content. Other working tools may be, for example, large buckets, buckets for slopes, buckets for dredging, and other types of buckets. In addition, other working tools may be working tools other than buckets, such as mixers, crushers, and grabs.

[0032] The boom 4 , the arm 5 , and the bucket 6 are hydraulically driven by a boom cylinder 7 , an arm cylinder 8 , and a bucket cylinder 9 as hydraulic actuators, respectively, by hydraulic oil discharged from a main pump 14 (described later).

[0033] In addition, the excavator 100 may be a structure in which a part of the driven elements such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5 and the bucket 6 are electrically driven. That is, the excavator 100 may be a hybrid excavator or an electric excavator in which a part of the driven elements are driven by an electric actuator.

[0034] The imaging device S6 is an example of an object detection device, is provided on the upper swing body 3 , detects objects around the shovel 100 , and outputs the detection result to the controller 30 .

[0035] The camera device S6 is provided on the upper swing body 3, captures objects existing around the shovel 100, and outputs image information indicating the objects existing around the shovel 100 to the controller 30. The camera device S6 includes a camera S6F for capturing images of the front of the shovel 100, a camera S6L for capturing images of the left side of the shovel 100, a camera S6R for capturing images of the right side of the shovel 100, and a camera S6B for capturing images of the rear side of the shovel 100.

[0036] The camera S6F is installed on the roof of the cab 10, the side of the boom 4, or the outside of the cab 10. In addition, the camera S6F may be installed, for example, on the ceiling of the cab 10, that is, inside the cab 10. The camera S6L is installed on the left end of the upper surface of the upper rotating body 3, the camera S6R is installed on the right end of the upper surface of the upper rotating body 3, and the camera S6B is installed on the rear end of the upper surface of the upper rotating body 3.

[0037] The camera S6 (cameras S6F, S6B, S6L, and S6R) is, for example, a single-lens wide-angle camera having a very wide field of view. The camera S6 has, for example, a camera element such as a CCD or a CMOS, and outputs the captured image to the display device 40. In addition, the image information captured by the camera S6 is also input to the controller 30.

[0038] The imaging device S6 according to the present embodiment is an example of an object detection device, and any device may be used as long as it can detect an object.

[0039] As an object detection device, in order to detect objects existing around the excavator 100, LIDAR (laser radar) can be used. LIDAR, for example, measures the distance between more than 1 million points within the monitoring range and the LIDAR. In addition, the present embodiment is not limited to the method using LIDAR, as long as it is an object detection device that can measure the distance to the object. For example, a stereo camera may be included, and a distance measuring device such as a range image camera or a millimeter wave radar may also be used. In the case of using a millimeter wave radar or the like as an object detection device, multiple signals (lasers, etc.) are emitted from the object detection device toward the object, and by receiving the reflected signal, the distance and direction of the object can be derived from the reflected signal.

[0040] Furthermore, as the object detection device, two or more devices may be combined. For example, a camera device may be combined with a LIDAR, a camera device may be combined with a millimeter wave radar, or a camera device may be combined with a stereo camera.

[0041] The controller (control device) 30 is a control device for controlling the excavator 100. For example, the controller 30 is mainly composed of a microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile auxiliary storage medium, various input and output interfaces, etc. In addition, the controller 30 reads a program from a non-volatile storage medium and loads it to a volatile storage medium, and realizes various functions by causing the CPU to execute it. The various functions include, for example, a device guidance function for guiding (guiding) the operator's manual operation of the excavator 100. The controller 30 may include a contact avoidance function for automatically or autonomously operating or stopping the excavator 100 in order to avoid contact between the excavator 100 and an object within a monitoring range existing around the excavator 100.

[0042] For example, the controller 30 sets a target rotation speed in accordance with an operation by an operator or the like, and performs drive control to rotate the engine 11 at a constant speed.

[0043] The boom angle sensor S1 is installed on the boom 4, and detects the pitch angle of the boom 4 relative to the upper rotating body 3 (hereinafter, "the boom angle"), for example, the angle formed by the straight line connecting the fulcrums at both ends of the boom 4 relative to the rotation plane of the upper rotating body 3 in a side view. The boom angle sensor S1 may include, for example, a rotary encoder, an acceleration sensor, a 6-axis sensor, an IMU (Inertial Measurement Unit), etc. In addition, the boom angle sensor S1 may also include a potentiometer using a variable resistor, a cylinder stroke sensor that detects the stroke amount of the hydraulic cylinder (boom cylinder 7) corresponding to the boom angle, etc. The same applies to the arm angle sensor S2 and the bucket angle sensor S3 below. The detection signal corresponding to the boom angle based on the boom angle sensor S1 is input to the controller 30.

[0044] The arm angle sensor S2 is mounted on the arm 5, and detects the rotation angle of the arm 5 relative to the boom 4 (hereinafter referred to as the "arm angle"), for example, the angle formed by a straight line connecting the fulcrums at both ends of the arm 5 relative to a straight line connecting the fulcrums at both ends of the boom 4 in a side view. A detection signal corresponding to the arm angle based on the arm angle sensor S2 is input to the controller 30.

[0045] The bucket angle sensor S3 is mounted on the bucket 6, and detects the rotation angle of the bucket 6 relative to the boom 5 (hereinafter referred to as the "bucket angle"), for example, the angle formed by a straight line connecting the fulcrum and the front end (shovel tip) of the bucket 6 relative to a straight line connecting the fulcrums at both ends of the boom 5 in a side view. A detection signal corresponding to the bucket angle based on the bucket angle sensor S3 is input to the controller 30.

[0046] The body tilt sensor S4 detects the tilt state of the body (the upper rotating body 3 or the lower walking body 1) relative to the horizontal plane. The body tilt sensor S4 is, for example, installed on the upper rotating body 3, and detects the tilt angles of the two axes of the excavator 100 (i.e., the upper rotating body 3) around the front-to-back direction and the left-to-right direction (hereinafter, "front-to-back tilt angle" and "left-to-right tilt angle"). The body tilt sensor S4 may include, for example, a rotary encoder, an acceleration sensor, a 6-axis sensor, an IMU, etc. The detection signal corresponding to the tilt angle (front-to-back tilt angle and left-to-right tilt angle) based on the body tilt sensor S4 is input to the controller 30.

[0047] The rotation angle sensor (an example of an angle detection device) S5 outputs detection information related to the rotation state of the upper rotating body 3 relative to the lower traveling body 1. The rotation angle sensor S5, for example, detects the rotation angle of the upper rotating body 3 relative to the lower traveling body 1. In addition, the rotation angle sensor S5 can also detect the rotation angular velocity of the upper rotating body 3 relative to the lower traveling body 1. The rotation angle sensor S5 can include, for example, a gyro sensor, a rotary transformer, a rotary encoder, etc. The detection signal corresponding to the rotation angle and the rotation angular velocity of the upper rotating body 3 based on the rotation angle sensor S5 is input to the controller 30.

[0048] The positioning device PS measures the position and orientation of the upper rotating body 3. The positioning device PS is, for example, a GNSS (Global Navigation Satellite System) compass, which detects the position and orientation of the upper rotating body 3, and a detection signal corresponding to the position and orientation of the upper rotating body 3 is input to the controller 30. In addition, the function of detecting the orientation of the upper rotating body 3 among the functions of the positioning device PS can also be replaced by an orientation sensor installed on the upper rotating body 3.

[0049] The cab 10 is a control room in which an operator rides, and is mounted on the front left side of the upper swing body 3 .

[0050] In addition, when the shovel 100 is remotely operated or works by fully automatic operation, the cab 10 may be omitted.

[0051] The communication device T1 communicates with external devices through a prescribed network including a mobile communication network with a base station as a terminal, a satellite communication network, an Internet network, etc. The communication device T1 is, for example, a mobile communication module corresponding to mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), a satellite communication module for connecting to a satellite communication network, etc.

[0052] The shovel 100 operates an actuator (e.g., a hydraulic actuator) according to an operation of an operator riding in the cab 10 to drive operating elements (hereinafter referred to as "driven elements") such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5 and the bucket 6.

[0053] Furthermore, the shovel 100 may be configured to be remotely operable (remote control) from the outside of the shovel 100 instead of being operable by the operator of the cab 10, or in addition to being operable by the operator of the cab 10, it may be configured to be remotely operable (remote control) from the outside of the shovel 100. When the shovel 100 is remotely operated, the interior of the cab 10 may be unmanned.

[0054] Furthermore, the excavator 100 can automatically operate the actuators regardless of the operator's operation content. Thus, the excavator 100 realizes the function of automatically operating at least a part of the driven elements such as the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5 and the bucket 6, that is, the so-called "automatic operation function" or "equipment control function".

[0055] The automatic operation function may include a function of automatically operating a driven element (actuator) other than the driven element (actuator) of the operation object according to the operator's operation of the operation device 26 or remote operation, that is, the so-called "semi-automatic operation function" or "operation support type equipment control function". In addition, the automatic operation function may include a function of automatically operating at least a part of a plurality of driven elements (hydraulic actuators) without the operator's operation of the operation device 26 or remote operation, that is, the so-called "full automatic operation function" or "full automatic equipment control function". In the excavator 100, when the full automatic operation function is effective, the interior of the cab 10 may be unmanned. In addition, in the semi-automatic operation function, the full automatic operation function, etc., a method in which the action content of the driven element (actuator) of the automatic operation object is automatically determined according to a predetermined rule may be included. Furthermore, the semi-automatic operation function, the fully automatic operation function, etc. may include a method in which the shovel 100 autonomously performs various judgments and autonomously determines the action content of the driven element (hydraulic actuator) of the automatic operation object based on the judgment result (the so-called "automatic operation function").

[0056] Next, refer to Figure 3 , a basic system mounted on the shovel 100 will be described. Figure 3 1 is a diagram showing a configuration example of a basic system mounted on the shovel 100 according to the present embodiment. Figure 3 In the figure, the mechanical power transmission line is represented by a double line, the working oil pipeline route is represented by a thick solid line, the pilot pipeline route is represented by a dotted line, the power line is represented by a thin solid line, and the electrical control line is represented by a single-point dashed line.

[0057] The basic system has a controller 30 and an ECU (engine control unit) 74. The basic system has an engine 11, a main pump 14, a pilot pump 15, a control valve unit 17, and a switching valve 18 as control objects. The basic system has an operating device 26, an operating pressure sensor 29, a control panel 32, a switch 35, a door lock rod 55, and a key sensor 62 as input parts. The basic system has an alarm device 49 as an output part. The basic system has a display device 40 as a display part. The display device 40 serves as both an input part and an output part.

[0058] The engine 11 is a diesel engine that adopts isochronous control to maintain the engine speed constant regardless of load increase or decrease. The fuel injection amount, fuel injection timing, supercharging pressure, etc. in the engine 11 are controlled by the ECU 74.

[0059] The engine 11 is connected to a main pump 14 and a pilot pump 15, which are hydraulic pumps. The main pump 14 is connected to a control valve unit 17 via a hydraulic oil line.

[0060] The control valve unit 17 is a hydraulic control device that controls the hydraulic system of the shovel 100. The control valve unit 17 is connected to hydraulic actuators such as the left travel hydraulic motor, the right travel hydraulic motor, the boom cylinder 7, the arm cylinder 8, the bucket cylinder 9, and the swing hydraulic motor.

[0061] The control valve unit 17 includes a plurality of slide valves corresponding to each hydraulic actuator. Each slide valve is displaceably configured according to the pilot pressure so as to be able to increase or decrease the opening area of ​​the PC port and the opening area of ​​the CT port. The PC port is a port that connects the main pump 14 to the hydraulic actuator. The CT port is a port that connects the hydraulic actuator to the working oil tank.

[0062] The pilot pump 15 is connected to the operating device 26 via a pilot line. The operating device 26 includes, for example, a left operating lever, a right operating lever, and a travel operating device. The travel operating device includes, for example, a left travel pedal, a right travel pedal, a left travel rod, and a right travel rod. Each operating device 26 is a hydraulic operating device, which is connected to the pilot port of the corresponding slide valve in the control valve unit 17 via a pilot line. However, the operating device 26 may also be an electrical operating device.

[0063] The pilot pump 15 may be omitted. In this case, the function of the pilot pump 15 may be realized by the main pump 14. That is, in addition to the function of supplying the working oil to the control valve unit 17, the main pump 14 may also have the function of supplying the working oil to the operating device 26 etc. after the pressure of the working oil is reduced by a throttle etc.

[0064] The operation pressure sensor 29 detects the operation content of the operation device 26 in the form of pressure. The operation pressure sensor 29 outputs the detection value to the controller 30. However, the operation content of the operation device 26 may be detected electrically.

[0065] The switching valve 18 is configured to be able to switch between an effective state and an invalid state of the operating device 26. The effective state of the operating device 26 is a state in which the operator can operate the hydraulic actuator using the operating device 26. The invalid state of the operating device 26 is a state in which the operator cannot operate the hydraulic actuator using the operating device 26. The switching valve 18 is a door lock valve configured in a manner to work according to a command from the controller 30. The switching valve 18 is configured to be arranged on a pilot line connecting the pilot pump 15 and the operating device 26, and can switch the disconnection / connection of the pilot line according to a command from the controller 30. The operating device 26 becomes effective when, for example, the door lock rod 55 is pulled up to open the switching valve 18 (door lock valve), and becomes invalid when the door lock rod 55 is pressed down to close the switching valve 18 (door lock valve).

[0066] The battery 70 is charged by, for example, the electric power generated by the alternator 11 a . The electric power of the battery 70 is also supplied to the controller 30 and the like. For example, the starter 11 b of the engine 11 is driven by the electric power from the battery 70 , and the engine 11 is started.

[0067] The ECU 74 transmits data related to the state of the engine 11, such as the cooling water temperature, to the controller 30. The regulator 14a of the main pump 14 transmits data related to the deflection angle of the swash plate to the controller 30. The discharge pressure sensor 14b transmits data related to the discharge pressure of the main pump 14 to the controller 30. The oil temperature sensor 14c provided in the pipeline between the working oil tank and the main pump 14 transmits data related to the temperature of the working oil flowing through the pipeline to the controller 30. The operating pressure sensor 29 transmits data related to the pilot pressure generated when the operating device 26 is operated to the controller 30. The controller 30 stores these data in a temporary storage unit (memory) and can transmit them to the display device 40 when necessary.

[0068] The alarm device 49 is a device for alerting the operator of the excavator 100. The alarm device 49 is composed of, for example, a combination of an indoor alarm device and an outdoor alarm device. The indoor alarm device is a device for alerting the operator of the excavator 100 in the cab 10, and includes, for example, at least one of a sound output device, a vibration generating device, and a light emitting device provided in the cab 10. The indoor alarm device may be a display device 40. The outdoor alarm device is a device for alerting the operator working around the excavator 100, and includes, for example, at least one of a sound output device and a light emitting device provided outside the cab 10. The sound output device as the outdoor alarm device includes, for example, a walking alarm device installed on the bottom surface of the upper rotating body 3. In addition, the outdoor alarm device may be a light emitting device provided on the upper rotating body 3. However, the outdoor alarm device may be omitted. For example, when the camera S6 functioning as an object detection device detects a predetermined object, the notification device 49 may notify the operator of the excavator 100 of the situation.

[0069] The key sensor 62 is a sensor that detects an electronic key of the shovel 100. The key sensor 62 detects an electronic key that exists near the key sensor 62. The key sensor 62 can detect, for example, an electronic key that exists in the cab 10. The key sensor 62 outputs a signal indicating that the electronic key has been detected to the controller 30. The key sensor 62 can detect the electronic key using, for example, short-range wireless communication such as Bluetooth (registered trademark).

[0070] The display device 40 displays various information. The display device 40 can be connected to the controller 30 via a communication network such as CAN, or can be connected to the controller 30 via a dedicated line. The display device 40 can display one or more image information captured by the camera included in the camera device S6, and a display area indicating the positional relationship between the shovel 100 and the detected person. In addition, the display device 40 can display a menu screen, etc. The display device 40 receives power supply from the battery 70 to operate. The display device 40 includes a control unit 40a, an image display unit 42, and an operation unit 43.

[0071] The control unit 40a controls the image displayed on the image display unit 42. The control unit 40a is a computer equipped with a CPU, RAM, NVRAM, ROM, etc. The control unit 40a reads the program corresponding to each functional element from the ROM and reads it into the RAM, and causes the CPU to execute the corresponding processing. Among them, each functional element can be composed of hardware or a combination of software and hardware. In addition, the image displayed on the image display unit 42 can also be controlled by the controller 30 or the camera device S6.

[0072] The image display unit 42 can display a menu screen when the operating unit 43 receives a menu display operation even when the excavator 100 is in any of the operable state and the inoperable state. The image display unit 42 can also display a menu screen when the operating unit 43 receives a menu display operation only when the excavator 100 is in the inoperable state. And, for example, it is configured in a manner that these states can be switched by a switching member such as a switching switch. The operable state of the excavator 100 refers to a state in which the operating device 26 is enabled, for example, by opening the switching valve 18 by pulling up the door lock rod 55. The state of the door lock rod 55 at this time is called a lock release state. The inoperable state of the excavator 100 refers to a state in which the operating device 26 is disabled, for example, by closing the switching valve 18 by pressing the door lock rod 55. The state of the door lock rod 55 at this time is called a locked state. Next, the display after the transition to the lock release state is explained.

[0073] The image display unit 42 displays image information captured by at least one of the cameras S6 and a shovel state display area indicating the positional relationship between the shovel 100 and a person together with the state of the shovel 100 under the control of the control unit 40a. Specific display contents of the shovel state display area will be described later.

[0074] Furthermore, the image display unit 42 according to the present embodiment is an example in which the image display unit 42 displays the rear image captured by the camera S6B and the right image captured by the camera S6R among the image information captured by the imaging device S6.

[0075] In addition, the present embodiment does not limit the display method of the image information captured by the camera S6. The image information displayed on the image display unit 42 may be, for example, any one of the front image captured by the camera S6F, the rear image captured by the camera S6B, the left image captured by the camera S6L, and the right image captured by the camera S6R. Furthermore, the image information may be two or more images selected from the front image, the rear image, the left image, and the right image. Furthermore, in the image display unit 42, the shovel state display area indicating the positional relationship between the shovel 100 and the detected person may be displayed instead of the image information captured by the camera S6.

[0076] The operation unit 43 is a touch panel provided in the display area of ​​the image display unit 42. The operation unit 43 receives an operation (for example, an operation on a menu screen) for changing the screen displayed on the image display unit 42. The information that the operation unit 43 receives using the touch panel is not particularly limited.

[0077] In addition, the present embodiment does not limit the operating unit 43 to a touch panel, but may be a switch panel including hardware. In addition, the operating unit 43 may be a switch panel, which means that the display device 40 may be a structure in which the operating unit 43 is changed to a touch panel and input is possible using physical buttons. Moreover, the operating unit 43 may also be a combination of a touch panel and a physical button. The operating unit 43 is not limited to being arranged on the display device 40, for example, it may be arranged on an operating lever, or it may be arranged on the left side console of the seat or the right side console of the seat on the left and right sides of the driver's seat. Here, in addition to the operating unit 43 arranged on the display device 40, a driver's seat side operating unit having the same function as the operating unit 43 may also be arranged on at least one of the operating lever, the left side console of the seat, and the right side console of the seat.

[0078] The display device 40 is just an example, and is not limited to the display device described here. Furthermore, the shovel 100 is not limited to include the display device 40. A person skilled in the art can design the display device 40 as appropriate.

[0079] Furthermore, the display device 40 is not limited to a device pre-installed in the cab 10, and may be a display that can be installed separately. Furthermore, the display device 40 may be any device that can display, and for example, a tablet terminal that can communicate with the communication device T1 may be used.

[0080] [Functional structure of the controller]

[0081] The controller 30 (an example of a control device) is, for example, installed in the cab 10 to control the driving of the excavator 100. The functions of the controller 30 can be realized by any hardware, software, or a combination thereof. For example, the controller 30 is composed of a microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile auxiliary storage medium, various input and output interfaces, etc. The controller 30 realizes various functions by, for example, executing various programs stored in the ROM or the non-volatile auxiliary storage medium on the CPU.

[0082] For example, the controller 30 sets a target rotation speed in accordance with an operation by an operator or the like, and performs drive control to rotate the engine 11 at a constant speed.

[0083] Furthermore, for example, the controller 30 outputs a control command to the regulator 14 a as necessary to change the discharge rate of the main pump 14 .

[0084] Furthermore, for example, the controller 30 performs control related to a device guidance function for guiding the operator's manual operation of the shovel 100 through the operating device 26. Furthermore, for example, the controller 30 performs control related to a device control function for automatically supporting the operator's manual operation of the shovel 100 through the operating device 26.

[0085] In addition, part of the functions of the controller 30 may be implemented by other controllers (control devices). That is, the functions of the controller 30 may be implemented in a distributed manner by multiple controllers. For example, the device guidance function and the device control function may be implemented by a dedicated controller (control device).

[0086] Return to Figure 3 , the structure of the controller 30 controlling according to the detected person is described. The controller 30 implements the acquisition unit 30a, the detection unit 30b, the position estimation unit 30c and the output control unit 30d as the device guidance function and the device control function by executing a program stored in a storage medium not shown.

[0087] The acquisition unit 30a acquires detection results from various sensors provided in the shovel 100. For example, the acquisition unit 30a acquires image information indicating the imaging result from the imaging device S6 via the display device 40. Furthermore, the acquisition unit 30a acquires the rotation angle of the upper rotating body 3 from the rotation angle sensor (an example of an angle detection device) S5.

[0088] The detection unit 30b performs a detection process of a person existing around the excavator 100 based on the image information acquired by the acquisition unit 30a. As for the method of detecting a person, any known method or any other method can be used. For example, it can be determined whether the feature quantity extracted from the image information is approximately equal to or greater than a predetermined value compared to the feature quantity representing a person that is predetermined in advance. In addition, when the detection unit 30b detects a person, it outputs information indicating the area where the person is photographed in the image information (for example, position coordinates).

[0089] When a person is detected by the detection unit 30b, the position estimation unit 30c estimates the position of the person in the actual space, for example, estimates the direction and distance of the person with respect to the shovel 100. For example, the position estimation unit 30c can estimate the direction and distance of the person based on the position and size of the person captured in the image information. In addition, when a LIDA R, a distance image camera, or a millimeter wave radar is mounted as an object detection device in addition to the camera S6, the direction and distance of the person can be estimated based on the detection result based on the object detection device. The position estimation unit 30c involved in this embodiment obtains the direction and distance of the person with respect to the shovel 100 as an example of the position information of the person existing around the shovel, but the position information of the person is not limited to the direction and distance of the person with respect to the shovel 100, and for example, information showing the position coordinates of the person in the world coordinate system can also be obtained.

[0090] In the present embodiment, the detection of a person and the estimation of the position of the person are not limited to the above-mentioned method, and the detection of a person and the estimation of the position of the person (for example, direction and distance) may be performed using a learned model.

[0091] For example, the learned model is constructed around a neural network. The learned model can use a so-called deep neural network with one or more intermediate layers (hidden layers) between the input layer and the output layer. The neural network specifies a weighting parameter representing the strength of connection with the lower layer for each of the multiple neurons constituting each intermediate layer. Moreover, the neurons of each layer construct the neural network in such a way that the sum of the values ​​obtained by multiplying each input value from the multiple neurons of the upper layer by the weighting parameter specified for each of the neurons of the upper layer is output to the neurons of the lower layer through a threshold function.

[0092] Furthermore, the learned model is subjected to machine learning, specifically, deep learning, which results in the optimization of the weighted parameters of the neural network.

[0093] The training data used in machine learning includes, for example, image information, information indicating whether a person is captured in the image information, information indicating the area in which the person is captured in the image information, and information related to the position of the person in the actual space (e.g., direction and distance). By using the training data for machine learning, the learned model can output information indicating whether a person is captured in the image information, the area in which the person is captured in the image information, and the position of the person (e.g., direction and distance) when the image information is input. In this way, both known methods and any other methods can be used for detecting people and inferring their positions.

[0094] In addition, the present embodiment is not limited to the method of detecting a person and estimating the position of the person by the controller 30, and for example, the method may be performed within the camera S6 or using an external cloud service.

[0095] The output control unit 30d outputs the rotation angle, information indicating the area where the person is photographed in the image information, position information of the person existing around the shovel 100 (information indicating the direction and distance of the detected person), and the detection results of various sensors to the control unit 40a of the display device 40. As a result, the display device 40 displays a screen showing the surroundings of the shovel 100 on the image display unit 42.

[0096] Next, refer to Figure 4 , an example of a display screen 85 displayed on the image display unit 42 is described. Figure 4 1 is a diagram showing an example of a display screen 85 displayed on the display device 40 according to the present embodiment. The display screen 85 is displayed on the image display unit 42 during the operation of the shovel 100 .

[0097] The control unit 40a according to the present embodiment generates a display screen 85 based on the image information input from the camera S6 and various information received from the controller 30. The information received from the controller 30 includes a rotation angle, information indicating an area where a person is photographed in the image information, position information of a person existing around the excavator (information indicating the direction and distance of the detected person), and detection results of various sensors.

[0098] In the image display unit 42, under the control of the control unit 40a, a display screen 85 including a date and time display area 42a, a travel mode display area 42b, an attachment display area 42c, a fuel consumption rate display area 42d, an engine control state display area 42e, an engine operation time display area, a cooling water temperature display area 42g, a fuel remaining amount display area 42h, a rotation speed level display area 42i, a urea water remaining amount display area 42j, a working oil temperature display area 42k, an excavator state display area 421, a first image display area 422, and a second image display area 423 is displayed. The display screen 85 may also include other display areas.

[0099] The travel mode display area 42b, the attachment display area 42c, the engine control state display area 42e, and the rotation speed level display area 42i are areas for displaying setting state information, which is information related to the setting state of the shovel 100. The fuel consumption rate display area 42d, the engine operation time display area, the cooling water temperature display area 42g, the fuel remaining amount display area 42h, the urea water remaining amount display area 42j, and the operating oil temperature display area 42k are areas for displaying information indicating the operating state of the shovel 100, which is information based on the detection results of various sensors, namely, operating state information.

[0100] The date and time display area 42a is an area for displaying the current date and time. The travel mode display area 42b is an area for displaying the current travel mode. The attachment display area 42c is an area for displaying an image representing the attachment currently installed. The fuel consumption rate display area 42d is an area for displaying fuel consumption rate information calculated by the controller 30. The fuel consumption rate display area 42d includes an average fuel consumption rate display area 42d1 for displaying the life average fuel consumption rate or the zone average fuel consumption rate and an instantaneous fuel consumption rate display area 42d2 for displaying the instantaneous fuel consumption rate.

[0101] The engine control state display area 42e is an area for displaying the control state of the engine 11. The engine operation time display area is an area for displaying the accumulated operation time of the engine 11. The cooling water temperature display area 42g is an area for displaying the current temperature state of the engine cooling water. The fuel remaining amount display area 42h is an area for displaying the remaining amount state of the fuel stored in the fuel tank.

[0102] The rotation speed level display area 42i is an area that displays the current level set by the control dial 32 by an image. The number indicating the selected level is displayed in the rotation speed level display area 42i. "1" displayed in the rotation speed level display area 42i indicates that the selected rotation speed level is "level 1". The number "n" displayed in the rotation speed level display area 42i indicates that the selected rotation speed level is "level n". "n" is a natural number. If the operator rotates the control dial 32, the number displayed on the rotation speed level display area 42i changes.

[0103] The urea water remaining amount display area 42j is an area for displaying the remaining amount of urea water stored in the urea water tank by an image. The hydraulic oil temperature display area 42k is an area for displaying the temperature state of the hydraulic oil in the hydraulic oil tank.

[0104] The shovel state display area 421 is an area that displays information on the positional relationship between the shovel 100 and a person detected around the shovel 100 .

[0105] The shovel state display area (an example of a predetermined display area) 421 is a display area that displays the actual space centered on the shovel 100 at a predetermined scale ratio. The shovel state display area (an example of a predetermined display area) 421 has a shovel icon 421b indicating the presence of the shovel 100 arranged at the center of the area.

[0106] In the shovel state display area (an example of a predetermined display area) 421, in addition to the shovel icon (an example of the first display information) 421b indicating the shovel 100, an icon ( Figure 4 In the example of FIG. 1 , the direction display icon 421 a and the icon indicating a person detected from the vicinity of the shovel 100 are shown. Figure 4 The excavator state display area 421 may be, for example, a region (in other words, a background) other than the excavator icon 421b, the direction display icon 421a, and the human detection icons 421e, 421f, and 421g, represented by a single color (e.g., black).

[0107] The shovel icon (an example of the first display information) 421b is an icon in which an image representing the upper swing body 3 and an image representing the lower traveling body 1 are combined according to the positional relationship between the upper swing body 3 and the lower traveling body 1 based on the swing angle.

[0108] The direction display icon (an example of the third display information) 421a is a triangle shape that indicates the direction in which the excavator 100 travels when the travel rod is tilted forward. In addition, this embodiment shows an example of an icon that indicates the direction in which the excavator 100 travels when the travel rod is tilted forward, and any shape may be used as long as it indicates the direction in which the excavator 100 can travel.

[0109] The human detection icons (an example of the second display information) 421e, 421f, and 421g are icons representing the human detected based on the image information captured by the camera S6. Specifically, the human detection icons (an example of the second display information) 421e, 421f, and 421g are arranged based on the position information of the human received from the controller 30. For example, the human detection icons (an example of the second display information) 421e, 421f, and 421g are arranged at a position where the direction and distance of the detected human being are multiplied by a predetermined scale ratio with respect to the shovel 100.

[0110] In this way, the positional relationship between the shovel icon 421 b and the human detection icons 421 e , 421 f , and 421 g corresponds to the positional relationship between the shovel 100 and the human beings existing around the shovel 100 in the real space.

[0111] However, there is a technology that displays the conditions around the excavator through a bird's-eye view image or the like. The bird's-eye view image is generated by, for example, synthesizing image information captured by a camera installed on the excavator. In this bird's-eye view image, in addition to the excavator and the people detected around the excavator, objects existing around the excavator are also displayed. In addition, the ground conditions are also displayed. Therefore, when the operator of the excavator refers to the bird's-eye view image, it is sometimes difficult to understand the people existing around the excavator because of the recognition of information related to various objects.

[0112] In contrast, in the present embodiment, an icon indicating the shovel 100 (e.g., shovel icon 421b), an icon indicating the possible travel direction of the shovel 100 (e.g., direction display icon 421a), and icons indicating people detected around the shovel 100 (e.g., person detection icons 421e, 421f, 421g) are displayed in the shovel state display area (an example of the prescribed display area) 421. In other words, the display of objects other than people existing around the shovel is suppressed in the shovel state display area (an example of the prescribed display area) 421.

[0113] That is, the operator can recognize the current status and the possible travel direction of the shovel 100 , and the positional relationship with people existing around the shovel 100 by referring to the shovel state display area 421 .

[0114] Furthermore, the operator can infer how the positional relationship between the shovel 100 and the people around it changes when the shovel 100 is moved by referring to the shovel state display area 421. Furthermore, since the display of objects other than the shovel and the people is suppressed in the shovel state display area 421, the operator can be prevented from being distracted by other objects and forgetting the existence of the people. Therefore, safety can be improved.

[0115] Furthermore, in the shovel state display area (an example of a prescribed display area) 421, with the shovel 100 as a reference, a first circular area (an example of a first area) 421c and a second circular area (an example of a first area) 421d determined according to the distance from the shovel 100 are displayed.

[0116] Figure 4 The first circular region (an example of the first region) 421c and the second circular region (an example of the first region) 421d shown are shown as circles for allowing the operator to recognize the relative distance from the shovel 100 based on the shovel icon 421b.

[0117] The first circular region 421 c is information indicating a range in which the attachment AT of the shovel 100 is currently rotating, based on the detection results of the boom angle sensor S1 , the arm angle sensor S2 , and the bucket angle sensor S3 .

[0118] The second circular area 421d is information indicating the range in which the attachment AT rotates when the attachment AT is extended to the maximum in the horizontal direction.

[0119] The image display unit 42 according to the present embodiment displays the first circular area 421c and the second circular area 421d. That is, the positional relationship between the person and the accessory AT can be identified based on the positional relationship between the first circular area 421c and the second circular area 421d and the person detection icons 421e, 421f, and 421g. Therefore, the operator can easily perform an operation to prevent the person from contacting the accessory AT, thereby improving safety.

[0120] Furthermore, the control unit 40a displays the human detection icons 421e, 421f, and 421g in different ways depending on whether they are included in the first circular area (an example of the first area) 421c and the second circular area (an example of the first area) 421d.

[0121] For example, the human detection icon 421e existing in the first circular area 421c is displayed in red, for example. And, the human detection icon 421f existing outside the first circular area 421c and in the second circular area 421d is displayed in yellow, for example. And, the human detection icon 421g existing outside the second circular area 421d is displayed in blue, for example. The control unit 40a involved in this embodiment makes the display method different depending on whether it is included in the first circular area (an example of the first area) 421c and the second circular area (an example of the first area) 421d. In other words, the color of the human detection icon changes according to the distance from the excavator 100. In this way, the display device 40 can remind the operator to pay attention according to the distance between the excavator 100 and the person by displaying the human detection icon whose color changes according to the distance. Therefore, it is possible to achieve improved safety.

[0122] The color of the human detection icon of this embodiment shows an example, and is not limited to this color. For example, the human detection icon can be displayed in grayscale. Furthermore, the color change of the human detection icon of this embodiment shows an example of the display method, and is not limited to the change of this color. For example, depending on whether it is included in the first circular area (an example of the first area) 421c and the second circular area (an example of the first area) 421d, the flashing period of the human detection icon can be different, and the contrast or brightness can be different.

[0123] and, Figure 4An example is shown in which a person detection icon is displayed in each area, i.e., within the first circular area (an example of the first area) 421c, outside the first circular area (an example of the first area) 421c and within the second circular area (an example of the first area) 421d, and outside the second circular area (an example of the first area) 421d. However, depending on the person detection result, multiple person detection icons may be displayed in each area. Furthermore, in the case where multiple person detection icons exist in each area, multiple person detection icons existing in one area may be displayed identifiably.

[0124] For example, when there are multiple human detection icons in each area, each human detection icon may be assigned a unique number and displayed. In addition, the human detection icon may be displayed in a manner in which each shape is changed. Furthermore, through this display, the person in each area can be uniquely identified. Therefore, even when there are multiple people, the operator can operate the excavator 100 based on identifying each person, thereby improving safety.

[0125] By referring to the shovel state display area 421 , the operator can recognize how the positional relationship between the attachment AT of the shovel 100 and the people present around the shovel 100 changes when the operator swings the shovel 100 . Therefore, safety can be improved.

[0126] In addition, this embodiment describes an example of displaying the first circular area 421c and the second circular area 421d, but this embodiment is not limited to the example of displaying the first circular area 421c and the second circular area 421d, and either the first circular area 421c or the second circular area 421d may be displayed.

[0127] In addition, the excavator status display area (an example of a specified display area) 421 is not limited to a method of displaying a circular area showing the range of rotation of the attachment AT in order to identify the distance between the excavator 100 and a person. For example, a circular area showing a specified distance (for example, 4 m) centered on the excavator 100 may also be displayed.

[0128] The image display unit 42 displays the shovel state display area 421 and the image information captured by the camera S6. The operator can recognize the specific situation around the shovel 100 by checking the image information together with the shovel state display area 421. Therefore, safety can be improved.

[0129] like Figure 4As shown, in the screen, the first image display area 422 and the second image display area 423 are areas for displaying image information captured by the camera device S6. The right image is displayed in the first image display area 422. The rear image is displayed in the second image display area 423. The right image is an image showing the right space of the excavator 100, including an image 422c of the right end of the upper surface of the upper rotating body 3. The right image is an actual viewpoint image generated by the control unit 40a, and is generated based on the image acquired by the camera S6R. The rear image is an image showing the rear space of the excavator 100, and includes an image 423c of the counterweight. The rear image is an actual viewpoint image generated by the control unit 40a, and is generated based on the image acquired by the camera S6B.

[0130] The first image display area 422 is displayed on the right with the excavator state display area 421 as a reference. And, the second image display area 423 is displayed below with the excavator state display area 421 as a reference. In addition, the present embodiment is set as an example in which the upper part of the image display unit 42 corresponds to the front of the upper rotating body 3. In other words, the second image display area 423 is displayed at a position corresponding to the rear with the excavator state display area 421 as a reference. That is, the image display unit 42 displays the image information captured by the camera device S6 in the direction captured by the camera device S6 with the excavator state display area 421 as a reference. In the present embodiment, the image information captured in the shooting direction is displayed with the excavator state display area 421 as a reference, so the operator can intuitively recognize in which direction the image information represents the condition when referring to the image information. Therefore, it is possible to achieve improved safety.

[0131] In addition, this embodiment shows an example of the arrangement of image information, but is not limited to this arrangement. For example, the first image display area 422 and the second image display area 423 may be arranged regardless of the direction in which the image is captured.

[0132] When a person is detected from one or more of the right image and the rear image, the controller 30 transmits information indicating the region where the person is detected to the control unit 40 a .

[0133] Then, based on the received information, the control unit 40a superimposes a frame indicating information on the region where the person is detected on the image in which the person is detected in the right image and the rear image.

[0134] As a result, a frame 422 b is displayed in the right image of the first image display area 422 , and a frame 423 b is displayed in the rear image of the second image display area 423 .

[0135] The control unit 40a makes the color of the frame 422b of the right image in the first image display area 422 match the color of the human detection icon 421e, and makes the color of the frame 423b of the rear image in the second image display area 423 match the color of the human detection icon 421f.

[0136] That is, in the image display unit 42, a frame representing the detected person is displayed for the right image and the rear image, and the correspondence between the person represented by the frame and the person detection icon is displayed in a recognizable manner. Thus, the operator can identify the condition of the person shown in the excavator status display area 421 by referring to the right image and the rear image. Therefore, the operator can operate the excavator 100 in consideration of the conditions of the people existing around the excavator 100. Therefore, it is possible to achieve improved safety. In addition, as an example of a display in which a recognizable correspondence relationship can be displayed, the present embodiment describes an example of making the color of the frame consistent with the color of the person detection icon. However, the present embodiment shows an example of a display in which a recognizable correspondence relationship can be displayed, and is not limited to a method of making the color of the frame consistent with the color of the person detection icon. For example, the correspondence relationship can be identified by the flashing cycle of the frame and the person detection icon.

[0137] As described above, the color of the human detection icon changes according to the distance from the excavator 100. The human detection icon in the excavator state display area 421 is made consistent with the color of the frame displayed in the right image and the rear image. Therefore, in the image display unit 42, the color (an example of a display method) of the frame displayed in the right image and the rear image is displayed differently according to the distance between the excavator 100 and the person. In the present embodiment, the color of the frame displayed in the right image and the rear image changes according to the distance from the excavator 100, so the operator can recognize the distance from the excavator 100 when referring to the color of the frame. Therefore, the operator can perform operations corresponding to the distance, so that safety can be improved.

[0138] In the present embodiment, the control unit 40 a displays the display screen 85 on the image display unit 42 , so that the operator can understand the situation around the shovel 100 .

[0139] Next, the shovel icon 421b will be described. Figure 5 1 and 2 are diagrams showing changes in the shovel icon 421 b and the direction display icon 421 a displayed on the image display unit 42 according to the present embodiment.

[0140] exist Figure 5 In the example of state 1501 , in the image display unit 42 , the direction display icon 421 a 1 , the shovel icon 421 b 1 , the first circular area 421 c , the second circular area 421 d , and the human detection icon are displayed in the shovel state display area 421 .

[0141] Then, the operator performs an operation to rotate the upper rotating body 3. For example, the operator performs an operation to rotate it to the right. Thus, the upper rotating body 3 starts to rotate. Then, the control unit 40a receives the rotation angle of the upper rotating body 3 from the controller 30.

[0142] When the control unit 40a detects a change in the rotation angle, the control unit 40a generates and displays an excavator icon 421b2 for moving the lower traveling body 1 based on the changed rotation angle, as shown in state 1502. In addition, the control unit 40a displays a direction display icon 421a2 indicating the movable direction of the lower traveling body 1 indicated by the excavator icon 421b2. In addition, the position of the human detection icon also changes according to the change in the rotation angle.

[0143] In addition, the operator continues the operation of rotating the upper rotating body 3. Then, the control unit 40a receives the rotation angle of the upper rotating body 3 from the controller 30.

[0144] When the control unit 40a detects a further change in the rotation angle, the control unit 40a generates and displays an excavator icon 421b3 for moving the lower traveling body 1 based on the changed rotation angle, as shown in state 1503. In addition, the control unit 40a displays a direction display icon 421a3 indicating the movable direction of the lower traveling body 1 indicated by the excavator icon 421b3. In addition, the position of the human detection icon also changes according to the change in the rotation angle.

[0145] As shown in states 1501 to 1503, the control unit 40a displays the excavator icon 421b3 on the upper side of the display area of ​​the image display unit 42 in a manner indicating the front side of the upper rotating body 3. Even if the upper rotating body 3 of the excavator 100 is rotated, the excavator icon 421b is displayed in a manner such that the front side of the upper rotating body 3 of the excavator 100 becomes the upper side of the display area of ​​the image display unit 42. That is, the excavator icon 421b is displayed in a manner corresponding to the viewpoint of the operator. Therefore, the operator can intuitively recognize the angle of the lower traveling body 1 based on the upper rotating body 3 by referring to the excavator icon 421b. In other words, the operator riding in the cab 10 can recognize to what extent the direction of the lower traveling body 1 is different when the traveling pole is tilted forward, from the direction the operator is currently facing. Furthermore, the operator can recognize in which direction the person is detected based on the current direction the operator is facing by referring to the shovel icon 421b and the person detection icon. Therefore, the operator can intuitively recognize the situation around the shovel 100, thereby improving safety.

[0146] in addition, Figure 5The display of the shovel state display area 421 shown is not limited to the method in which the display mode changes synchronously with the current rotation angle of the upper rotating body 3 during the rotation of the upper rotating body 3. For example, when the rotation of the upper rotating body 3 stops, the display mode may be displayed in a manner that changes.

[0147] (Variation 1 of the first embodiment)

[0148] The above embodiment shows one form of the display screen displayed on the image display unit 42. The display screen displayed on the image display unit 42 is not limited to the above display form. Therefore, in a modification of the first embodiment, another form of the shovel state display area 421 will be described.

[0149] The position estimation unit 30c of the controller 30 according to the present modification determines whether the detected person exists within the specified range of the movable direction of the lower traveling body 1 in the actual space (the actual space where the shovel 100 exists) based on the rotation angle and the direction and distance of the detected person. The position estimation unit 30c transmits the determination result of whether the detected person exists within the specified range to the control unit 40a. The control unit 40a changes the display mode in the shovel state display area 421 according to the determination result.

[0150] Figure 6 1 is a diagram showing changes in the shovel state display area 421 displayed on the display device 40 according to the present modification. The shovel state display area 421 changes its display mode depending on whether a person is present in the movable direction of the lower traveling body 1.

[0151] Figure 6 (A) shows the direction display icon 421 a , the shovel icon 421 b , the first circular area 421 c , and the second circular area 421 d in the shovel state display area 421 .

[0152] The position estimation unit 30c of the controller 30 determines whether there is a person within a predetermined range of the actual space corresponding to the front area 1601 based on the rotation angle and the direction and distance of the detected person. Figure 6 In the example (A), it is determined that no person exists in the actual space corresponding to the front area 1601, so a normal screen is displayed on the image display unit 42. The line indicating the front area 1601 may be displayed in the shovel state display area 421 or may not be displayed.

[0153] exist Figure 6In (B), the position estimation unit 30c determines that a person exists within a predetermined range of the actual space corresponding to the front area 1601 based on the rotation angle and the direction and distance of the detected person. In other words, the presence of a person within the predetermined range means that the front area 1601 includes the person detection icon 1602.

[0154] At this time, the control unit 40a displays the direction display icon 621a4 included in the shovel state display area 421 of the image display unit 42 in a color indicating a warning (for example, red).

[0155] This display allows the operator to easily recognize whether a person is present in the movable direction of the lower traveling body 1. Therefore, safety can be improved.

[0156] Next, a description will be given of a processing procedure executed by the controller 30 and the display device 40 according to the present modification. Figure 7 1 is a flowchart showing a processing procedure for the controller 30 and the display device 40 according to the present modification to display a display screen.

[0157] First, the acquisition unit 30a acquires image information captured by the imaging device S6 (S1701).

[0158] Next, the acquisition unit 30a acquires the angles of the attachments (boom angle, arm angle, bucket angle) from the angle sensors S1 to S3, and acquires the swing angle from the swing angle sensor S5 (S1702).

[0159] Then, the detection unit 30b detects a person based on the image information, and the position estimation unit 30c estimates the distance and direction to the detected person (S1703).

[0160] Then, the position estimation unit 30c determines whether or not there is a person within a predetermined range in the traveling direction of the lower traveling body 1 based on the rotation angle and the distance and direction to the detected person (S1704).

[0161] When the position estimation unit 30c determines that there is no person within the predetermined range (S1704: No), the output control unit 30d transmits information for generating a display screen to the display device 40 together with the determination result indicating that there is no person within the predetermined range.

[0162] Then, the control unit 40a of the display device 40 displays the display screen including the shovel state display area 421 in the normal display mode together with the image information (S1705).

[0163] On the other hand, when the position estimation unit 30c determines that a person exists within the predetermined range (S1704: Yes), the output control unit 30d sends information for generating a display screen to the display device 40 together with the determination result indicating that a person exists within the predetermined range.

[0164] Then, the control unit 40a of the display device 40 displays the display screen including the shovel state display area 421 in which the direction display icon is represented in red together with the image information (S1706).

[0165] Since the shovel 100 according to the present modification example performs the above-described processing, the operator can recognize the situation around the shovel 100 , and thus safety can be improved.

[0166] (Variation 2 of the first embodiment)

[0167] The above embodiment shows one mode of the display screen displayed by the image display unit 42. The display screen displayed by the image display unit 42 is not limited to the display mode of the above embodiment. For example, the above embodiment does not limit the number of image information displayed on the display screen to two. Therefore, in the second variation of the first embodiment, the case of changing the number of image information displayed is described.

[0168] Next, refer to Figure 8 , an example of a display screen 85A displayed by the image display unit 42 according to the control from the control unit 40a is described. Figure 8 FIG. 8 is a diagram showing an example of a display screen 85A displayed on the display device 40 according to the present modification. The display screen 85A is displayed on the image display unit 42 during the operation of the shovel 100. Figure 4 The same display contents are assigned the same symbols, and description thereof is omitted.

[0169] In the image display unit 42, as Figure 4 The display screen 85A including the shovel state display area 1421, the first image display area 1422, the second image display area 1423, and the third image display area 1424 is displayed with different display contents. The display screen 85A may include other display areas.

[0170] The shovel state display area 1421 is an area for displaying information indicating the positional relationship between the shovel 100 and a person detected around the shovel 100 .

[0171] The shovel state display area (an example of a predetermined display area) 1421 is a display area that displays the actual space centered on the shovel 100 at a predetermined scale ratio. The shovel state display area (an example of a predetermined display area) 1421 has a shovel icon 1421b indicating the presence of the shovel 100 arranged at the center of the area.

[0172] In the shovel state display area (an example of a predetermined display area) 1421, in addition to the shovel icon (an example of the first display information) 1421b indicating the shovel 100, an icon ( Figure 8 In the example of FIG. 1 , the direction display icon 1421a) and the icon indicating a person detected from the vicinity of the shovel 100 ( Figure 8 In the example, human detection icons 1421e, 1421f, 1421g).

[0173] Furthermore, in the shovel state display area (an example of a prescribed display area) 1421, with the shovel 100 as a reference, a first circular area (an example of a first area) 1421c and a second circular area (an example of a first area) 1421d determined according to the distance from the shovel 100 are displayed.

[0174] exist Figure 8 In the screen shown, the first image display area 1422, the second image display area 1423, and the third image display area 1424 are areas for displaying image information captured by the camera S6. The right image is displayed in the first image display area 1422. The rear image is displayed in the second image display area 1423. The left image is displayed in the third image display area 1424. The right image is an image showing the right space of the shovel 100, the rear image is an image showing the rear space of the shovel 100, and the left image is an image showing the left space of the shovel 100. The third image display area 1424 is displayed on the left with the shovel state display area 1421 as a reference. That is, the image display unit 42 displays the image information captured by the camera S6 in the direction captured by the camera S6 with the shovel state display area 1421 as a reference. In the present modification, a left image is added compared to the above-described embodiment, so that the situation around the shovel 100 can be further understood.

[0175] When the controller 30 detects a person from any one or more of the right image, the rear image, and the left image, the controller 30 transmits information indicating the region where the person is detected to the control unit 40 a .

[0176] Then, the control unit 40a superimposes a frame indicating information on the region where a person is detected on the image in which a person is detected among the right image, the rear image, and the left image.

[0177] As a result, a frame 1422 b is displayed in the right image of the first image display area 1422 so as to surround the person 1422 a , and a frame 1424 b is displayed in the left image of the third image display area 1424 so as to surround the person 1424 a .

[0178] Furthermore, a person who is far away relative to the excavator 100 may not be surrounded by a frame. For example, a person 1423a is present in the rear image of the second image display area 1423. However, as shown in the excavator state display area 1421, the person detection icon 1421g is located outside the second circular area (an example of the first area) 1421d. Therefore, when generating the display screen, the control unit 40a suppresses the use of a frame to surround the person 1423a corresponding to the person detection icon 1421g for the third image display area 1424. Thus, with respect to a person who is far away and does not require attention, since the frame is not displayed, the operator can only grasp the person who should be paid attention to.

[0179] (Variation 3 of the first embodiment)

[0180] In the first embodiment, an example of displaying the right image and the rear image is described. As for the left image, since the operator only needs to face the left from the cab 10, the display is omitted. However, even if there is no left image, if there is a person on the left, it is preferable to make the operator recognize that there is a person on the left. Therefore, in the second modification, a display method that can recognize that there is a person on the left is described.

[0181] Next, refer to Fig. 9 , an example of a display screen 85B displayed by the image display unit 42 according to the control from the control unit 40a is described. Fig. 9 85B is a diagram showing an example of a display screen 85B displayed on the display device 40 according to the present modification. The display screen 85B is displayed on the image display unit 42 during the operation of the shovel 100. Figure 4 The same display contents are assigned the same symbols, and description thereof is omitted.

[0182] The image display unit 42 displays a display screen 85B including a shovel state display area 2421, a first image display area 2422, and a second image display area 2423. The display screen 85B may include other display areas.

[0183] The shovel state display area 2421 is an area for displaying information indicating the positional relationship between the shovel 100 and a person detected around the shovel 100 .

[0184] The shovel state display area (an example of a predetermined display area) 2421 is a display area that displays the actual space centered on the shovel 100 at a predetermined scale ratio. The shovel state display area (an example of a predetermined display area) 2421 has a shovel icon 2421b indicating the presence of the shovel 100 arranged at the center of the area.

[0185] In the shovel state display area (an example of the prescribed display area) 2421, in addition to the shovel icon (an example of the first display information) 2421b indicating the shovel 100, an icon ( Fig. 9 In the example of FIG. 1 , the direction display icon 2421a) and the icon indicating a person detected from the vicinity of the shovel 100 ( Fig. 9 In the shovel state display area 2421, the shovel icon (an example of the first display information) 2421b and the human detection icons 2421e, 2421f, 2421g are arranged to indicate the positional relationship between the shovel 100 and the detected person.

[0186] Furthermore, in the shovel state display area (an example of a prescribed display area) 2421, with the shovel 100 as a reference, a first circular area (an example of the first area) 2421c and a second circular area (an example of the first area) 2421d determined according to the distance from the shovel 100 are displayed.

[0187] exist Fig. 9 In the screen shown, the first image display area 2422 and the second image display area 2423 are areas for displaying image information captured by the camera S6. The right image is displayed in the first image display area 2422. The rear image is displayed in the second image display area 2423.

[0188] The first image display area 2422 is displayed on the right with respect to the shovel state display area 2421. Furthermore, the second image display area 2423 is displayed behind with respect to the shovel state display area 2421.

[0189] When the controller 30 detects a person from one or more of the right image and the rear image, the controller 30 transmits information indicating the area where the person is detected to the control unit 40 a .

[0190] Then, the control unit 40a superimposes a frame indicating information on the region where a person is detected on the image in which a person is detected in the right image and the rear image.

[0191] As a result, in the rear image of the second image display area 2423, a frame 2423b is displayed so as to surround the person 2423a.

[0192] Furthermore, a person who is far away relative to the excavator 100 may not be surrounded by a frame. For example, a person 2423c is present in the rear image of the second image display area 2423. However, as shown in the excavator state display area 2421, the person detection icon 2421g is located outside the second circular area (an example of the first area) 2421d. Therefore, when generating the display screen, the control unit 40a suppresses the use of a frame to surround the person 2423c corresponding to the person detection icon 2421g in the second image display area 2423. Thus, with respect to a person who is far away and does not require attention, since the frame is not displayed, the operator can only grasp the person who should be paid attention to.

[0193] Furthermore, the controller 30 according to the present modification detects a person in the space to the left of the shovel 100 based on the image information captured by the camera S6. Therefore, the position information (indicating the position where a person exists) sent by the output control unit 30d to the control unit 40a also includes the position information of the person detected in the space to the left of the shovel 100.

[0194] Furthermore, the control unit 40a determines whether there is a person in the area not represented as image information based on the received position information. When the control unit 40a determines that there is a person in the area not represented as image information, the control unit 40a arranges display information indicating the presence of a person (e.g., image information 2421h) in the excavator state display area 1421 in the direction where the person is present, with reference to the excavator icon 2421b.

[0195] In this modification, Fig. 9 In the shovel state display area (an example of a prescribed display area) 2421, image information 2421h indicating that a person is detected from the left space is displayed.

[0196] The control unit 40a makes the color of the image information 2421h match the color of the human detection icon 2421e.

[0197] That is, when a person is detected from the directions not displayed as the first image display area 2422 and the second image display area 2423, the control unit 40a displays the bar-shaped image information (for example, image information 2421h) in the direction where the person is detected in the shovel state display area 1421 on the image display unit 42. By making the color of the bar-shaped image information correspond to the color of the person detection icon 2421e, the operator can recognize the correspondence.

[0198] Thus, the operator can recognize the presence of a person who is not captured in the image information and is present near the shovel 100 by referring to the display screen 85B.

[0199] When a person is detected from a direction not displayed as the first image display area 2422 and the second image display area 2423, the control unit 40a according to this modification displays the bar-shaped image information (e.g., image information 2421h) at a position corresponding to the direction, thereby enabling the operator to recognize the person. Therefore, safety can be improved.

[0200] In addition, in this variation, the display of the rod-shaped image information is not limited to the case where a person is detected from a direction not displayed in each of the first image display area 2422 and the second image display area 2423, but can be performed when a person is detected in the first image display area 2422 or the second image display area 2423.

[0201] (Second embodiment)

[0202] In the second embodiment, a case where an operator performs remote control of the shovel 100 will be described.

[0203] Fig.10 1 is a schematic diagram showing a configuration example of a remote support system SYS for the shovel 100 according to the present embodiment. Fig.10 In the example shown, the shovel 100 and the remote operation room RC are connected via a communication network NW. This enables information to be transmitted and received between the shovel 100 and the remote operation room RC.

[0204] The shovel 100 transmits the detection results from various sensors installed in the shovel 100 to the remote operation room RC using the communication device T1 installed in the shovel 100. For example, the shovel 100 transmits the image information captured by the camera S6, the rotation angle, the area where a person is captured in the image information, the position information of the person existing around the shovel (information indicating the direction and distance of the detected person), and the detection results of various sensors to the remote operation room RC.

[0205] The remote support system SYS according to the present embodiment includes a remote operation room RC. The remote operation room RC includes a display device DR, an operation device R26, an operation sensor R29, an operation seat DS, a remote controller R30, and a communication device T2.

[0206] The remote controller R30 displays a display screen based on the image information captured by the camera device S6, the rotation angle, the area where a person is captured in the image information, the position information of a person existing around the shovel (information showing the direction and distance of the detected person), and the detection results of various sensors on the display device DR. As a result, the operator OP existing in the operator seat DS can also confirm the situation around the shovel 100 when existing in the remote operation room RC.

[0207] In the display screen, the shovel state display area is displayed together with the image information showing the surroundings of the shovel 100. The shovel state display area is set to be the same as in the above-mentioned embodiment, and the description thereof is omitted. Furthermore, the arrangement relationship between the shovel state display area and the image information showing the surroundings of the shovel 100 may be in any manner. As in the above-mentioned embodiment and the modified example, the image information based on the shovel state display area may be arranged.

[0208] The operator OP in the operator seat DS of the remote control room RC operates the operating device R26. The operating sensor R29 detects the operation content received by the operating device R26. The controller 30 generates a control signal corresponding to the operation content. Then, the communication device T2 sends the generated control signal to the shovel 100. The remote controller R30 can remotely operate the shovel 100 by sending the control signal.

[0209] Thus, the operator OP can recognize a person existing around the shovel 100 , similarly to the above-described embodiment and the modified example.

[0210] (Variation of the Second Embodiment)

[0211] In the above embodiment, the operator is not in the cab 10 for remote control from the remote control room RC. However, the above embodiment does not limit the operator to the remote control, and the shovel 100 may also perform autonomous control.

[0212] In this case, there may be a remote monitoring system for monitoring the shovel 100 that performs autonomous control. The monitor can monitor the shovel 100 from the monitoring device provided in the remote monitoring system. The monitor can check the conditions around the shovel 100 by referring to the shovel state display area together with the image information captured by the camera S6, as in the above-mentioned embodiment and the modified example. Thus, it is possible to improve safety.

[0213] <Function>

[0214] In the above-mentioned embodiment and modification, the direction of travel of the shovel 100 can be identified by the shovel icon, and the positional relationship between the shovel and the person can be identified based on the positional relationship between the shovel icon and the person detection icon. Therefore, even when there are various objects around the shovel 100, the operator can identify the positional relationship between the shovel 100 and the person and operate the shovel 100. Therefore, safety can be improved. In addition, since the operator can smoothly perform the operation, productivity can be improved.

[0215] The above describes the embodiments of the excavator, display device, and excavator control system involved in the present invention, but the present invention is not limited to the above embodiments, etc. Various changes, modifications, substitutions, additions, deletions, and combinations can be made within the scope described in the scope of the patent claim. These contents certainly also belong to the technical scope of the present invention.

Claims

1. An excavator comprising: Lower walking body; An upper rotating body capable of rotating relative to the lower walking body; An object detection device is provided on the upper rotating body and detects objects existing around the excavator; An angle detection device detects the rotation angle of the upper rotating body relative to the lower walking body; a control device configured to obtain the rotation angle, the detection result of the object detection device, and the position information of a person existing around the excavator; and A display device displays first display information and second display information simultaneously in a specified display area in a manner indicating the positional relationship between the excavator and the person, wherein the first display information indicates the excavator in a manner capable of identifying the angle between the upper rotating body and the lower walking body based on the rotation angle, and the second display information indicates the person existing at the position indicated by the position information.

2. The excavator according to claim 1, wherein: When the change in the rotation angle is detected, the display device displays the first display information for moving the lower traveling body according to the rotation angle, and displays third display information indicating the movable direction of the lower traveling body of the shovel indicated by the first display information.

3. The excavator according to claim 2, wherein: The display device displays the first display information in such a manner that the front side of the upper rotating body is displayed on the upper side of the display area of ​​the display device.

4. The excavator according to claim 2, wherein: The display device changes the display mode in the predetermined display area when the person is detected in the direction indicated by the third display information.

5. The excavator according to claim 2, wherein: The display device displays a plurality of first areas determined according to the distance from the shovel in the predetermined display area, and displays the second display information in different manners depending on whether or not the second display information is included in each of the first areas.

6. The excavator according to claim 5, wherein: When the display device includes a plurality of the second display information in one first area, the display device displays the plurality of the second display information in a identifiable manner.

7. The excavator according to claim 5, wherein: The first area displayed by the display device is at least one of a range in which the attachment of the shovel is currently rotated and a range in which the attachment of the shovel is rotated to the maximum in the horizontal direction based on the first display information.

8. The excavator according to claim 1, wherein: The object detection device includes a camera device for photographing objects existing around the excavator. The display device displays the first display information and the second display information in the predetermined display area, and displays image information captured by the imaging device.

9. The excavator according to claim 8, wherein: The display device displays, for the image information, a region indicating the person detected from the image information, and identifiably displays a correspondence relationship between the person indicated in the region and the person indicated by the second display information.

10. The excavator according to claim 9, wherein: The display device displays the area shown in the image information and the second display information in a different manner according to the distance between the shovel and the person.

11. The excavator according to claim 8, wherein: The display device displays the image information captured by the imaging device in a direction captured by the imaging device, with the prescribed display area as a reference.

12. The excavator according to claim 8, wherein: The display device arranges display information indicating that a person exists in the predetermined display area in a direction where the person exists, when determining based on the position information that a person exists in the area not displayed as the image information.

13. A display device, configured to simultaneously display first display information and second display information in a specified display area in a manner indicating a positional relationship between an excavator and a person, wherein the first display information indicates the excavator in a manner capable of identifying an angle between the upper rotating body and the lower traveling body based on a rotation angle of the upper rotating body relative to a lower traveling body of the excavator, and the second display information indicates the person detected by an object detection device provided on the upper rotating body.

14. A control system for an excavator, comprising: An excavator comprises: a lower traveling body; an upper rotating body capable of rotating relative to the lower traveling body; an object detection device provided on the upper rotating body and detecting objects existing around the excavator; an angle detection device detecting the rotation angle of the upper rotating body relative to the lower traveling body; and a first communication device transmitting the rotation angle, the detection result of the object detection device and the position information of people existing around the excavator; and A control device comprises: a second communication device for receiving the rotation angle and the position information; and a display device for simultaneously displaying first display information and second display information in a prescribed display area in a manner indicating the positional relationship between the excavator and the person, wherein the first display information indicates the excavator in a manner capable of identifying the angle between the upper rotating body and the lower walking body based on the rotation angle, and the second display information indicates the person existing at the position indicated by the position information.

Images