Shovel and shovel control system

Abstract

The invention relates to an excavator and a control system of the excavator, which can improve detection precision. A shovel according to one embodiment of the present invention is provided with: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; an imaging device mounted on the upper rotating body; a display device that displays captured image data captured by the imaging device; and a control device configured so as to, upon receiving an operation specifying an object indicated in the captured image data, perform a setting for suppressing detection of the object with respect to information indicating the object.

Description

Technical Field

[0001] This application claims priority based on Japanese Patent Application No. 2023-223040 filed on December 28, 2023. The entire contents of this Japanese application are incorporated herein by reference.

[0002] The invention relates to an excavator and a control system of the excavator. Background Art

[0003] Conventionally, there has been proposed a technique for detecting an object existing around a shovel to monitor the area around the shovel (for example, refer to Patent Document 1).

[0004] Patent Document 1: Japanese Patent Application Publication No. 2014-224411

[0005] Patent document 1 describes a technology for detecting objects around a construction machine using a stereo camera. However, when a detection device such as a stereo camera is used to detect an object, errors may occur in the detection result of the object. If information is output based on the erroneous detection result, it may cause trouble to the operator. Summary of the invention

[0006] One aspect of the present invention proposes a technology that can improve detection accuracy by correcting detection results through operations.

[0007] An excavator involved in one embodiment of the present invention comprises: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; a camera device installed on the upper rotating body; a display device that displays camera image data captured by the camera device; and a control device that is configured to, when receiving an operation specifying an object represented in the camera image data, set the information representing the object to suppress detection of the object.

[0008] Effects of the Invention

[0009] According to one aspect of the present invention, the detection result can be corrected through operation, thereby achieving improvement in detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a side view showing an example of the shovel according to the first embodiment.

[0011] Figure 2 It is a plan view showing an example of the shovel according to the first embodiment.

[0012] Figure 3 It is a diagram showing a configuration example of a drive control system of the shovel according to the first embodiment.

[0013] Figure 4 It is a functional block diagram showing a structural example of a controller for an excavator according to the first embodiment.

[0014] Figure 5 It is a diagram showing an example of a display screen displayed on a first display device and an input device according to the first embodiment.

[0015] Figure 6 It is a diagram showing the transition of a rear image arranged in a second image display area based on the control of the controller and the first display device according to the first embodiment.

[0016] Figure 7 It is a flowchart showing the controller and the first display device according to the first embodiment for suppressing the setting order of a detection object.

[0017] Figure 8 It is a flowchart showing the processing sequence when making a determination using an object storage database when displaying captured image data in the controller and the first display device according to the first embodiment.

[0018] Figure 9 It is a side view showing an example of an excavator according to the second embodiment.

[0019] Figure 10 It is a schematic diagram showing an example of a control system according to the third embodiment.

[0020] Figure 11 It is a timing diagram showing the setting order of suppressing a detection object in the control system according to the third embodiment.

[0021] Figure 12 It is a timing diagram showing the processing sequence when making a determination using an object storage database when displaying captured image data in the control system according to the third embodiment.

[0022] Explanation of symbols

[0023] 100, 100A - Excavator, 1 - Lower traveling body, 2 - Slewing mechanism, 3 - Upper slewing body, 4 - Boom, 5 - Arm, 6 - Bucket, S1 - Boom angle sensor, S2 - Arm angle sensor, S3 - Bucket angle sensor, S4 - Body tilt sensor, S5 - Slewing angle sensor, S6 - Imaging device, S7 - Spatial recognition device, PS - Positioning device, T1 - Communication device, D3 - First display device, D3a - Control unit, D4 - Auxiliary storage device, D4A - Object storage database, LM - Learned model, 30, 30A - Controller, 301 - Acquisition unit, 302 - Detection unit, 303 - Output control unit, 304 - Operation reception unit, 305 - Setting unit, 306 - Judgment unit, RC - Remote operation room, R30 - Remote controller, T2 - Communication device, 2000 - Management server. Detailed implementation mode

[0024] Hereinafter, with reference to the drawings, embodiments of the present invention will be described. Note that the embodiments described below are examples and do not limit the invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention. In addition, the same or corresponding structures are denoted by the same or corresponding reference numerals in the respective drawings, and the description may sometimes be omitted.

[0025] Hereinafter, in an embodiment of the present invention, as an example of construction machinery, an example of using an excavator will be described, but it is not limited to an excavator. It can also be applied to construction machinery, standard machines, application machines, forestry machinery, or conveying machinery based on hydraulic excavators.

[0026] (First embodiment)

[0027] First, with reference to Figure 1 the outline of the excavator 100 according to this embodiment will be described. Figure 1 is a side view of the excavator 100 according to the first embodiment. Figure 2 is a top view of the excavator 100 according to the first embodiment.

[0028] On the lower traveling body 1 of the excavator 100, the upper slewing body 3 is rotatably mounted via the slewing mechanism 2. The boom 4 is mounted on the upper slewing body 3. The arm 5 is mounted at the front end of the boom 4, and the bucket 6 as an end attachment is mounted at the front end of the arm 5. The end attachment can be a bucket for slopes or a bucket for dredging.

[0029] The boom 4, the arm 5, and the bucket 6 constitute an excavation attachment as an example of the attachment device AT, and are hydraulically driven by the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9, respectively. A boom angle sensor S1 is installed on the boom 4, an arm angle sensor S2 is installed on the arm 5, and a bucket angle sensor S3 is installed on the bucket 6. A bucket tilt mechanism may be provided on the excavation attachment.

[0030] The boom angle sensor S1 detects the rotation angle of the boom 4. In the present embodiment, the boom angle sensor S1 is an acceleration sensor and can detect the rotation angle of the boom 4 with respect to the upper swing body 3, that is, the boom angle. The boom angle becomes the minimum angle, for example, when the boom 4 is lowered to the maximum extent, and becomes larger as the boom 4 is lifted.

[0031] The boom angle sensor S1 may include, for example, a rotary encoder, an acceleration sensor, a 6-axis sensor, an IMU (Inertial Measurement Unit), etc. Further, the boom angle sensor S1 may also include a potentiometer using a variable resistor, a cylinder stroke sensor that detects the stroke amount of the hydraulic cylinder (boom cylinder 7) corresponding to the boom angle, and the like. The same applies to the arm angle sensor S2, the bucket angle sensor S3, and the body tilt sensor S4 hereinafter. The detection signal corresponding to the boom angle based on the boom angle sensor S1 is input to the controller 30.

[0032] The arm angle sensor S2 detects the rotation angle of the arm 5. In the present embodiment, the arm angle sensor S2 is an acceleration sensor and can detect the rotation angle of the arm 5 with respect to the boom 4, that is, the arm angle. The arm angle becomes the minimum angle, for example, when the arm 5 is closed to the maximum extent, and becomes larger as the arm 5 is opened.

[0033] The bucket angle sensor S3 detects the rotation angle of the bucket 6. In the present embodiment, the bucket angle sensor S3 is an acceleration sensor and can detect the rotation angle of the bucket 6 with respect to the arm 5, that is, the bucket angle. The bucket angle becomes the minimum angle, for example, when the bucket 6 is closed to the maximum extent, and becomes larger as the bucket 6 is opened.

[0034] The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 may be a potentiometer using a variable resistor, a stroke sensor that detects the stroke amount of the corresponding hydraulic cylinder, or a rotary encoder that detects the rotation angle around the connecting pin, etc. The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 constitute a posture sensor that detects the posture of the excavation attachment.

[0035] A cab 10 serving as an operator's cab is provided on the upper revolving body 3, and a power source such as an engine 11 is mounted thereon. Further, a body tilt sensor S4, a rotation angle sensor S5, and a camera device S6 are mounted on the upper revolving body 3. Also, a communication device T1 and a positioning device PS are mounted on the upper revolving body 3.

[0036] The body tilt sensor S4 is configured to detect the tilt of the upper revolving body 3 with respect to a specified plane. In the present embodiment, the body tilt sensor S4 is an acceleration sensor that detects the tilt angle of the upper revolving body 3 about the front-rear axis and the tilt angle about the left-right axis with respect to the horizontal plane. The front-rear axis and the left-right axis of the upper revolving body 3 are orthogonal to each other, for example, and pass through a point on the rotation axis of the excavator 100, that is, the excavator center point.

[0037] The rotation angle sensor S5 is configured to detect the rotational angular velocity of the upper revolving body 3. In the present embodiment, the rotation angle sensor S5 is a gyro sensor. The rotation angle sensor S5 may also be a resolver or a rotary encoder, etc. The rotation angle sensor S5 can detect the rotational speed. The rotational speed can be calculated based on the rotational angular velocity.

[0038] Further, when the body tilt sensor S4 includes a gyro sensor, a six-axis sensor, an IMU, etc. that can detect the angular velocity about three axes, the rotational state (e.g., rotational angular velocity) of the upper revolving body 3 can also be detected based on the detection signal of the body tilt sensor S4. In this case, the rotation angle sensor S5 can be omitted.

[0039] The camera device S6 is configured to acquire an image of the periphery of the excavator 100. In the present embodiment, the camera device S6 includes a left camera S6L that captures the left space of the excavator 100, a right camera S6R that captures the right space of the excavator 100, and a rear camera S6B that captures the rear space of the excavator 100.

[0040] The camera device S6 is, for example, a monocular camera having an imaging element such as a CCD or a CMOS, and outputs the captured image to the first display device D3 via the controller 30.

[0041] The input device D2 receives an operation input from the operator and outputs it to the controller 30. The input device D2 includes, for example, any hardware operation mechanism such as a touch panel, a touch pad, buttons, a toggle key, a rotary knob, etc. Also, the input device D2 may further include, for example, virtual button icons on the operation screen displayed on the first display device D3, etc., which are software operation mechanisms that can be operated by the hardware operation mechanism.

[0042] As Figure 2As shown, the left camera S6L is installed at the left end of the upper surface of the upper slewing body 3. The right camera S6R is installed at the right end of the upper surface of the upper slewing body 3. The rear camera S6B is installed at the rear end of the upper surface of the upper slewing body 3.

[0043] The rear camera S6B, the left camera S6L, and the right camera S6R are all installed on the upper slewing body 3 in such a way that their optical axes face obliquely downward and a part of the upper slewing body 3 is included in the imaging range. Therefore, each of the imaging ranges of the rear camera S6B, the left camera S6L, and the right camera S6R has a field of view angle of about 180 degrees, for example, when viewed from above. In Figure 2 the example, the imaging range AB represents an example of the imaging range of the rear camera S6B, the imaging range AL represents an example of the imaging range of the left camera S6L, and the imaging range AR represents an example of the imaging range of the right camera S6R. As Figure 2 shown, the three monocular cameras are preferably installed on the upper slewing body 3 in such a way that they do not protrude from the upper surface of the upper slewing body 3.

[0044] By configuring the imaging device S6 as described above in this embodiment, it is possible to image objects existing around the excavator 100. The number of imaging devices S6 provided on the upper slewing body 3 in this embodiment is not limited and may be two or less or four or more. At this time, a front camera may also be provided on the upper surface of the cab 10 of the upper slewing body 3, for example. The front camera can image a predetermined imaging range in front of the upper slewing body 3, for example.

[0045] The positioning device PS is configured to obtain information related to the position of the excavator 100. In this embodiment, the positioning device PS is configured to measure the position and orientation of the excavator 100. Specifically, the positioning device PS is a GNSS receiver equipped with an electronic compass, which measures the latitude, longitude, and altitude of the current position of the excavator 100 and also measures the orientation of the excavator 100.

[0046] Figure 3 is a diagram showing Figure 1 a structural example of the drive control system of the excavator 100. In Figure 3 it, the mechanical power transmission system is represented by double lines, the working oil pipeline is represented by thick solid lines, the pilot pipeline is represented by dashed lines, and the electric drive control system is represented by thin solid lines.

[0047] The engine 11 is the power source of the excavator 100. In this embodiment, the engine 11 is a diesel engine that employs isochronous control to maintain a constant engine speed regardless of the increase or decrease of the engine load. The fuel injection amount, fuel injection timing, supercharging pressure, etc. in the engine 11 are controlled by the engine control unit (ECU) D7.

[0048] The rotating shafts of a main pump 14 and a pilot pump 15, which are hydraulic pumps, are connected to the rotating shaft of an engine 11. The main pump 14 is connected to a control valve unit 17 via a working oil pipeline.

[0049] The control valve unit 17 is a hydraulic control device that controls the hydraulic system of an excavator 100. Hydraulic actuators such as left and right travel hydraulic motors, a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, and a swing hydraulic motor are connected to the control valve unit 17 via working oil pipelines. Additionally, the swing hydraulic motor can be a swing motor-generator.

[0050] Figure 3 The connection relationship between a controller 30 and a first display device D3 and a second display device D3S is shown. In the present embodiment, the first display device D3 and the second display device D3S are connected to the controller 30. The first display device D3, the second display device D3S, and the controller 30 can be connected via a communication network such as CAN.

[0051] The first display device D3 includes a control unit D3a that generates an image. In the present embodiment, the control unit D3a generates a display camera image based on the output of a camera as an imaging device S6. The imaging device S6 is connected to the first display device D3 via a dedicated line, for example. The first display device D3 can display the display camera image generated by the control unit D3a. At this time, the first display device D3 can display the display camera images generated from the respective imaging devices S6 provided on the excavator 100.

[0052] The control unit D3a generates a display image based on the output of the controller 30. In the present embodiment, the control unit D3a converts various information output by the controller 30 into an image signal. The information output by the controller 30 includes, for example, data indicating the engine coolant temperature, data indicating the working oil temperature, data indicating the fuel level, data indicating the urea water level, data indicating the position of the working part of the bucket 6, data indicating the orientation of the slope of the work object, data indicating the orientation of the excavator 100, and data indicating the operation direction for aligning the excavator 100 with the slope.

[0053] Similar to the first display device D3, the second display device D3S includes a control unit D3Sa that generates an image. In the present embodiment, the second display device D3S is not directly connected to the imaging device S6. Therefore, the control unit D3Sa does not generate a camera image. However, when the second display device D3S is directly connected to the imaging device S6, the control unit D3Sa can generate a camera image. Whether the second display device D3S is directly connected to the imaging device S6 or not, a display camera image generated according to the imaging device S6 can be displayed. Moreover, the display camera images generated from the respective imaging devices S6 can be separately displayed on the first display device D3 and the second display device D3S respectively.

[0054] The control unit D3Sa generates a display image according to the output of the controller 30. In the present embodiment, the control unit D3Sa converts various information output by the controller 30 into an image signal.

[0055] The control unit D3a can be implemented as a function of the controller 30, rather than as a function of the first display device D3. The same applies to the control unit D3Sa. At this time, the imaging device S6 is connected to the controller 30, rather than to the first display device D3.

[0056] The first display device D3 and the second display device D3S operate by receiving power supply from the storage battery 70. The storage battery 70 is charged by the power generated by the alternator 11a (generator) of the engine 11. The power of the storage battery 70 is supplied not only to the controller 30, the first display device D3, and the second display device D3S, but also to the electrical fittings 72 of the excavator 100 and the like. The starter 11b of the engine 11 is driven by the power from the storage battery 70 and starts the engine 11.

[0057] The engine 11 is controlled by the engine controller unit D7. Various data indicating the state of the engine 11 are always sent from the engine controller unit D7 to the controller 30. Various data indicating the state of the engine 11 are an example of the operation information of the excavator 100, and include, for example, data indicating the coolant temperature detected by the water temperature sensor 11c as the operation information acquisition unit. The controller 30 can store this data in the temporary storage unit (memory) 30a and send it to the first display device D3 when needed.

[0058] As follows, various data are supplied to the controller 30 as the operation information of the excavator 100 and stored in the temporary storage unit 30a of the controller 30.

[0059] For example, data indicating the swash plate deflection angle is supplied from the governor 13 of the variable displacement hydraulic pump, i.e., the main pump 14, to the controller 30. Also, data indicating the discharge pressure of the main pump 14 is supplied from the discharge pressure sensor 14b to the controller 30. These data are stored in the temporary storage unit 30a. Further, an oil temperature sensor 14c is provided in the pipeline between the fuel tank storing the hydraulic oil sucked by the main pump 14 and the main pump 14, and data indicating the temperature of the hydraulic oil flowing through this pipeline is supplied from the oil temperature sensor 14c to the controller 30. The governor 13, the discharge pressure sensor 14b, and the oil temperature sensor 14c are examples of the operation information acquisition unit.

[0060] Data indicating the fuel storage amount is supplied from the fuel storage amount detection unit 55a in the fuel storage unit 55 to the controller 30. In the present embodiment, data indicating the remaining fuel state is supplied from the fuel level sensor, which serves as the fuel storage amount detection unit 55a, in the fuel tank serving as the fuel storage unit 55 to the controller 30.

[0061] Specifically, the fuel level sensor is composed of a float that follows the liquid level and a variable resistor (potentiometer) that converts the up and down movement amount of the float into a resistance value. With this structure, the fuel level sensor can continuously display the remaining fuel state on the first display device D3. The detection method of the fuel storage amount detection unit can be appropriately selected according to the usage environment and the like, and a detection method capable of stepwise displaying the remaining fuel state can be adopted. These structures are the same for the urea water tank.

[0062] The operation device 26 is provided near the driver's seat in the cab 10 and is used for the operator to operate various driven components. Specifically, the operation device 26 is used for the operator to operate hydraulic actuators such as the left and right travel hydraulic motors, the boom cylinder 7, the arm cylinder 8, the bucket cylinder 9, and the swing hydraulic motor. As a result, the operator can operate the driven components that are the driving targets of the hydraulic actuators. The operation device 26 includes a pedal device and a lever device for operating each driven component.

[0063] The operation sensor 29 is configured to detect the operation content of the operator using the operation device 26. In the present embodiment, the operation sensor 29 detects the operation direction and operation amount of the operation device 26 corresponding to each hydraulic actuator, and outputs an electric signal (hereinafter, also referred to as an operation signal) corresponding to the detected value to the controller 30. In the present embodiment, the controller 30 controls the opening area of the proportional valve 31 according to the output of the operation sensor 29. Moreover, the controller 30 supplies the working oil discharged from the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17. The pressure of the working oil (pilot pressure) supplied to each pilot port is, in principle, a pressure corresponding to the operation direction and operation amount of the operation device 26 corresponding to each hydraulic actuator. Thus, the operation device 26 is configured to be able to supply the working oil discharged from the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17. Thereby, the hydraulic actuator can be driven.

[0064] Moreover, the directional control valve for driving each hydraulic actuator built in the control valve unit 17 may be an electromagnetic solenoid type. At this time, the operation signal output from the operation device 26 can be directly input to the control valve unit 17 (i.e., the electromagnetic solenoid type directional control valve).

[0065] In addition, the operation device 26 may also be a hydraulic pilot type. Specifically, the operation device 26 uses the working oil supplied from the pilot pump 15 through the pilot pipeline to output a pilot pressure corresponding to the operation content to the secondary side pilot pipeline. Moreover, the secondary side pilot pipeline is connected to the control valve unit 17. Thereby, a pilot pressure corresponding to the operation content related to various driven components (hydraulic actuators) in the operation device 26 can be input to the control valve unit 17. Therefore, the control valve unit 17 can drive each hydraulic actuator according to the operation content of the operator or the like on the operation device 26. At this time, an operation sensor 29 for acquiring information related to the operation state of the operation device 26 is provided, and the output of the operation sensor 29 is input to the controller 30. Thereby, the controller 30 can grasp the operation state of the operation device 26. The operation sensor 29 is, for example, a pressure sensor that acquires information related to the pilot pressure (operation pressure) of the secondary side pilot pipeline of the operation device 26.

[0066] Moreover, part or all of the hydraulic actuators may be replaced with electric actuators. At this time, for example, the controller 30 can output an operation instruction corresponding to the operation content of the operation device 26 and the content of the remote operation specified by the remote operation signal to the electric actuator or the driver that drives the electric actuator, etc. And, by inputting an operation signal from the operation device 26 to the electric actuator or the driver, etc., the electric actuator can be configured to be operable by the operation device 26.

[0067] Moreover, when the excavator 100 is specifically remotely operated or specifically operates through the full-automatic operation function, the operation device 26 can be omitted.

[0068] The proportional valve 31 functions as a control valve for mechanical control and is provided for each driven element (hydraulic actuator) of the operation target of the operation device 26 and for each movement direction (e.g., the raising direction and the lowering direction of the boom 4) of the driven element (hydraulic actuator). For example, two proportional valves 31 are provided for each double-acting hydraulic actuator for driving the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5, the bucket 6, etc. The proportional valve 31 can be provided, for example, in the pilot pipe line between the pilot pump 15 and the control valve unit 17 and is configured to be able to change its flow path area (i.e., the cross-sectional area through which the working oil can flow). Thereby, the proportional valve 31 can output a specified pilot pressure to the secondary side pilot pipe line by using the working oil of the pilot pump 15 supplied through the primary side pilot pipe line. Therefore, the proportional valve 31 can cause a specified pilot pressure corresponding to the operation instruction from the controller 30 to act on the control valve unit 17. Therefore, for example, the controller 30 can directly supply the pilot pressure corresponding to the operation content (operation signal) of the operation device 26 from the proportional valve 31 to the control valve unit 17, and realize the operation of the excavator 100 based on the operation of the operator.

[0069] Moreover, the controller 30 can control the proportional valve 31 to realize the full-automatic operation function of the excavator 100. Specifically, the controller 30 outputs an operation instruction corresponding to the full-automatic operation function to the proportional valve 31. Thereby, the controller 30 can realize the operation of the excavator 100 based on the full-automatic operation function.

[0070] Moreover, the controller 30 controls the proportional valve 31 to realize the remote operation of the excavator 100. Specifically, the controller 30 outputs an operation instruction corresponding to the operation content specified by the operation signal received from the remote operation room RC to the proportional valve 31 through the communication device T1. Thereby, the controller 30 can supply the pilot pressure corresponding to the remote operation content from the proportional valve 31 to the control valve unit 17, and realize the operation of the excavator 100 based on the remote operation of the operator.

[0071] In addition, when the operating device 26 is a hydraulic pilot type, a shuttle valve can be provided in the pilot pipe line between the operating device 26 and the proportional valve 31 and the control valve unit 17. The shuttle valve has two inlet ports and one outlet port, and outputs the working oil having the higher pilot pressure of the pilot pressures input to the two inlet ports to the outlet port. Similarly to the proportional valve 31, the shuttle valve is provided for each driven component (hydraulic actuator) of the operation target of the operating device 26 and for each movement direction of the driven component (hydraulic actuator). For example, two shuttle valves are provided for each double-acting hydraulic actuator for driving the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5, the bucket 6, and the like. One of the two inlet ports of the shuttle valve is connected to the secondary side pilot pipe line of the operating device 26 (specifically, the above-mentioned lever device and pedal device included in the operating device 26), and the other is connected to the secondary side pilot pipe line of the proportional valve 31. The outlet port of the shuttle valve is connected to the pilot port of the corresponding switching valve of the control valve unit 17 through a pilot pipe line. The corresponding switching valve refers to the switching valve that drives the hydraulic actuator that is the operation target of the above-mentioned lever device and pedal device connected to one inlet port of the shuttle valve. Therefore, each of these shuttle valves can apply the higher pilot pressure of the pilot pressure in the secondary side pilot pipe line of the operating device 26 and the pilot pressure in the secondary side pilot pipe line of the proportional valve 31 to the pilot port of the corresponding switching valve. That is, the controller 30 can control the corresponding switching valve regardless of the operation of the operator on the operating device 26 by outputting a pilot pressure higher than the secondary side pilot pressure of the operating device 26 from the proportional valve 31. Thereby, the controller 30 can control the operations of the driven components (the lower traveling body 1, the upper swing body 3, the boom 4, the arm 5, and the bucket 6) regardless of the operation state of the operator on the operating device 26, and realize the automatic operation function and the remote operation function.

[0072] Further, when the operating device 26 is a hydraulic pilot type, in addition to the reciprocating valve, a pressure reducing valve may be provided in the pilot pipe line between the operating device 26 and the reciprocating valve. The pressure reducing valve is configured to operate according to a control signal input from the controller 30 and can change its flow path area. Thus, when the operating device 26 is operated by the operator, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26. Therefore, even when the operating device 26 is operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. And, for example, even when the operating device 26 is operated, the controller 30 can reduce the pilot pressure output from the operating device 26 through the pressure reducing valve and make it lower than the pilot pressure output from the proportional valve 31. Therefore, the controller 30 can reliably apply a desired pilot pressure to the pilot port of the switching valve in the control valve unit 17 regardless of the operation content of the operating device 26 by controlling the proportional valve 31 and the pressure reducing valve. Therefore, the controller 30 controls the pressure reducing valve in addition to controlling the proportional valve 31, thereby enabling the automatic operation function and the remote operation function of the excavator 100 to be more appropriately realized.

[0073] The communication system of the excavator 100 according to the present embodiment includes a communication device T1.

[0074] The communication device T1 is connected to an external communication line and communicates with a device separately provided from the excavator 100. Among the devices separately provided from the excavator 100, in addition to the devices located outside the excavator 100, it may also include a portable terminal device (mobile terminal) brought into the cab 10 by the user of the excavator 100. The communication device T1 may include, for example, a mobile communication module conforming to standards such as 4G (4th Generation) and 5G (5th Generation). And, the communication device T1 may also include a satellite communication module, for example. And, the communication device T1 may also include a Wi-Fi communication module, a Bluetooth (registered trademark) communication module, etc. And, when there are multiple connectable communication lines, multiple communication devices T1 may be included corresponding to the types of communication lines.

[0075] For example, the communication device T1 communicates with an external device such as a remote operation room in the work site through a local communication line constructed in the work site. The local communication line is, for example, a mobile communication line based on local 5G (so-called local 5G) constructed in the work site, a local area network based on Wi-Fi.

[0076] And, the communication device T1 is configured to transmit and receive information with a communication device provided in the remote operation room through a wide area communication line, i.e., a wide area network, including the work site.

[0077] In the present embodiment, a case where the auxiliary device AT operates, the upper swing body 3 swings, and traveling are performed by driving a hydraulic pump with the engine 11 as a drive source and using the driving force generated by the engine 11 will be described. However, the drive source in the present embodiment is not limited to the engine 11, and a motor can be used as the drive source. That is, the control described in the present embodiment can be applied to a so-called electric excavator in which a drive source, that is, a motor is driven by electric power supplied from a battery, and can also be applied to an excavator equipped with a plurality of drive sources.

[0078] <Outline of Processing by Controller and Display Device>

[0079] The controller 30 according to the present embodiment detects an object from the captured image data captured by the imaging device S6. Further, the first display device D3 displays display information for indicating the detected object in a recognizable manner together with the captured image data. In the present embodiment, as an example of the display information, an example of a display frame is assumed.

[0080] For example, the first display device D3 can surround a person reflected in the captured image data with a frame (an example of display information) or the like in the captured image data, thereby enabling the operator to recognize the presence of a person around the excavator 100.

[0081] However, in the detection of an object by the controller 30, false detection sometimes occurs. For example, it is conceivable that the controller 30 falsely detects an object (not a person) existing around the excavator 100 as a person. At this time, the first display device D3 displays the non-human object surrounded by a frame or the like as a person. When the operator refers to the situation where the non-human object is surrounded by a frame, the operator may be confused.

[0082] Moreover, the controller 30 may perform safety control according to the content of the false detection. As the safety control performed based on the excavator 100, for example, it includes any one or more of operation restriction of the excavator 100, stop of the excavator 100, alarm output based on sound, light, or vibration, and emphasized display of the detected object in the first display device D3.

[0083] When the controller 30 restricts the operation of the excavator 100 as safety control, for example, restricts the swing operation of the upper swing body 3 or the opening / closing operation of the auxiliary device AT, the current operation performed by the excavator 100 may be inhibited. Further, when the controller 30 stops the excavator 100 as safety control, the current operation performed by the excavator 100 is stopped. When the controller 30 restricts the operation of the excavator 100 or stops the excavator 100 because it falsely detects an object as a person, operation inhibition or operation stop is performed, and thus the operation efficiency is reduced.

[0084] Furthermore, when the controller 30 outputs an alarm as a safety control, the operator stops the operation based on the excavator 100 and performs a surrounding confirmation operation. If there is actually a person around, there is no problem with the confirmation operation, but in the case of a false detection, the surrounding confirmation operation is performed even if there is no person around. In this way, regardless of whether it is a false detection or not, the operator needs to stop the current operation based on the excavator 100 once and perform the confirmation operation according to the alarm output. When the controller 30 outputs an alarm because it mistakenly detects an object as a person, the operator performs the surrounding confirmation operation, so the work efficiency is reduced.

[0085] Furthermore, when the controller 30 highlights the detected object on the first display device D3 as a safety control, the operator needs to look at the first display device D3 to determine what the highlighted object is. The operator's confirmation operation of the highlighted object is performed even if the highlighted object is not a person, etc., when it is a result of an erroneous detection. When the controller 30 highlights a non-human object, the operator needs to identify what the highlighted object is, which is troublesome, and the operator interrupts the operation to identify what the object is, thereby reducing the work efficiency.

[0086] Therefore, the controller 30 according to the present embodiment has a function for correcting the detection result when an erroneous detection of an object occurs.

[0087] <Functional block structure of excavator controller>

[0088] Figure 4 2 is a functional block diagram showing a configuration example of the controller 30 of the shovel 100 according to the present embodiment. Figure 4 The illustrated example shows a functional block structure of the controller 30 of the shovel 100 .

[0089] The controller 30 receives information output from the boom angle sensor S1, the arm angle sensor S2, the bucket angle sensor S3, the machine body tilt sensor S4, the rotation angle sensor S5, the camera device S6, the input device D2, the communication device T1, the positioning device PS, etc. Then, the controller 30 performs various operations based on the received information and the information stored in the auxiliary storage device D4, and outputs the operation results to the first display device D3, the second display device D3S and the proportional valve 31, etc.

[0090] In addition, in this embodiment, an example in which the controller 30 controls the shovel 100 is described, but part of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized by a plurality of controllers mounted on the shovel 100 in a distributed manner.

[0091] The excavator 100 operates an actuator (e.g., a hydraulic actuator) according to the operation of an operator riding in the cab 10, and drives moving components such as the lower traveling body 1, the upper revolving body 3, the boom 4, the arm 5, and the bucket 6 (hereinafter referred to as "driven components").

[0092] Moreover, the excavator 100 can be configured to be operable by the operator in the cab 10, or alternatively, it can be configured to be remotely operable (remotely controlled) from outside the excavator 100. When the excavator 100 is remotely operated, the interior of the cab 10 can be in an unmanned state.

[0093] The auxiliary storage device D4 stores the learned model LM and the object storage database D4A.

[0094] When the captured image data captured by the imaging device S6 is input from the input layer, the learned model LM outputs, from the output layer, the coordinate region where the person represented in the captured image data exists and the frame size for surrounding the person.

[0095] Furthermore, the learned model LM according to the present embodiment can also detect objects other than people. For example, the learned model LM can detect construction machinery. Specifically, when the captured image data captured by the imaging device S6 is input from the input layer, the learned model LM outputs, from the output layer, the coordinate region where the construction machinery represented in the captured image data exists and the frame size for surrounding the construction machinery.

[0096] In addition, the present embodiment is not limited to the method in which the learned model LM according to the present embodiment outputs the coordinate region where the construction machinery or the person exists, and the frame size for surrounding the construction machinery or the person, as long as it can output information for identifying the region where the construction machinery or the person exists.

[0097] As the machine learning for generating the learned model LM, for example, a neural network can be applied. Specifically, machine learning using a deep neural network (DNN) can be applied, and it is deep learning. As deep learning, for example, a convolutional neural network, an RNN (Recurrent Neural Networks), or an LSTM (Long Short Term Memory) can be applied.

[0098] The learned model LM is generated by performing machine learning based on a pre-generated training data set in an information processing device (not shown).

[0099] Specifically, the learned model LM is generated by machine learning based on the captured image data representing a person or a construction machine included in the training dataset, as well as the coordinate region and the frame size where the person or the construction machine represented in the captured image data exists.

[0100] In addition, the learned model LM can be updated by making the existing learned model LM additionally learn a new training dataset.

[0101] The object storage database D4A is set as a database for storing objects to be suppressed from detection among the objects detected as a person or a construction machine by the learned model LM.

[0102] The object storage database D4A according to the present embodiment stores the image data representing the object to be suppressed from detection. In addition, the present embodiment is not limited to the manner of storing the image data representing the object to be suppressed from detection, and the feature information extracted from the image data representing the object may also be stored.

[0103] The number of the image data that can be stored in the object storage database D4A can be arbitrary. For example, it can be dozens, or it can be hundreds or more. The image data stored in the object storage database D4A can be initialized at any time. For example, it can be initialized every time the job site is changed, or it can be initialized on a daily basis.

[0104] The controller 30 includes an acquisition unit 301, a detection unit 302, an output control unit 303, an operation reception unit 304, a setting unit 305, and a determination unit 306.

[0105] The acquisition unit 301 acquires various information from various sensors. For example, the acquisition unit 301 acquires the captured image data captured by the imaging devices S6 (left camera S6L, right camera S6R, and rear camera S6B).

[0106] The acquisition unit 301 acquires the detection information respectively detected by the boom angle sensor S1, the arm angle sensor S2, the bucket angle sensor S3, the body tilt sensor S4, and the slewing angle sensor S5. The acquisition unit 301 acquires the position and orientation of the excavator 100 from the positioning device PS.

[0107] The detection unit 302 performs detection processing of people and construction machinery existing around the excavator 100 based on the captured image data obtained by the acquisition unit 301. The detection unit 302 according to this embodiment inputs the captured image data into the learned model LM, and thereby receives from the learned model LM the coordinate regions where the construction machinery or people exist and the frame sizes for surrounding the construction machinery or people. In this embodiment, a method of using the learned model LM to detect one or more of people and construction machinery will be described. However, this embodiment does not limit the method of detecting one or more of people and / or construction machinery, and any method can be used regardless of whether it is a well-known method. For example, a determination can be made as to whether a feature amount extracted from the captured image data is approximately equal to or greater than a specified value compared to a preset feature amount representing a person. Also, the detection target is not limited to people or construction machinery, and can also be other objects.

[0108] The output control unit 303 outputs the rotation angle, the coordinate regions and frame sizes of one or more of people and construction machinery, and the detection results of various sensors to the control unit D3a of the first display device D3. Thereby, the first display device D3 displays a screen showing the surroundings of the excavator 100.

[0109] Next, referring to Figure 5 , an example of the display screen displayed on the first display device D3 will be described. Figure 5 FIG. is an example of the display screen 41 displayed on the first display device D3 and the input device D2 according to this embodiment.

[0110] The control unit D3a according to this embodiment generates a display screen based on the image data input from the imaging device S6 and various information received from the controller 30. The information received from the controller 30 includes the rotation angle, the coordinate regions where the construction machinery or people exist, the frame sizes for surrounding the construction machinery or people, and the detection results of various sensors.

[0111] As Figure 5 shown, the display screen 41 includes a date and time display area 41a, a travel mode display area 41b, an attachment display area 41c, a fuel consumption rate display area 41d, an engine control state display area 41e, an engine operation time display area 41f, a coolant water temperature display area 41g, a fuel remaining amount display area 41h, a rotation speed mode display area 41i, a urea water remaining amount display area 41j, a working oil temperature display area 41k, an air conditioner operation state display area 41m, an image display area 41n, and a menu display area 41p.

[0112] The traveling mode display area 41b, the attachment display area 41c, the engine control status display area 41e, the rotation speed mode display area 41i, and the air conditioner operation status display area 41m are areas that display information related to the set state of the excavator 100, that is, set state information. The fuel consumption rate display area 41d, the engine operation time display area 41f, the coolant water temperature display area 41g, the fuel remaining amount display area 41h, the urea water remaining amount display area 41j, and the working oil temperature display area 41k are areas that display information related to the operation state of the excavator 100, that is, operation state information.

[0113] Specifically, the date and time display area 41a is an area that displays the current date and time. The traveling mode display area 41b is an area that displays the current traveling mode. The attachment display area 41c is an area that displays an image indicating the currently installed attachment. The fuel consumption rate display area 41d is an area that displays the fuel consumption rate information calculated by the controller 30. The fuel consumption rate display area 41d includes an average fuel consumption rate display area 41d1 that displays the life average fuel consumption rate or the interval average fuel consumption rate, and an instantaneous fuel consumption rate display area 41d2 that displays the instantaneous fuel consumption rate.

[0114] The engine control status display area 41e is an area that displays the control status of the engine 11. The engine operation time display area 41f is an area that displays the cumulative operation time of the engine 11. The coolant water temperature display area 41g is an area that displays the temperature status of the current engine coolant water. The fuel remaining amount display area 41h is an area that displays the remaining amount status of the fuel stored in the fuel tank. The rotation speed mode display area 41i is an area that displays the current rotation speed mode set by the engine rotation speed adjustment dial 75 in an image. The urea water remaining amount display area 41j is an area that displays the remaining amount status of the urea water stored in the urea water tank in an image. The working oil temperature display area 41k is an area that displays the temperature status of the working oil in the working oil tank.

[0115] The air conditioner operation status display area 41m includes an air outlet display area 41m1 that displays the position of the current air outlet, an operation mode display area 41m2 that displays the current operation mode, a temperature display area 41m3 that displays the current set temperature, and an air volume display area 41m4 that displays the current set air volume.

[0116] The image display area 41n is an area that displays the image captured by the imaging device S6. Figure 5In the example, the image display area 41n displays an overhead view image FV and a rear image CBT. The overhead view image FV is a virtual viewpoint image generated by the control unit D3a based on the images respectively acquired by the rear camera S6B, the left camera S6L, and the right camera S6R. And, an excavator graphic GE corresponding to the excavator 100 is arranged in the central part of the overhead view image FV. This is to enable the operator to more intuitively grasp the positional relationship between the excavator 100 and the objects existing around the excavator 100. The rear image CBT is an image showing the rear space of the excavator 100, including an image GC of the counterweight. The rear image CBT is an actual viewpoint image generated by the control unit D3a based on the image acquired by the rear camera S6B.

[0117] And, the image display area 41n has a first image display area 41n1 located above and a second image display area 41n2 located below. In Figure 5 the example, the overhead view image FV is arranged in the first image display area 41n1, and the rear image CBT is arranged in the second image display area 41n2. However, the image display area 41n may arrange the overhead view image FV in the second image display area 41n2 and arrange the rear image CBT in the first image display area 41n1. And, in Figure 5 the example, the overhead view image FV and the rear image CBT are arranged adjacent to each other vertically, but they may also be arranged with an interval therebetween. And, in Figure 5 the example, the image display area 41n is a vertically long area, but the image display area 41n may also be a horizontally long area. When the image display area 41n is a horizontally long area, the image display area 41n may arrange the overhead view image FV as the first image display area 41n1 on the left side and arrange the rear image CBT as the second image display area 41n2 on the right side. At this time, they may be arranged with an interval therebetween left and right, or the positions of the overhead view image FV and the rear image CBT may be swapped. Moreover, when a front camera is provided on the upper swing body 3, in the image display areas included in the image display area 41n, a front image showing the front space of the excavator 100 imaged by the front camera may also be arranged.

[0118] When the controller 30 detects a construction machine or a person from the rear image (an example of captured image data) CBT disposed in the second image display area 41n2, a frame (an example of display information) is displayed in the second image display area 41n2 so as to surround the construction machine or the person. Specifically, a frame 1501b for indicating the detected person 1501a is displayed in the second image display area 41n2. Moreover, a frame 1502b for indicating the detected dump truck 1502a is displayed in the second image display area 41n2. This frame is an example of display information configured according to the coordinate area and the frame size received from the controller 30. This embodiment has described an example in which a frame is used as display information for indicating a construction machine or a person. However, this embodiment does not limit the display information for indicating a construction machine or a person to a frame, and for example, it may also be an icon (a mark indicating an exclamation point, an icon of a face, or an icon urging a sense of crisis, etc.).

[0119] Moreover, the color of the frame can be changed according to the type of the object surrounded by the frame. For example, the color of the frame surrounding a person can be made different from the color of the frame surrounding a construction machine.

[0120] In addition, this embodiment has described an example in which a dump truck as an example of a construction machine is surrounded by a frame, but does not limit the construction machine surrounded by the frame to a dump truck. For example, it may also be an excavator, a crawler crane, a jib crane, an asphalt paver, or a roller, etc.

[0121] The menu display area 41p has labels 41p1 to 41p7. In Figure 5 the example, at the lowermost part of the display screen 41, the labels 41p1 to 41p7 are arranged at intervals from each other left and right. Icons for displaying various information are displayed on the labels 41p1 to 41p7.

[0122] A menu detailed item icon for displaying menu detailed items is displayed on the label 41p1. If the operator selects the label 41p1, the icons displayed on the labels 41p2 to 41p7 are switched to icons associated with the menu detailed items.

[0123] An icon for displaying information related to a digital level is displayed on the label 41p4. If the operator selects the label 41p4, the rear image CBT is switched to a screen showing information related to the digital level. However, a screen showing information related to the digital level may also be displayed by overlapping it on the rear image CBT or reducing the rear image CBT. Moreover, the bird's-eye view image FV may also be switched to a screen showing information related to the digital level, and a screen showing information related to the digital level may also be displayed by overlapping it on the bird's-eye view image FV or reducing the bird's-eye view image FV.

[0124] An icon for displaying information related to information-based construction is shown on label 41p6. If the operator selects label 41p6, the rear image CBT switches to a screen representing information related to information-based construction. However, a screen representing information related to information-based construction can also be displayed by overlapping it on the rear image CBT or reducing the rear image CBT. Also, the bird's-eye view image FV can be switched to a screen representing information related to information-based construction, and a screen representing information related to information-based construction can also be displayed by overlapping it on the bird's-eye view image FV or reducing the bird's-eye view image FV.

[0125] An icon for displaying information related to the crane mode is shown on label 41p7. If the operator selects label 41p7, the rear image CBT switches to a screen representing information related to the crane mode. However, a display screen representing information related to the crane mode can also be displayed by overlapping it on the rear image CBT or reducing the rear image CBT. Also, the bird's-eye view image FV can be switched to a screen representing information related to the crane mode, and a screen representing information related to the crane mode can also be displayed by overlapping it on the bird's-eye view image FV or reducing the bird's-eye view image FV.

[0126] No icons are shown on labels 41p2, 41p3, and 41p5. Therefore, even if the operator operates labels 41p2, 41p3, and 41p5, there will be no change in the image displayed on the display screen 41.

[0127] In addition, the icons displayed on labels 41p1 to 41p7 are not limited to the above examples, and icons for displaying other information can also be shown.

[0128] Next, the input device D2 will be described. As Figure 5 shown, the input device D2 is composed of one or more push-button switches for the operator to select, set, and input labels 41p1 to 41p7. In Figure 5 the example, the input device D2 includes seven switches 42a1 to 42a7 arranged in the upper section and seven switches 42b1 to 42b7 arranged in the lower section. The switches 42b1 to 42b7 are arranged below the respective switches 42a1 to 42a7. However, the number, method, and arrangement of the switches of the input device D2 are not limited to the above example. For example, it can be a method in which a roller, a scroll switch, etc. aggregate the functions of multiple push-button switches into one, and the input device D2 can be separated from the first display device D3.

[0129] The switches 42a1 to 42a7 are respectively arranged below the labels 41p1 to 41p7 corresponding to the labels 41p1 to 41p7, and respectively function as switches for selecting the labels 41p1 to 41p7. Since the switches 42a1 to 42a7 are respectively arranged below the labels 41p1 to 41p7 corresponding to the labels 41p1 to 41p7, the operator can intuitively select the labels 41p1 to 41p7.

[0130] The switch 42b1 is a switch for switching the captured image displayed in the image display area 41n. It is configured such that each time the switch 42b1 is operated, the captured image displayed in the first image display area 41n1 of the image display area 41n is switched, for example, among the rear image, the left image, the right image, and the overhead image. And it can be configured such that each time the switch 42b1 is operated, the captured image displayed in the second image display area 41n2 of the image display area 41n is switched, for example, among the rear image, the left image, the right image, and the overhead image. And it can be configured such that each time the switch 42b1 is operated, the captured image displayed in the first image display area 41n1 and the captured image displayed in the second image display area 41n2 are swapped. Thus, the switch 42b of the input device D2 can switch the screen displayed in the first image display area 41n1 or the second image display area 41n2, and can also switch the screens displayed in the first image display area 41n1 and the second image display area 41n2. And the switch for switching the screen displayed in the second image display area 41n2 can be separately provided.

[0131] The switches 42b2 and 42b3 are switches for adjusting the air volume of the air conditioner. In Figure 5 the example, it is configured such that if the switch 42b2 is operated, the air volume of the air conditioner becomes smaller, and if the switch 42b3 is operated, the air volume of the air conditioner becomes larger.

[0132] The switch 42b4 is a switch for switching the ON / OFF (on / off) of the cooling and heating functions. In Figure 5 the example, it is configured such that each time the switch 42b4 is operated, the ON / OFF (on / off) of the cooling and heating functions is switched.

[0133] The switches 42b5 and 42b6 are switches for adjusting the set temperature of the air conditioner. In Figure 5 the example, it is configured such that if the switch 42b5 is operated, the set temperature becomes lower, and if the switch 42b6 is operated, the set temperature becomes higher.

[0134] The switch 42b7 is a switch for switching the display of the engine operation time display area 41f.

[0135] Moreover, the switches 42a2 to 42a6 and 42b2 to 42b6 are configured to be able to input the numbers displayed on or near each switch. Further, the switches 42a3, 42a4, 42a5, and 42b4 are configured to be able to move the cursor leftward, upward, rightward, and downward, respectively, when the cursor is displayed on the menu screen.

[0136] In addition, as an example of the functions assigned to the switches 42a1 to 42a7 and 42b1 to 42b7, they may also be configured to be able to perform other functions.

[0137] Furthermore, the input device D2 according to the present embodiment includes a touch panel that can accept an operation indicating arbitrary position coordinates of the display screen 41 displayed on the first display device D3. Thus, the input device D2 can directly operate the labels 41p1 to 41p7. Moreover, the input device D2 can directly operate the top-view image FV and the rear image CBT in the image display area 41n. The direct operation of the rear image CBT will be described later.

[0138] Return to Figure 4 The operation receiving unit 304 receives the information input to the operation device 26 from the operation sensor 29. Moreover, the operation receiving unit 304 receives the information input to the input device D2 from the input device D2.

[0139] For example, the operation receiving unit 304 receives an operation of specifying an object surrounded by a frame in the captured image data displayed in the first image display area 41n1 or the second image display area 41n2 via the touch panel of the input device D2. In the present embodiment, an intuitive operation is achieved by specifying an object via the touch panel, so that the operability can be improved. In addition, as an example of the operation of specifying an object, the example via the touch panel is described in the present embodiment, but it is not limited to the operation via the touch panel, and other operations may also be possible. For example, an object can be selected by pressing a button of the input device D2.

[0140] For example, when the controller 30 erroneously detects an object, the first display device D3 displays a frame indicating the erroneously detected object for the object. In the present embodiment, in order to make the frame non-displayed for the erroneously detected object, the operation receiving unit 304 receives an operation of specifying an object surrounded by a frame.

[0141] When the setting unit 305 receives an operation for specifying an object represented in the captured image data, it performs a setting for suppressing the detection of the object in the image data (an example of information representing the object) representing the object. Specifically, the setting unit 305 registers the image data representing the specified object in the object storage database D4A for storing the objects whose detection is to be suppressed. In other words, the image data registered in the object storage database D4A is regarded as the data for which the setting for suppressing the detection has been performed. In addition, in this embodiment, an example of registering in the object storage database D4A is described as an example of the setting for suppressing the detection, but as long as it is a process capable of suppressing the detection, other methods may also be used.

[0142] The determination unit 306 determines whether a part of the region represented by the coordinate region and the frame size detected by the detection unit 302 in the captured image data is similar to the image data registered in the object storage database D4A by a specified threshold or more. The specified threshold can be, for example, 80%, and is determined according to the embodiment.

[0143] When the determination unit 306 determines that it is similar by a specified threshold or more, the object is regarded as an object to be suppressed from detection, and the output of the coordinate region and the frame size related to the object to the first display device D3 is suppressed. As a result, the first display device D3 suppresses the display of the frame indicating the object.

[0144] In this way, when an object represented by the image data registered in the object storage database D4A is detected from the captured image data, the first display device D3 suppresses the display of the frame (an example of display information) indicating the object. In this embodiment, it is possible to suppress the display of a person or the like that has been misdetected and surrounded by a frame. Therefore, when the operator refers to the display screen, there is no need to confirm that the object surrounded by the frame is not a person, so troublesomeness can be suppressed. Therefore, an improvement in convenience can be achieved.

[0145] Figure 6 FIG. is a diagram showing the transition of the rear image CBT arranged in the second image display area 41n2 based on the control of the controller 30 and the first display device D3.

[0146] In Figure 6 In the rear image CBT arranged in the second image display area 41n2 shown in (a) of, an example is assumed in which a person drawn on the sign 1602a is detected together with the person 1601a. Therefore, the first display device D3 displays the frame 1601b surrounding the person 1601a and also displays the frame 1602b surrounding the person drawn on the sign 1602a.

[0147] In addition, Figure 6The example shown illustrates an example of false detection, but human detection is not limited to humans drawn on signs or the like. For example, even construction machinery coated with reflective materials existing in the distance may be falsely detected as a person wearing a fluorescent material work uniform, and traffic cones or the like existing at the work site may also be falsely detected as a person.

[0148] That is, various objects exist at the work site, so the controller 30 may falsely detect various objects as a person. When these objects are falsely detected as a person, the first display device D3 displays a frame (indicating a person) attached to the falsely detected object. At this time, when the operator refers to the screen displayed on the first display device D3, it is necessary to recognize the situation of false detection, so it may be troublesome. Moreover, the controller 30 may perform safety control based on the result of false detection. In Figure 6 In the example shown in (a) of

[0149] Since the sign 1602a is falsely detected as a person, the controller 30 may actuate a function that restricts walking or turning to avoid contact with the sign 1602a. Moreover, the controller 30 may output an alarm sound when a person approaches. In this case, the operator may not be able to perform effective work.

[0150] Moreover, as Figure 6 shown in (b) of

[0151] When a press on the frame 1602b or inside the frame 1602b is received, the first display device D3 displays a pop-up window 1620. In the pop-up window 1620, the message "Do not detect this object hereafter?" is displayed, and an OK button 1621 and a cancel button 1622 are displayed. When the operation receiving unit 304 receives a press on the cancel button 1622, the first display device D3 closes the pop-up window 1620.

[0152] When the operation receiving unit 304 receives a press on the OK button 1621, the setting unit 305 registers the image data of the object corresponding to the frame 1602b (in other words, the image data represented by the coordinate area and the frame size) in the object storage database D4A. Moreover, the first display device D3 closes the pop-up window 1620.

[0153] When a press on the OK button 1621 is received and the image data of the object corresponding to the frame 1602b is registered in the object storage database D4A, the determination unit 306 determines the person drawn on the sign 1602a as an object to be suppressed from detection.

[0153] Therefore, as Figure 6As shown in (c) of FIG. 0, the first display device D3 suppresses the display of the frame 1602b that encloses the person drawn on the sign 1602a.

[0154] In the present embodiment, by performing the above control, it is possible to make the frame displayed due to false detection non-displayed.

[0155] As described above, in the present embodiment, based on the setting by the setting unit 305 for suppressing the detection target object, it is performed according to the operation received from the input device D2 during the operation of the excavator 100. That is, during the operation of the excavator 100, when false detection of an object existing at the work site occurs, it is possible to immediately suppress the detection of the object. Therefore, after the operator makes the setting, during the operation by the excavator 100, it is not necessary to confirm whether false detection occurs, so the operation can be performed comfortably. Moreover, it is possible to suppress the safety control by the controller 30 according to the result of false detection, so that an improvement in operation efficiency can be achieved. In addition, the present embodiment is not limited to the method of setting for suppressing the detection target object during operation. For example, the setting for suppressing the detection target object can be performed before the excavator 100 starts running or the like.

[0156] During the operation of the excavator 100, it is set to at least the period when the power of the excavator 100 is turned on, and is the period when the operation of the excavator 100 can be performed according to the operator's operation. As a specific example, it is set to the period when the excavator 100 performs excavation, leveling, loading, rotation, or movement. Even when false detection occurs during the operation of the excavator 100, the operator performs an operation for suppressing false detection to suppress false detection in subsequent operations. That is, in the present embodiment, it is possible to suppress false detection without waiting for the end of the operation.

[0157] This embodiment is in Figure 6 In the example shown, an example of suppressing the detection of the person drawn on the sign 1602a is described, but the detection suppression target can be any object, and as long as it is an object indicated by a frame, any object can be registered in the object storage database D4A.

[0158] The controller 30 according to the present embodiment determines whether there is similarity between a partial area of the captured image data and the image data registered in the object storage database D4A. That is, when the object represented in a partial area of the captured image data is a stationary object, it can be determined as similar with higher accuracy.

[0159] Assume that when human image data is erroneously registered in the object storage database D4A, since the human is moving, in the similarity determination between a partial area of the captured image data showing the human performed by the determination unit 306 and the human image data registered in the object storage database D4A, the possibility of determining dissimilarity is relatively high. That is, since it is difficult to set the human as the suppression object of detection, safety can be maintained.

[0160] In the past, every time the upper rotating body rotated, when an approaching object was erroneously detected as a human, safety control such as outputting an alarm sound was activated. Therefore, it became a troublesome situation for the operator. In contrast, in the present embodiment, every time the upper rotating body 3 rotates, the image of the object is registered in the object storage database D4A multiple times, thereby suppressing the erroneous detection of the object that occurs every time the upper rotating body 3 rotates. Therefore, the activation of safety control can be suppressed, and thus an improvement in work efficiency can be achieved.

[0161] The first display device D3 according to the present embodiment is not limited to the method of restricting the frame to non-display for the object for which suppression detection has been set. For example, the first display device D3 can be such that for the object for which suppression detection has been set, the frame is set to non-display, and information indicating suppression detection is displayed near the object. As the information indicating suppression detection, for example, a small icon showing an exclamation mark is set. The information indicating suppression detection is set to be displayed to such an extent that it does not interfere with the operation of the operator. Moreover, by displaying such an icon near the object, the first display device D3 can enable the operator to recognize that the detection of the object is suppressed.

[0162] Next, the processing sequence executed by the controller 30 and the first display device D3 according to the present embodiment will be described. Figure 7 It is a flowchart showing the setting sequence for suppressing the detection of an object in the controller 30 and the first display device D3 according to the present embodiment. Additionally, Figure 7 the shown flowchart is set for the case where no image data is registered in the object storage database D4A. The processing sequence shown in the flowchart according to the present embodiment is repeated at regular intervals.

[0163] First, the acquisition unit 301 acquires the captured image data captured by the imaging device S6 (step S1701).

[0164] The detection unit 302 inputs the captured image data acquired by the acquisition unit 301 into the learned model LM, thereby receiving the coordinate area and frame size of a person or construction machinery existing around the excavator 100 (S1702).

[0165] The first display device D3 surrounds the detected person or construction machinery with a frame according to the coordinate area and the frame size, and displays it together with the captured image data (S1703).

[0166] The operation reception unit 304 determines whether an operation of designating an object surrounded by a frame in the first image display area 41n1 or the second image display area 41n2 via the touch panel of the input device D2 is received (S1704). When it is determined that the designation operation is not received (S1704: "No"), the process ends.

[0167] On the other hand, when it is determined that the operation reception unit 304 has received an operation of designating an object surrounded by a frame (S1704: "Yes"), the first display device D3 displays a pop-up window for confirming the suppression of the detection of the object (S1705).

[0168] On the other hand, it is determined whether the operation reception unit 304 has received a press of the OK button 1621 (step S1706). When it is determined that the operation reception unit 304 has not received a press of the OK button 1621, in other words, when a press of the cancel button is received (step S1706: "No"), the process ends.

[0169] On the other hand, when it is determined that the operation reception unit 304 has received a press of the OK button 1621 (step S1706: "Yes"), the setting unit 305 registers the image data represented by the coordinate area and the frame size of the object in the object storage database D4A as a setting for suppressing the detection of the object (step S1707).

[0170] The controller 30 and the first display device D3 according to the present embodiment can achieve a setting for suppressing detection by performing the above control.

[0171] In the present embodiment, according to Figure 7 the processing sequence shown, when the image data related to the object is registered in the object storage database D4A, the controller 30 determines whether to suppress the detection of the object when displaying the captured image data on the first display device D3.

[0172] Figure 8 FIG. is a flowchart showing the processing sequence in the controller 30 and the first display device D3 according to the present embodiment when making a determination using the object storage database D4A when displaying the captured image data.

[0173] First, the acquisition unit 301 acquires the captured image data captured by the imaging device S6 (S1801).

[0174] The detection unit 302 inputs the captured image data acquired by the acquisition unit 301 into the learned model LM, thereby receiving the coordinate regions and bounding box sizes of the people or construction machinery existing around the excavator 100 (S1802).

[0175] The determination unit 306 determines whether a part of the region determined based on the received coordinate regions and bounding box sizes in the captured image data is consistent with the image data registered in the object storage database D4A by a specified threshold or more (S1803). When it is determined that it is not consistent by the specified threshold or more (S1803: "No"), the received coordinate regions and bounding box sizes are output to the first display device D3 (S1804).

[0176] On the other hand, when the determination unit 306 determines that a part of the region determined based on the received coordinate regions and bounding box sizes in the captured image data is consistent with the image data registered in the object storage database D4A by a specified threshold or more (S1803: "Yes"), it suppresses the output of the received coordinate regions and bounding box sizes to the first display device D3 (S1805).

[0177] Moreover, the determination unit 306 determines whether the determination of all the coordinate regions and bounding box sizes received from the learned model LM has been completed (S1806). When it is determined that it has not been completed (S1806: "No"), the process proceeds again from S1803.

[0178] When the determination unit 306 determines that it has been completed (S1806: "Yes"), the first display device D3 displays the people or construction machinery surrounded by a bounding box together with the captured image data based on the input coordinate regions and bounding box sizes (S1807).

[0179] In the present embodiment, according to the above processing sequence, it is possible to suppress the display surrounded by a bounding box for the objects excluded from the detection target.

[0180] Moreover, when the controller 30 has a function of performing safety control based on the detected objects, the safety control based on the excluded objects is suppressed. That is, the controller 30 can suppress the safety control based on the erroneously detected objects, and on the other hand, make the safety control based on the appropriately detected objects function. Therefore, the present embodiment can achieve an improvement in work efficiency and safety.

[0181] (Modification Example of the First Embodiment)

[0182] In the above embodiment, an example of suppressing detection using the image data stored in the object storage database D4A has been described. However, the above embodiment is not limited to the method of using the image data stored in the object storage database D4A only for suppression detection.

[0183] Therefore, in the modification of the first embodiment, the image data stored in the object storage database D4A can be used as training data to perform relearning of the learned model LM. For example, relearning can be performed when a predetermined number of image data are stored in the object storage database D4A.

[0184] To perform relearning, the controller 30 can generate training data from the image data stored in the object storage database D4A, or an additionally provided information processing device can generate training data from the image data.

[0185] Furthermore, regarding the relearning of the learned model LM based on the generated training data, it can be performed by the controller 30 or by an additionally provided information processing device. The relearned learned model LM is stored in the auxiliary storage device D4. Thereafter, the same processing as in the above embodiment is performed. In this modification, by performing relearning using the image data related to the erroneously detected object, an improvement in detection accuracy can be achieved.

[0186] (Second Embodiment)

[0187] In the above embodiment, an example of detecting a person from the captured image data captured by the imaging device S6 has been described. However, the above embodiment is not limited to the method of detecting a person from the captured image data captured by the imaging device S6. Therefore, in the second embodiment, a case where a space recognition device S7 is provided in addition to the imaging device S6 will be described.

[0188] Figure 9 FIG. 17 is a side view of the excavator 100A according to the second embodiment. In addition, in the present embodiment, the same reference numerals are assigned to the same structures as those in the first embodiment, and the description thereof is omitted.

[0189] The space recognition device S7 detects whether there is an object in the surrounding space of the excavator 100A and the distance to the object, etc. The space recognition device S7 outputs the result of measuring the space as measurement information to the controller 30A.

[0190] The space recognition device S7 includes a rear space recognition device S7B that detects the rear space of the excavator 100A, a left space recognition device S7L that detects the left space of the excavator 100A, and a right space recognition device S7R that detects the right space of the excavator 100A.

[0191] The space recognition device S7 can use LIDAR (Light Detection and Ranging) to detect objects present around the excavator 100A. LIDAR measures the distance between, for example, more than one million points within the monitoring range and the LIDAR. Additionally, the method of the present embodiment is not limited to using LIDAR, and any space recognition device capable of measuring the distance to an object can be used. For example, a stereo camera can be used, or a distance image camera or a ranging device such as a millimeter-wave radar can be used. When using a millimeter-wave radar or the like as the space recognition device S7, by sending multiple signals (such as laser beams) from the space recognition device S7 towards the object and receiving the reflected signals, the distance and direction of the object can be derived from the reflected signals.

[0192] The rear space recognition device S7B is installed at the rear end of the upper surface of the upper swing body 3. The left space recognition device S7L is installed at the left end of the upper surface of the upper swing body 3. The right space recognition device S7R is installed at the right end of the upper surface of the upper swing body 3.

[0193] The rear space recognition device S7B, the left space recognition device S7L, and the right space recognition device S7R are all installed on the upper swing body 3 in such a way that their optical axes face obliquely downward and a part of the upper swing body 3 is included in the detection range. Therefore, the detection ranges of the rear space recognition device S7B, the left space recognition device S7L, and the right space recognition device S7R each have a field of view angle of approximately 180 degrees, for example, when viewed from above.

[0194] Moreover, the controller 30 maintains the correspondence relationship between the detection ranges of the rear space recognition device S7B, the left space recognition device S7L, and the right space recognition device S7R and the imaging ranges of the rear camera S6B, the left camera S6L, and the right camera S6R. That is, when an object is detected by the rear space recognition device S7B, the left space recognition device S7L, and the right space recognition device S7R, the area where the object exists can be identified based on the captured image data obtained by the rear camera S6B, the left camera S6L, and the right camera S6R.

[0195] The difference between the controller 30A according to the present embodiment and the controller 30 of the above embodiment is that the detection results of the space recognition device S7 are used to determine the coordinate area and the bounding box size where a person or construction machinery exists.

[0196] In the controller 30A according to the present embodiment, the position and size of a person or construction machinery in the actual space are inferred based on the detection result of the space recognition device S7. As a method for inferring the position and size where a person or construction machinery exists based on the detection result, a known method can be used. For example, a learned model can be used. For example, the controller 30A can receive the position and size where a person or construction machinery exists by inputting the detection result of the space recognition device S7 into the learned model.

[0197] Moreover, in the controller 30A, the received position and size in the actual space are converted into a coordinate region and a frame size in the captured image data. This conversion is performed according to the above correspondence relationship, and the description thereof is omitted.

[0198] After the controller 30A obtains the coordinate region and the frame size where a person or construction machinery exists, the same control as in the above embodiment is performed. That is, the first display device D3 overlays and displays a frame indicating the object detected by the space recognition device S7 on the captured image data.

[0199] Moreover, when the operation reception unit 304 receives an operation for suppressing the detection object from the operator, the setting unit 305 registers the detection data (for example, the position and size in the actual space) indicating the result of detecting the object by the space recognition device S7 into the object storage database D4A as a setting for suppressing the detection of the object.

[0200] Then, when the controller 30A infers the position and size of a person or construction machinery in the actual space based on the detection data of the space recognition device S7, the determination unit 306 determines whether the position and size of the person or construction machinery are similar to the detection data registered in the object storage database D4A by a threshold value or more.

[0201] Moreover, when the controller 30A determines that the position and size of the person or construction machinery inferred based on the detection data of the space recognition device S7 are similar to the detection data registered in the object storage database D4A by a threshold value or more, the first display device D3 suppresses the display of the frame indicating the person or construction machinery. And the controller 30A can also suppress the safety control based on the person or construction machinery.

[0202] In the present embodiment, when the space recognition device S7 is used, the same effect as in the above embodiment can also be obtained. When the detection result of the space recognition device S7 is used, it is also possible to suppress the trouble for the operator and improve the convenience. Moreover, when an object is misdetected by the space recognition device S7, the misdetection can be corrected, so that the detection accuracy can be improved.

[0203] This embodiment shows a case where detection data is used as an example of information representing an object, and the first embodiment shows a case where image data is used as an example of information representing an object. However, the above embodiments do not limit the information representing the object to detection data or image data, as long as it is information for determining the object. For example, the characteristic information of the object or the like can be used as the information representing the object.

[0204] (Third Embodiment)

[0205] In the above embodiment, an example of processing the single excavator 100 ridden by an operator is described. However, the above embodiment is not limited to the method of processing by the single excavator 100. For example, it can also be processed by a management server connected to the excavator 100. Therefore, in the third embodiment, a case of processing by a system composed of the excavator 100, a management server that manages the excavator 100, and a client is described.

[0206] For this reason, with reference to Figure 10 , the outline of the control system SYS of the excavator related to the third embodiment is described. Figure 10 It is a schematic diagram showing an example of the control system SYS related to the third embodiment.

[0207] As Figure 10 shown, the control system SYS related to the third embodiment includes an excavator 100, a management server 2000, and a remote operation room RC.

[0208] The excavator 100 related to this embodiment can be operated by an operator riding in the cab 10, or can be operated by an operator OP present in the remote operation room RC.

[0209] <Example of the Structure of the Remote Operation Room>

[0210] In the remote operation room RC, a communication device T2, a remote controller R30, an operation device R26, an operation sensor R29, and a display device DR are provided. And, an operation seat DS on which the operator OP who remotely operates the excavator 100 sits is provided in the remote operation room RC.

[0211] The communication device T2 is configured to control communication with the communication device T1 installed on the excavator 100.

[0212] The remote controller (an example of a remote operation device) R30 is an arithmetic device that performs various operations. In this embodiment, the remote controller R30 is composed of a microcomputer including a CPU and a memory. Moreover, various functions of the remote controller R30 are realized by the CPU executing a program stored in the memory.

[0213] The display device DR displays a screen based on the information sent from the excavator 100, enabling the operator OP located in the remote operation room RC to visually recognize the surroundings of the excavator 100. Even though the operator is located in the remote operation room RC, the display device DR can also enable the operator to confirm the situation of the work site including the surroundings of the excavator 100.

[0214] Moreover, similar to the display screen 41 shown in Figure 5 the first embodiment, the display device DR displays the captured image data of a person or a construction machine surrounded by a frame.

[0215] An operation sensor R29 for detecting the operation content of the operation device R26 is provided in the operation device R26. The operation sensor R29 is, for example, an inclination sensor for detecting the inclination angle of the operation lever, or an angle sensor for detecting the swing angle of the operation lever around the swing axis, etc. The operation sensor R29 can be composed of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor R29 outputs information related to the detected operation content of the operation device R26 to the remote controller R30. The remote controller R30 generates an operation signal based on the received information and sends the generated operation signal to the excavator 100. The operation sensor R29 can also be configured to generate an operation signal. In this case, the operation sensor R29 can output the operation signal to the communication device T2 without passing through the remote controller R30. Thus, it is possible to remotely operate the excavator 100 from the remote operation room RC.

[0216] Moreover, the communication device T1 of the excavator 100 receives the operation signal from the communication device T2 of the remote controller R30. The controller 30 of the excavator 100 performs various operations at the work site according to the received operation signal.

[0217] <Control related to the management server>

[0218] The excavator 100 uses the communication device T1 provided in the excavator 100 to send the detection results from various sensors provided in the excavator 100 to the management server 2000. For example, the excavator 100 sends the captured image data taken by the imaging device S6 to the management server 2000. And, when a space recognition device S7 is provided on the excavator 100, the detection result of the space recognition device S7 is sent to the management server 2000.

[0219] The management server 2000 according to this embodiment has the same structure as the controller 30 of the above embodiment, and stores the learned model LM and the object storage database D4A.

[0220] Therefore, when the management server 2000 receives the captured image data from the excavator 100, by inputting the received captured image data into the learned model LM, the coordinate regions and bounding box sizes of the people or construction machinery existing around the excavator 100 are thereby received.

[0221] Moreover, when the excavator 100 is operated by an operator, the management server 2000 sends the coordinate regions and bounding box sizes of the people or construction machinery to the communication device T1 of the excavator 100.

[0222] And, when the excavator 100 is operated from the remote operation room RC, the management server 2000 sends the coordinate regions and bounding box sizes of the people or construction machinery to the communication device T2 of the remote operation room RC.

[0223] Thereby, the first display device D3 of the excavator 100 or the display device DR of the remote operation room RC can display the people or construction machinery represented in the captured image data surrounded by a bounding box.

[0224] Moreover, similarly to the above-described embodiment, the management server 2000 also performs control for correcting the detection result.

[0225] Figure 11 It is a timing chart showing the setting order for suppressing detection objects in the control system SYS according to the present embodiment. Figure 11 The example shown illustrates the case where the operator riding in the cab 10 performs an operation. Additionally, when the operator OP in the remote operation room RC performs an operation, the same control is performed except for the destination of the information sent from the management server 2000, and the description thereof is omitted. Additionally, Figure 11 The shown timing chart is set for the case where the image data is not registered in the object storage database D4A.

[0226] First, the controller 30 of the excavator 100 acquires the captured image data captured by the imaging device S6 (S2101).

[0227] Then, the controller 30 sends the acquired captured image data to the management server 2000 via the communication device T1 (S2102).

[0228] The management server 2000 inputs the received captured image data into the learned model LM, thereby receiving the coordinate regions and bounding box sizes of the people or construction machinery existing around the excavator 100 (S2103).

[0229] The management server 2000 sends the received coordinate regions and bounding box sizes to the communication device T1 of the excavator 100 (S2104).

[0230] The first display device D3 of the excavator 100 displays a detected person or construction machine surrounded by a frame together with the captured image data according to the received coordinate area and frame size (S2105).

[0231] The controller 30 of the excavator 100 accepts an operation of designating an object surrounded by a frame in the first image display area 41n1 or the second image display area 41n2 via the touch panel of the input device D2 (step S2106).

[0232] When an operation of designating an object is received, the first display device D3 displays a pop-up window for confirming suppression of detection of the object (S2107). The pop-up window is the same as Figure 6 and will not be described again for brevity.

[0233] The controller 30 of the excavator 100 accepts a press of the OK button of the pop-up window via the touch panel of the input device D2 (S2108).

[0234] Moreover, when the controller 30 receives a press of the OK button, it sends the image data represented by the coordinate area and frame size of the object to the management server 2000 (S2109).

[0235] The management server 2000 registers the received image data in the object storage database D4A (S2110).

[0236] Through the above processing, when image data is registered in the object storage database D4A, a determination using the object storage database D4A is performed.

[0237] Figure 12 FIG. is a timing chart showing the processing sequence when making a determination using the object storage database D4A when displaying captured image data in the control system SYS according to the present embodiment.

[0238] First, the controller 30 of the excavator 100 acquires captured image data captured by the imaging device S6 (step S2201).

[0239] Then, the controller 30 sends the acquired captured image data to the management server 2000 via the communication device T1 (S2202).

[0240] The management server 2000 inputs the received captured image data into the learned model LM, thereby receiving the coordinate area and frame size of a person or construction machine existing around the excavator 100 (S2203).

[0241] The management server 2000 calculates the similarity (S2204) between a partial area determined based on the received coordinate area and frame size in the captured image data and the image data registered in the object storage database D4A.

[0242] When the management server 2000 determines that the similarity is lower than a specified threshold, it sends the received coordinate area and frame size to the communication device T1 of the excavator 100 (S2205).

[0243] On the other hand, when the management server 2000 determines that the similarity is equal to or higher than the specified threshold, it suppresses sending the received coordinate area and frame size to the communication device T1 of the excavator 100 (S2206).

[0244] In the present embodiment, when there are multiple received coordinate areas and frame sizes from the learned model LM, the management server 2000 repeats the processes of S2204 to S2206 the same number of times as the number of the coordinate areas and frame sizes.

[0245] Moreover, the first display device D3 of the excavator 100 displays a person or construction machinery surrounded by a frame together with the captured image data based on the received coordinate area and frame size (S2207).

[0246] In the present embodiment, the excavator 100 included in the control system SYS may be one or multiple. Thus, when the image data for suppressing false detection is registered by the operation of one excavator 100 in the control system SYS, false detection is also suppressed in other excavators 100. That is, in other excavators 100, since false detection can be suppressed without operation, improvement in detection accuracy and improvement in work efficiency can be achieved.

[0247] <Function>

[0248] In the above embodiment, the controllers 30, 30A, the first display device D3, or the management server 2000 suppress false detection of an object through the above control, thereby enabling improvement in detection accuracy. And since there is no need to set a new sensor to suppress false detection, high cost can be suppressed.

[0249] Moreover, in the above embodiment, since the display of the frame based on false detection is suppressed, the trouble when the operator refers to the display screen can be suppressed, and thus improvement in convenience can be achieved.

[0250] The above has described the embodiments of the excavator and the control system of the excavator according to the present invention. However, the present invention is not limited to the above embodiments and the like. Within the scope described in the technical solution, various changes, modifications, substitutions, additions, deletions, and combinations can be made. Of course, all of these are within the technical scope of the present invention.

Claims

1. An excavator comprising: Lower walking body; An upper rotating body, mounted on the lower walking body so as to be rotatable; A camera device, mounted on the upper rotating body; A display device for displaying the camera image data captured by the camera device; and The control device is configured to, when receiving an operation for designating an object represented by the captured image data, set information representing the object to suppress detection of the object.

2. The excavator according to claim 1, wherein: The setting for suppressing detection of the object is performed based on the operation received while the shovel is working.

3. The excavator according to claim 1, wherein: The display device displays display information indicating the detected object in the captured image data, The control device sets the information indicating the object to suppress detection of the object when receiving an operation to designate the object indicated by the display information.

4. The excavator according to claim 3, wherein: The display device suppresses display of the display information indicating the object when detecting the object indicated by the information for which detection suppression is set.

5. The excavator according to claim 4, wherein: The control device sets the image data showing the object to suppress detection of the object, When a partial area of ​​the captured image data captured by the imaging device is similar to the set image data by a predetermined threshold or more, display of the display information indicating the object is suppressed.

6. The excavator according to claim 4, wherein: It also includes a space recognition device installed on the upper rotating body, The display device displays the display information indicating the object detected by the space recognition device in the captured image data, The control device sets the detection data indicating the result of the object being detected by the space recognition device to suppress detection of the object, The control device suppresses display of the display information indicating the object when the detection result based on the space recognition device is similar to the set detection data by a predetermined threshold or more.

7. The excavator according to claim 1, wherein: further comprising a touch panel capable of accepting an operation indicating arbitrary position coordinates of the captured image data displayed on the display device, The control device receives an operation of designating the object represented in the captured image data via the touch panel.

8. A control system for an excavator, comprising: An excavator comprising a lower traveling body, an upper rotating body rotatably mounted on the lower traveling body, and a camera device mounted on the upper rotating body; A display device for displaying the camera image captured by the camera device; and The control device is configured to, when receiving an operation for designating an object represented in the captured image, set information representing the object to suppress detection of the object.

Images