Construction machine and control system for construction machine

Abstract

The invention provides a construction machine and a control system of the construction machine, which can easily communicate with surrounding people. A construction machine according to one embodiment of the present invention is provided with: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; a sound collection device which is provided on the upper rotating body and is provided so as to be able to detect the direction of sound around the construction machine; and a control device configured so as to notify a speech direction in accordance with the direction of the sound detected by the sound collection device when a person present in the periphery of the construction machine speaks.

Description

[0001] This application claims the benefit of priority based on Japanese Patent Application No. 2024-037494, filed on March 11, 2024. The entire contents of this Japanese patent application are incorporated herein by reference. Technical Field

[0002] The present invention relates to a construction machine and a control system of the construction machine. Background Art

[0003] Conventionally, workers usually work around the excavator. Therefore, a technique is known for detecting people around the excavator based on images captured by a camera mounted on the upper rotating body (for example, see Patent Document 1).

[0004] Patent Document 1: Japanese Patent Application Laid-Open No. 2023-63993

[0005] Incidentally, when working with construction machinery, people nearby may accost them. Voices detected by a microphone installed on the construction machinery are output through a speaker to the operator. This allows the operator to understand the content of the conversation. However, the operator does not know the direction of the speaker, and therefore may be unsure how to respond. Summary of the Invention

[0006] One embodiment of the present invention makes it easy to identify the speaker and communicate with the speaker by notifying the speaker's direction.

[0007] A construction machine involved in one embodiment of the present invention comprises: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; a sound collecting device arranged on the upper rotating body and arranged in a manner capable of detecting the direction of sounds around the construction machine; and a control device configured to notify the direction of speech based on the direction of the sound detected by the sound collecting device when a person existing around the construction machine speaks.

[0008] Effects of the Invention

[0009] According to one aspect of the present invention, communication with a speaker is facilitated. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a side view of the construction machine according to the first embodiment.

[0011] Figure 2 It is a plan view of the construction machine according to the first embodiment.

[0012] Figure 3 Is installed on Figure 1 A diagram showing a structural example of an external sound collection device and an information transmission device of a construction machine.

[0013] Figure 4 This is a diagram schematically showing an example of the structure of the construction machine according to the first embodiment.

[0014] Figure 5 It is a plan view of the interior of the cab according to the first embodiment.

[0015] Figure 6 This is a conceptual diagram showing a two-way conversation between an operator riding on the construction machine according to the first embodiment and workers present around the construction machine.

[0016] Figure 7 This is a diagram showing an example of a display screen displayed by the display device according to the first embodiment.

[0017] Figure 8 It is a diagram for explaining an example of correction of the utterance direction in the output control unit according to the first embodiment.

[0018] Figure 9 This is a diagram showing an example of a display screen displayed by the display device according to the first embodiment.

[0019] Figure 10 This is a diagram showing an example of a display screen displayed by the display device according to the first embodiment.

[0020] Figure 11 This is a flowchart showing a processing procedure for displaying an utterance direction by the controller according to the first embodiment.

[0021] Figure 12 It is a plan view of another structural example of the construction machine according to the second embodiment.

[0022] Figure 13 It is a schematic diagram showing a configuration example of a remote support system for a construction machine according to a third embodiment.

[0023] In the figure: 100-construction machinery, 1-lower walking body, 2-rotating mechanism, 3-upper rotating body, 4-boom, 5-arm, 6-bucket, S6-camera device, G1-information transmission device, M1-external sound collection device, M2, M2C, M2E-internal sound collection device, SP1-external sound output device, SP2, SP2C, SP2E-internal sound output device, SW-switch, KS-speak button, DL1-external volume control panel, DL2-internal volume control panel, T1, T2-communication device, 30-controller, 301-acquisition unit, 302-detection unit, 303-position inference unit, 304-sound processing unit, 305-determination unit, 306-judgment unit, 307-output control unit, RC-remote operation room, R40-remote controller, 200-management device. DETAILED DESCRIPTION

[0024] The following describes embodiments of the present invention with reference to the accompanying drawings. The embodiments described below are illustrative rather than limiting, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention. In addition, in the drawings, identical or corresponding components may be denoted by identical or corresponding reference numerals, and their description may be omitted.

[0025] The construction machine 100 according to the embodiment of the present invention is a shovel. The construction machine 100 may be a machine other than a shovel, such as a crane, asphalt roller, or forklift. While the shovel in the illustrated example is an excavator equipped with a bucket 6 as an end attachment, the construction machine 100 may also be an application machine such as a forestry machine equipped with an end attachment other than the bucket 6.

[0026] Hereinafter, the construction machine 100 is an excavator equipped with a bucket 6 as an attachment, but the present invention is not limited to an excavator. The construction machine 100 may be an application machine such as a forestry machine equipped with an attachment other than the bucket 6 .

[0027] (First embodiment)

[0028] First, refer to Figure 1 and Figure 2 , an overview of the construction machine 100 is described. Figure 1 is a side view of the construction machine 100. Figure 2 It is a top view of the construction machine 100 .

[0029] The construction machine 100 includes a lower traveling unit 1; an upper revolving unit 3 rotatably mounted on the lower traveling unit 1 via a revolving mechanism 2; an attachment AT for performing various operations; and a cab 10. When viewing the construction machine 100 from directly above (viewed from above) along the revolving axis of the upper revolving unit 3, the front of the construction machine 100 (the upper revolving unit 3) corresponds to the direction in which the attachment AT extends relative to the upper revolving unit 3. Furthermore, the left and right directions of the construction machine 100 (the upper revolving unit 3) correspond to the left and right directions, respectively, as viewed from the operator's seat in the cab 10.

[0030] The lower traveling body 1 includes, for example, a pair of left and right crawler tracks 1C. Specifically, the crawler tracks 1C include a left crawler track 1CL and a right crawler track 1CR. The left crawler track 1CL is driven by a left traveling hydraulic motor 2ML, and the right crawler track 1CR is driven by a right traveling hydraulic motor 2MR. The left traveling hydraulic motor 2ML is a traveling drive unit that drives the left crawler track 1CL, which is a driven unit, and is capable of rotating the left crawler track 1CL. The right traveling hydraulic motor 2MR is a traveling drive unit that drives the right crawler track 1CR, which is a driven unit, and is capable of rotating the right crawler track 1CR. Alternatively, the traveling drive unit may be an electric motor.

[0031] The upper slewing body 3 is swiveled relative to the lower traveling body 1 by driving the slewing mechanism 2 via the slewing hydraulic motor 2A. The slewing hydraulic motor 2A is a slewing drive unit that drives the upper slewing body 3, which is the driven unit, and can change the orientation of the upper slewing body 3. Alternatively, the slewing drive unit may be an electric motor.

[0032] A boom 4 is rotatably mounted at the front center of the upper slewing body 3, an arm 5 is rotatably mounted at the front end of the boom 4, and a bucket 6 is rotatably mounted at the front end of the arm 5. In the illustrated example, the boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of an attachment AT. The boom 4, arm 5, and bucket 6 are driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.

[0033] Bucket 6 is an example of a work tool (end attachment). Bucket 6 is used, for example, for excavation operations. Other work tools can be attached to the tip of arm 5 in place of bucket 6, depending on the type of work being done. Examples of these other work tools include large buckets, slope buckets, dredging buckets, and other types of buckets. Furthermore, these other work tools can include mixers, crushers, grapple hooks, lifting magnets, and other types of work tools other than buckets.

[0034] The swing hydraulic motor 2A, the left travel hydraulic motor 2ML, the right travel hydraulic motor 2MR, the boom cylinder 7 , the arm cylinder 8 , and the bucket cylinder 9 are hydraulic actuators driven by hydraulic oil discharged from a hydraulic pump.

[0035] Furthermore, in the construction machine 100, all or part of the driven parts, such as the lower traveling body 1, upper swing body 3, boom 4, arm 5, and bucket 6, may be electrically driven. In other words, the construction machine 100 may be a hybrid excavator or an electric excavator, for example, in which all or part of the driven parts are driven by electric actuators.

[0036] Furthermore, the construction machine 100 is equipped with an information transmission device G1 , an external sound collecting device M1 , an imaging device S6 , and an external sound output device SP1 .

[0037] The camera S6 is installed on the upper swing body 3 or the cab 10 to capture images of the surroundings of the construction machine 100 and obtain image information representing the surroundings of the construction machine 100. In the illustrated example, the camera S6 includes a front camera S6F, a left camera S6L, a right camera S6R, and a rear camera S6B.

[0038] The front camera S6F is a camera that captures the front of the construction machine 100 and is mounted on the exterior of the cab 10, such as the ceiling of the cab 10 or the side of the boom 4. Alternatively, the front camera S6F may be mounted on the roof of the cab 10, i.e., inside the cab 10. The left camera S6L captures the left side of the construction machine 100, the right camera S6R captures the right side of the construction machine 100, and the rear camera S6B captures the rear side of the construction machine 100. Specifically, the front camera S6F, the left camera S6L, the right camera S6R, and the rear camera S6B are all monocular wide-angle cameras equipped with imaging elements such as CCDs or CMOS sensors, and they output captured images to the display device D1. Furthermore, information about the images captured by the camera device S6 is input to the controller 30.

[0039] In the example shown in the figure, the front camera S6F is installed on the ceiling of the cab 10, the left camera S6L is installed on the left end of the upper surface of the upper rotating body 3, the right camera S6R is installed on the right end of the upper surface of the upper rotating body 3, and the rear camera S6B is installed on the rear end of the upper surface of the upper rotating body 3.

[0040] The camera S6 can constitute an object detection device that detects objects located around the construction machine 100. The object detection device can be composed of devices other than cameras. For example, the object detection device can be a LiDAR. LiDAR is, for example, a device that can measure the distance between a group of more than 1 million points within the monitoring range and the LiDAR (laser source). In addition, the object detection device can also be other devices that can measure the distance to the object, such as a stereo camera, a range image camera, or a millimeter wave radar. In the case of using a millimeter wave radar or the like as the object detection device, multiple signals (laser light, etc.) can be sent from the object detection device to the object, and the reflected signals can be received, thereby deriving the distance and direction of the object. Alternatively, the object detection device can also be a combination of two or more devices. For example, the object detection device can be a combination of a camera and a LiDAR, a combination of a camera and a millimeter wave radar, or a combination of a camera and a stereo camera.

[0041] The external sound collection device M1 is a device that collects external sounds, also known as a microphone. In the illustrated example, the external sound collection device M1 is installed on the upper rotating body 3 or the cab 10. It converts the sounds (air vibrations) generated around the construction machine 100 into mechanical vibrations, and then converts these mechanical vibrations into electrical signals. Specifically, the external sound collection device M1 includes a front microphone M1F, a left microphone M1L, a right microphone M1R, and a rear microphone M1B.

[0042] The front microphone M1F collects sounds generated in front of the construction machine 100 and is mounted on the exterior of the cab 10, such as the ceiling of the cab 10 or the side of the boom 4. Alternatively, the front microphone M1F may be mounted on the roof of the cab 10, i.e., inside the cab 10. The left microphone M1L collects sounds generated to the left of the construction machine 100, the right microphone M1R collects sounds generated to the right of the construction machine 100, and the rear microphone M1B collects sounds generated to the rear of the construction machine 100. The electrical signals generated by the front microphone M1F, the left microphone M1L, the right microphone M1R, and the rear microphone M1B are input to the controller 30.

[0043] In the example shown in the figure, the front microphone M1F is installed on the ceiling of the cab 10, the left microphone M1L is installed on the left end of the upper surface of the upper rotating body 3, the right microphone M1R is installed on the right end of the upper surface of the upper rotating body 3, and the rear microphone M1B is installed on the rear end of the upper surface of the upper rotating body 3. In this way, the four external sound collecting devices M1 (front microphone M1F, left microphone M1L, right microphone M1R and rear microphone M1B) are arranged at different positions of the upper rotating body 3. Therefore, the controller 30 can detect the direction of the sound source based on the difference in the sounds collected by the four external sound collecting devices M1 (for example, the difference in volume). In addition, when an array microphone is used as the external sound collecting device M1, for example, the direction of the sound source can be detected based on the phase shift or the difference in volume.

[0044] In the illustrated example, each of the four external sound collection devices M1 is positioned to correspond to each of the four camera devices S6. Specifically, the front microphone M1F is positioned adjacent to the front camera S6F, the left microphone M1L is positioned adjacent to the left camera S6L, the right microphone M1R is positioned adjacent to the right camera S6R, and the rear microphone M1B is positioned adjacent to the rear camera S6B.

[0045] The external sound output device SP1 is a device that outputs sound to the surroundings of the construction machine 100. In the illustrated example, the external sound output device SP1 is an omnidirectional speaker configured to output sound uniformly in all directions. However, the external sound output device SP1 may also be a directional speaker that outputs sound forward.

[0046] The information transmission device G1 is used to notify the outside of the construction machine 100 of the status of the construction machine 100. In the illustrated example, the information transmission device G1 is installed on the upper swing structure 3 or the cab 10 and is configured to convey the status of the construction machine 100 to workers located around the construction machine 100. Specifically, the information transmission device G1 includes a front light bar G1F, a left light bar G1L, a right light bar G1R, and a rear light bar G1B.

[0047] The front light bar G1F is a light-emitting device that visually conveys information to a worker, etc., located in front of the construction machine 100. It is mounted on the exterior of the cab 10, such as the ceiling of the cab 10 or the side of the boom 4. Alternatively, the front light bar G1F can be mounted on the roof of the cab 10, i.e., inside the cab 10. The left light bar G1L is a light-emitting device that visually conveys information to a worker, etc., located to the left of the construction machine 100. The right light bar G1R is a light-emitting device that visually conveys information to a worker, etc., located to the right of the construction machine 100. The rear light bar G1B is a light-emitting device that visually conveys information to a worker, etc., located behind the construction machine 100. The front light bar G1F, left light bar G1L, right light bar G1R, and rear light bar G1B each emit light in response to an electrical signal from the controller 30. In the illustrated example, the light-emitting devices are LEDs, but other light-emitting devices such as halogen lamps may also be used. Furthermore, the light emitting device is a multi-color light emitting type, but may also be a single-color light emitting type.

[0048] In the illustrated example, the front light bar G1F is mounted on the ceiling of the cab 10, the left light bar G1L is mounted on the left end of the upper surface of the upper revolving body 3, the right light bar G1R is mounted on the right end of the upper surface of the upper revolving body 3, and the rear light bar G1B is mounted on the rear end of the upper surface of the upper revolving body 3. Thus, the four information transmission devices G1 (front light bar G1F, left light bar G1L, right light bar G1R, and rear light bar G1B) are installed at different locations on the upper revolving body 3. Therefore, by activating each of the four information transmission devices G1, the controller 30 can communicate the current status of the construction machine 100 to workers, etc., located in front, to the left, to the right, and to the rear of the construction machine 100.

[0049] In the illustrated example, each of the four information transmission devices G1 is positioned corresponding to each of the four external sound collection devices M1. Specifically, the front light bar G1F is positioned adjacent to the front microphone M1F, the left light bar G1L is positioned adjacent to the left microphone M1L, the right light bar G1R is positioned adjacent to the right microphone M1R, and the rear light bar G1B is positioned adjacent to the rear microphone M1B.

[0050] Figure 3 1 is a diagram showing a configuration example of an external sound collecting device M1 and an information transmitting device G1 mounted on a construction machine 100. Specifically, Figure 3 This is a perspective view of the left microphone M1L and the left light bar G1L mounted on a substantially rectangular parallelepiped frame. Figure 3 The following description relates to the combination of left microphone M1L and left light bar G1L, but also applies to the combination of front microphone M1F and front light bar G1F, the combination of right microphone M1R and right light bar G1R, and the combination of rear microphone M1B and rear light bar G1B.

[0051] like Figure 3 As shown, the left microphone M1L and the left light bar G1L are positioned on the left side of the roughly rectangular parallelepiped housing, facing the left side of the construction machine 100. This arrangement allows the left microphone M1L to effectively collect sounds generated to the left of the construction machine 100, and the left light bar G1L to effectively communicate the status of the construction machine 100 to a worker located to the left of the construction machine 100. For example, the left microphone M1L can capture the voice of a worker located to the left of the construction machine 100, and the left light bar G1L can illuminate a predetermined color to notify the worker that the left microphone M1L has captured the worker's voice. In this case, a worker located to the left of the construction machine 100 who speaks to the left microphone M1L can confirm that their voice has reached the left microphone M1L (i.e., the operator of the construction machine 100) by observing the left light bar G1L illuminated in the predetermined color.

[0052] The information transmission device G1 can be installed on the upper portion of each of the four sides of the cab 10. For example, the information transmission device G1 can be configured such that a front light bar G1F is installed on the upper portion of the front surface of the cab 10, a left light bar G1L is installed on the upper portion of the left surface of the cab 10, a right light bar G1R is installed on the upper portion of the right surface of the cab 10, and a rear light bar G1B is installed on the upper portion of the rear surface of the cab 10. Furthermore, the information transmission device G1 can be a rotating light such as a NICO TORCH installed on the upper surface of the cab 10, or a display device such as a liquid crystal display or an organic EL display.

[0053] The controller 30 is an example of a control device, and is composed of, for example, a computer including a CPU, a volatile storage device, a non-volatile storage device, and various input and output interfaces. Furthermore, the controller 30 reads a program from the non-volatile storage device and loads it into the volatile storage device, which is executed by the CPU, thereby realizing various functions. In the illustrated example, the controller 30 is configured to realize various functions and control the construction machine 100. The various functions include, for example, an equipment guidance function that guides (instructs) the operator's manual operation of the construction machine 100. The various functions may include a contact avoidance function that causes the construction machine 100 to automatically or autonomously move or stop the construction machine 100 to avoid contact between the construction machine 100 and objects existing within the monitoring range around the excavator 100.

[0054] The boom angle sensor S1 detects the boom angle, which is the rotation angle of the boom 4 relative to the upper swing body 3. The arm angle sensor S2 detects the arm angle, which is the rotation angle of the arm 5 relative to the boom 4. The bucket angle sensor S3 detects the bucket angle, which is the rotation angle of the bucket 6 relative to the arm 5.

[0055] The boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 may be, for example, a rotary encoder, an acceleration sensor, a six-axis sensor, an IMU (Inertial Measurement Unit), or a potentiometer using a variable resistor, a cylinder stroke sensor for detecting the stroke of a hydraulic cylinder, or the like.

[0056] A detection signal corresponding to the boom angle from the boom angle sensor S1 , a detection signal corresponding to the arm angle from the arm angle sensor S2 , and a detection signal corresponding to the bucket angle from the bucket angle sensor S3 are input to the controller 30 .

[0057] The body tilt sensor S4 detects the tilt of the body (the upper rotating body 3 or the lower traveling body 1) relative to the horizontal plane. The body tilt sensor S4 is mounted, for example, on the upper rotating body 3 and detects the tilt angle of the construction machine 100 (i.e., the upper rotating body 3) about two axes: the fore-aft and the lateral. The body tilt sensor S4 can be, for example, an accelerometer, a six-axis sensor, or an IMU. A detection signal corresponding to the tilt angle generated by the body tilt sensor S4 is input into the controller 30.

[0058] The rotation sensor S5 outputs information related to the rotation of the upper rotating body 3. For example, the rotation sensor S5 detects the angular velocity of the upper rotating body 3 relative to the lower traveling body 1. The rotation sensor S5 can also detect the rotation angle. For example, the rotation sensor S5 can be a gyroscope, a resolver, or a rotary encoder. Detection signals from the rotation sensor S5, corresponding to the rotation angle and angular velocity of the upper rotating body 3, are input to the controller 30.

[0059] The positioning device PS measures the position of the upper rotating body 3. The positioning device PS, for example, a GNSS (Global Navigation Satellite System) compass, detects the position and orientation of the upper rotating body 3. Detection signals corresponding to the position and orientation of the upper rotating body 3 are input to the controller 30. The function of detecting the orientation of the upper rotating body 3 can be achieved by an orientation sensor attached to the upper rotating body 3.

[0060] The cab 10 is a cab for an operator to ride in, and is mounted on the front left side of the upper swing body 3. However, the cab 10 may be omitted when the construction machine 100 is remotely operated or when the construction machine 100 is operated in a fully automatic manner.

[0061] The communication device T1 communicates with external devices via a communication network, such as a mobile communication network, a satellite communication network, or the Internet. For example, the communication device T1 may be a mobile communication module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation), or a satellite communication module for connecting to a satellite communication network.

[0062] The construction machine 100 operates actuators in response to operations by an operator riding in the cab 10 , thereby driving driven parts such as the lower traveling body 1 , the upper swing body 3 , the boom 4 , the arm 5 , and the bucket 6 .

[0063] Alternatively, the construction machine 100 may be configured to be remotely operated from outside the construction machine 100. When the construction machine 100 is remotely operated, the interior of the cab 10 may be unmanned.

[0064] Furthermore, the construction machine 100 can also automatically operate the actuators independently of the operator's operation. Thus, the construction machine 100 realizes the function of automatically operating at least a portion of the driven parts, such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, and the bucket 6, which is the so-called "equipment control function."

[0065] Figure 4 1 is a diagram schematically showing an example of the structure of the construction machine 100. Figure 4 In the figure, the mechanical power transmission system, the working oil pipeline, the pilot pipeline, and the electrical control system are represented by double lines, thick solid lines, thick dashed lines, and dotted lines, respectively.

[0066] The drive system of the construction machine 100 includes an engine 11, a regulator 13, a main pump 14, and a control valve unit 17. Furthermore, the hydraulic drive system of the construction machine 100 includes hydraulic actuators such as a swing hydraulic motor 2A, a left travel hydraulic motor 2ML, a right travel hydraulic motor 2MR, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9.

[0067] The engine 11 is an example of a power source for the construction machine 100 and is mounted, for example, on the rear portion of the upper rotating body 3. Alternatively, the power source for the construction machine 100 may be a combination of a power source such as a battery or fuel cell and an electric motor. Specifically, the engine 11 rotates constantly at a predetermined target speed under direct or indirect control by the controller 30 and drives the main pump 14 and the pilot pump 15. For example, the engine 11 is a diesel engine that uses diesel as fuel. Alternatively, the engine 11 may be a gasoline engine or a hydrogen engine.

[0068] The regulator 13 controls the discharge volume of the main pump 14 . For example, the regulator 13 controls the discharge volume of the main pump 14 by adjusting the angle (deflection angle) of the swash plate of the main pump 14 in accordance with a control command from the controller 30 .

[0069] The main pump 14 is mounted, for example, at the rear of the upper slewing structure 3, similarly to the engine 11, and supplies hydraulic oil through a hydraulic oil line to the control valve unit 17. In the illustrated example, the main pump 14 is a variable displacement hydraulic pump.

[0070] The control valve unit 17 is a hydraulic control device that controls the hydraulic system in the construction machine 100. In the illustrated example, the control valve assembly 17 includes control valves 171 to 176. The control valve unit 17 is configured to selectively supply hydraulic fluid discharged from the main pump 14 to one or more hydraulic actuators via the control valves 171 to 176. The control valves 171 to 176 control the flow rate of hydraulic fluid from the main pump 14 to the hydraulic actuators and the flow rate of hydraulic fluid from the hydraulic actuators to the hydraulic fluid tank. The hydraulic actuators include the boom cylinder 7, the arm cylinder 8, the bucket cylinder 9, the left travel hydraulic motor 2ML, the right travel hydraulic motor 2MR, and the swing hydraulic motor 2A. Specifically, control valve 171 corresponds to the swing hydraulic motor 2A, control valve 172 corresponds to the right travel hydraulic motor 2MR, and control valve 173 corresponds to the left travel hydraulic motor 2ML. Furthermore, control valve 174 corresponds to the bucket cylinder 9, control valve 175 corresponds to the boom cylinder 7, and control valve 176 corresponds to the arm cylinder 8.

[0071] The pilot pump 15 is an example of a pilot pressure generating device, and is configured to supply hydraulic oil to the hydraulic control equipment via a pilot line. In the illustrated example, the pilot pump 15 is a fixed-displacement hydraulic pump. However, the pilot pressure generating device can be implemented by the main pump 14. That is, in addition to supplying hydraulic oil to the control valve unit 17 via the hydraulic oil line, the main pump 14 can also supply hydraulic oil to various hydraulic control equipment via the pilot line. In this case, the pilot pump 15 can also be omitted.

[0072] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In the example shown in the figure, the discharge pressure sensor 28 outputs the detected value to the controller 30.

[0073] The operating device 26 is a device used by the operator to operate the actuator. The operating device 26 includes, for example, an operating lever and an operating pedal. The actuator can be a hydraulic actuator or an electric actuator.

[0074] The operation sensor 29 is configured to detect the content of an operation performed by an operator using the operating device 26. In this embodiment, the operation sensor 29 detects the operation direction and amount of the operating device 26 corresponding to each actuator and outputs the detected values ​​to the controller 30. In the illustrated example, the controller 30 is capable of controlling the opening area of ​​the proportional valve 31 based on the output of the operation sensor 29. Furthermore, the controller 30 supplies the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve within the control valve unit 17. In principle, the pressure of the hydraulic oil supplied to each pilot port (pilot pressure) is a pressure corresponding to the operation direction and amount of the operating device 26 corresponding to each hydraulic actuator. In this way, the operating device 26 is configured to supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve within the control valve unit 17.

[0075] A proportional valve 31, functioning as a control valve for controlling the equipment, is disposed in the conduit connecting the pilot pump 15 and the pilot port of the control valve within the control valve unit 17. The proportional valve 31 is configured to be variable in its flow path area. In the illustrated example, the proportional valve 31 operates in response to a control command output by the controller 30. Therefore, the controller 30 can adjust the pilot pressure acting on the pilot port of the control valve via the proportional valve 31, regardless of the operator's operation of the operating device 26.

[0076] With this configuration, even when a specific operating device 26 is not being operated, the controller 30 can operate the hydraulic actuator corresponding to the specific operating device 26 .

[0077] The control system of the construction machine 100 includes a controller 30, a display device D1, an input device D2, a speech button KS, an external sound collection device (an example of a sound collection device) M1, an internal sound collection device M2, an external sound output device SP1, an internal sound output device SP2, an external volume control panel DL1, an internal volume control panel DL2, a switch SW and a communication device T1, etc.

[0078] The controller 30 is configured to output a control command to the regulator 13 as needed to change the discharge rate of the main pump 14 .

[0079] Furthermore, the controller 30 may be configured to, for example, perform control related to a device guidance function that guides (instructs) manual operations performed by an operator on the construction machine 100 via the operating device 26. Furthermore, the controller 30 may be configured to, for example, perform control related to a device control function that automatically supports manual operations performed by an operator on the construction machine 100 via the operating device 26.

[0080] Furthermore, some of the functions of controller 30 may be implemented by other controllers (control devices). In other words, the functions of controller 30 may be implemented in a distributed manner by multiple controllers. For example, the device guidance function and the device control function may be implemented by a dedicated controller (control device).

[0081] Here, reference Figure 5 , the interior of the cab 10 will be described. Figure 5 This is a top view of the interior of the cab 10. The construction machine 100 includes a driver's seat 50, an operating device 26, a display device D1, and the like, which are disposed within the cab 10. A door for boarding and disembarking is provided on the left side of the driver's seat 50. The operator can open the door to enter the cab 10.

[0082] When viewed from above, the driver's seat 50 is located in the center of the cab 10. The driver's seat 50 includes a seat portion 51 for the operator and a backrest 52. The driver's seat 50 is a reclining chair, and the inclination angle of the backrest 52 is adjustable. A left armrest 53L is located on the left side of the driver's seat 50, and a right armrest 53R is located on the right side. The left armrest 53L and the right armrest 53R are rotatably supported by the backrest 52.

[0083] A left console 54L is disposed on the left side of the driver's seat 50, and a right console 54R is disposed on the right side. The left console 54L and the right console 54R extend in the front-to-rear direction. The driver's seat 50 is slidable in the front-to-rear direction. The driver's seat 50 can be configured to slide in the front-to-rear direction together with the left console 54L and the right console 54R.

[0084] The left armrest 53L is arranged on the left console 54L. The right armrest 53R is arranged on the right console 54R. When viewed from above, the left armrest 53L is arranged to cover a portion of the left console 54L. When viewed from above, the right armrest 53R is arranged to cover a portion of the right console 54R.

[0085] The operating device 26 includes a left operating lever 26L, a right operating lever 26R, a left travel pedal 26PL, a right travel pedal 26PR, a left travel lever 26DL, and a right travel lever 26DR.

[0086] The left operating lever 26L is provided at the front of the left console 54L. Similarly, the right operating lever 26R is provided at the front of the right console 54R. The operator sitting on the driver's seat 50 can operate the left operating lever 26L while holding the left operating lever 26L with his left hand, and can operate the right operating lever 26R while holding the right operating lever 26R with his right hand. The operator sitting on the driver's seat 50 can drive the boom cylinder 8 and the rotary hydraulic motor 2A by operating the left operating lever 26L with his left hand. Furthermore, the operator sitting on the driver's seat 50 can drive the boom cylinder 7 and the bucket cylinder 9 by operating the right operating lever 26R with his right hand. In addition, the base of each of the left operating lever 26L and the right operating lever 26R is covered by a rod cover 27.

[0087] The left and right travel pedals 26PL and 26PR are located on the floor in front of the operator's seat 50. An operator seated in the operator's seat 50 can operate the left travel pedal 26PL with their left foot to drive the left travel hydraulic motor 2ML. Furthermore, an operator seated in the operator's seat 50 can operate the right travel pedal 26PR with their right foot to drive the right travel hydraulic motor 2MR.

[0088] When viewed from above, the left travel rod 26DL and the right travel rod 26DR are arranged between the left travel pedal 26PL and the right travel pedal 26PR. The left travel rod 26DL and the right travel rod 26DR extend upward from the floor surface in front of the driver's seat 50. The operator sitting in the driver's seat 50 can operate the left travel rod 26DL with his left hand while holding it, thereby driving the left travel hydraulic motor 2ML in the same way as operating via the left travel pedal 26PL. In addition, the operator sitting in the driver's seat 50 can operate the right travel rod 26DR with his right hand while holding it, thereby driving the right travel hydraulic motor 2MR in the same way as operating via the right travel pedal 26PR. In addition, the left travel rod 26DL and the right travel rod 26DR are arranged so that the operator can operate the left travel rod 26DL and the right travel rod 26DR simultaneously with one hand.

[0089] The display device D1 is located in front of the right side of the operator's seat 50. The display device D1 displays various visual information. The display device D1 includes a display screen that displays information such as the operating conditions and operating status of the construction machine 100. The operator seated in the operator's seat 50 can perform work on the construction machine 100 while viewing the various information displayed on the display device D1. The display device D1 may also be provided with an input device D2.

[0090] Input device D2 is located within reach of the operator seated in cab 10, receives various operational inputs from the operator, and outputs signals corresponding to the operational inputs to controller 30. Input device D2 includes a touch panel mounted on the display of display device D1, which displays various information images; a rotary knob switch located at the tip of one or more of the multiple operating levers included in operating device 26; or a push button switch, joystick, selector switch, or rotary dial located around display device D1. Signals corresponding to the content of the operations performed on input device D2 are input to controller 30.

[0091] A latch 55 is attached to the front surface of the front end of the left console 54L. The latch 55 operates in conjunction with the operation of a door lock lever GL provided on the left console 54L. The latch 55 is attached to a frame inside the left console 54L so that it can rise and fall around the left-right axis of the upper end.

[0092] The door lock lever GL is a mechanical input operating portion for switching a state in which the construction machine 100 can be operated by the operating device 26 (operable state) and a state in which the construction machine 100 cannot be operated by the operating device 26 (inoperable state). In the illustrated example, the door lock lever GL is configured so that the operator can switch between a first operating position for achieving an inoperable state and a second operating position for achieving an operable state. The controller 30 switches between the operable state and the inoperable state according to the operating state of the door lock lever GL. In the illustrated example, the controller 30 switches between the connected and disconnected pilot lines according to the operating state of the door lock lever GL, thereby switching between the operable state and the inoperable state of the construction machine 100.

[0093] Furthermore, when the door lock lever GL is located at the second operating position, Figure 5 As shown, the latch 55 is in a forward-standing state (pass-through state) to prevent the operator from passing through the boarding and alighting door. On the other hand, when the door lock lever GL is in the first operating position, the latch 55 is housed inside the left console 54L (pass-through state) to not prevent the operator from passing through the boarding and alighting door.

[0094] This structure prevents the operator from operating the construction machine 100 unless the door lock lever GL is set to the second operating position and the door latch 55 is set to the no-pass state. Therefore, even if the operator accidentally touches the operating device 26 while getting on or off the construction machine, this structure prevents the construction machine 100 from unintentionally moving. Consequently, this structure improves the safety of the construction machine 100.

[0095] Furthermore, the construction machine 100 may be configured so that a predetermined operation for starting the engine 11 can be accepted only when the door lock lever GL is in the second operating position and the door latch 55 is in the passage-prohibiting state. In other words, the construction machine 100 may be configured so that the engine 11 cannot be started only when the door lock lever GL is in the first operating position and the door latch 55 is in the passage-permitting state.

[0096] A switch SW is provided on the right console 54R. A window console 56 is provided to the right of the right console 54R. The window console 56 extends along the entire length of the cab 10 in the front-to-rear direction and is arranged parallel to the right console 54R. A display device D1 is provided in front of the window console 56. The window console 56 also includes an external volume control dial DL1, an internal volume control dial DL2, an internal sound collection device M2, and the like.

[0097] Furthermore, a radio tuner 57 and the like are provided on the window side console 56. The radio tuner 57 and the like may also be provided on the console 54L or the console 54R.

[0098] The radio tuner 57 is provided with a switch. By pressing the switch and rotating the switch, the radio can be turned on and off and the volume can be adjusted. Moreover, the operation for controlling the radio tuner 57 is not limited to the operation of the physically provided switch, but can also be an operation using the touch panel (input device D2) provided on the display device D1. For example, the controller 30 can perform the on-off control and volume adjustment control of the radio tuner 57 by accepting the operation of the touch panel. The volume adjustment of the radio tuner 57 can be adjusted independently of the volume adjustment of the external sound (the sound output to the operator in the car), or can be adjusted integrally.

[0099] The interior sound collecting device M2 is a device for collecting sounds generated in the cab 10. In the illustrated example, the interior sound collecting device M2 is a room microphone and is configured to be able to pick up sounds made by an operator in the cab 10.

[0100] The horn button HS is a button operated by the operator of the construction machine 100 to sound the horn. In the illustrated example, the horn button HS is a rotary switch provided at the front end of the left operating lever 26L.

[0101] The talk button KS is a button operated by the operator of the construction machine 100 when the operator of the construction machine 100 talks to a worker located around the construction machine 100. In the illustrated example, the talk button KS is a rotary switch provided at the front end of the right operating lever 26R.

[0102] The internal sound output device SP2 is a device that outputs sound to the operator in the cab 10 and is installed in the cab 10. In addition, the internal sound output device SP2 converts the electrical signal input from the controller 30 into physical sound (air vibration) and outputs it. The internal sound output device SP2 can be set at any position, for example, it can be set near the display device D1, near the input device D2, or near the door of the cab 10. In the example shown in the figure, the internal sound output device SP2 includes: a left indoor speaker SP2L, which is installed in the upper left corner of the rear wall of the cab 10; and a right indoor speaker SP2R, which is installed in the upper right corner of the rear wall of the cab 10. In addition, the internal sound output device SP2 can be a headset or earphones worn by the operator. In this case, the headset or earphones are connected in a manner that can communicate with the controller 30 via Bluetooth (registered trademark) or the like.

[0103] The external volume control dial DL1 is configured to adjust the volume of the sound output by the external sound output device SP1. Furthermore, the volume of the sound output by each external sound output device SP1 can be adjusted using a device other than the external volume control dial DL1, such as a touch panel attached to the display device D1.

[0104] The external volume control dial DL1 may be configured to be infinitely rotatable in both clockwise and counterclockwise directions. This is because it allows for simultaneous use of volume adjustment using the external volume control dial DL1 and volume adjustment using a device other than the external volume control dial DL1.

[0105] The internal volume control dial DL2 is configured to adjust the volume of the sound output by the internal sound output device SP2. Furthermore, the volume of the sound output by each internal sound output device SP2 can be adjusted using a device other than the internal volume control dial DL2, such as a touch panel mounted on the display device D1.

[0106] The internal volume control dial DL2 may be configured to be infinitely rotatable in both clockwise and counterclockwise directions. This is because it allows for simultaneous use of volume adjustment using the internal volume control dial DL2 and volume adjustment using a device other than the internal volume control dial DL2.

[0107] The storage device 35 is, for example, located within the cab 10 and stores various information under the control of the controller 30. The storage device 35 is, for example, a non-volatile storage medium such as a semiconductor memory. The storage device 35 stores information for suppressing sound output from the internal sound output device SP2. The storage device 35 can, for example, store data related to a target construction surface, obtained via the communication device T1 or set via the input device D2. This target construction surface can be set (saved) by the operator of the construction machine 100 or by a construction manager or the like.

[0108] The switch SW switches whether or not the sound output function, which outputs the sound collected by the external sound collection device M1 from the internal sound output device SP2, is activated. In the illustrated example, the switch SW is located on the top surface of the right console 54R. However, the switch SW may be implemented as one of the input devices D2, a touch panel provided on the display device D1, or a rotary switch.

[0109] The dialogue function is used to implement Figure 6 The illustrated function is a communication function between the operator OP of the construction machine 100 and the workers WK located around the construction machine 100 . Figure 6 It is a perspective view of the construction machine 100 on which the operator OP is riding and the worker WK positioned in front of the left side of the construction machine 100 . Figure 6 This indicates a state where the operator OP's voice is collected by the internal sound collecting device M2 and output from the external sound output device SP1, and the worker WK's voice is collected by the external sound collecting device M1 and output from the internal sound output device SP2. Figure 6 In the construction machine 100 shown, the information transmission device G1 is provided on the upper portion of each of the four side surfaces of the cab 10. Furthermore, the front light bar G1F provided on the upper portion of the front surface of the cab 10 emits green light, and the left light bar G1L provided on the upper portion of the left surface of the cab 10 emits white light. Figure 6 In FIG, a dot pattern is added to the front light bar G1F that emits green light. The worker WK who observes the front light bar G1F that emits green light can recognize that his voice is detected by the front microphone M1F. Figure 6 For the sake of clarity, illustrations of other devices such as the camera S6 are omitted.

[0110] The operation state of the dialogue function includes a state in which the operator OP and the worker WK can communicate with each other, that is, a conduction state ( Figure 6) and a disconnected state in which the operator OP and the worker WK cannot communicate. However, the operational state of the dialogue function may additionally include at least one of a state in which the operator OP can hear the voice of the worker WK but the worker WK cannot hear the voice of the operator OP, namely, a audible state (related to the operator OP), and a state in which the worker WK can hear the voice of the operator OP but the operator OP cannot hear the voice of the worker WK, namely, a speech-enabled state (related to the operator OP).

[0111] Specifically, if the switch SW is operated to switch the operation state of the dialogue function to the on state, the external sound collection device M1, the external sound output device SP1, the internal sound collection device M2, and the internal sound output device SP2 become available. On the contrary, if the switch SW is operated to switch the operation state of the dialogue function to the off state, the external sound collection device M1, the external sound output device SP1, the internal sound collection device M2, and the internal sound output device SP2 become unavailable. Furthermore, if the switch SW is operated to switch the operation state of the dialogue function to the audible state, the external sound collection device M1 and the internal sound output device SP2 become available. Furthermore, if the switch SW is operated to switch the operation state of the dialogue function to the speaking state, the external sound output device SP1 and the internal sound collection device M2 become available. In the illustrated example, when the internal sound collection device M2 is in the available state, the operator OP speaks while pressing the speak button KS, thereby being able to talk to the operator WK using the external sound output device SP1.

[0112] Furthermore, under the control of the controller 30 of this embodiment, the internal sound output device SP2 located inside the cab 10 outputs the sound collected by the external sound collection device M1, while the external sound output device SP1 located outside the cab 10 outputs the sound collected by the internal sound collection device M2. Furthermore, the controller 30 can switch between sound output from the internal sound output device SP2 and sound output from the external sound output device SP1 in response to an operation by the operator OP, thereby enabling two-way communication. The controller 30 of this embodiment switches sound output based on the operation, thereby preventing simultaneous sound output from both the internal sound output device SP2 and the external sound output device SP1, thereby suppressing the occurrence of whistling, etc. Furthermore, in this embodiment, the switching control method is not limited; sound can also be output simultaneously from both the external sound output device SP1 and the internal sound output device SP2.

[0113] When the controller 30 simultaneously outputs the sound collected by the external sound collecting device M1 from the internal sound output device SP2 provided on the inner side of the cab 10 and outputs the sound collected by the internal sound collecting device M2 from the external sound output device SP1 provided on the outer side of the cab 10, the sound collecting devices M2 and M1 collect the sound output from the sound output devices SP1 and SP2, thereby possibly generating an echo (reverberant sound) such as an "echo". Therefore, the controller 30 can perform an echo cancellation function on the sound represented by the sound signal obtained from the external sound collecting device M1 or the internal sound collecting device M2. The echo cancellation function only needs to be a function that removes the echo. For example, an echo suppressor method that does not pick up the voice of another person while hearing the voice of one person can be used, or an echo canceller method that removes the echo (reverberant sound) collected by the sound collecting devices M2 and M1 can be used. Moreover, with regard to the echo cancellation function, regardless of the known methods mentioned above, all methods can be used.

[0114] [Functional structure of the controller]

[0115] The controller 30 is configured to perform control based on a detected person. The controller 30 includes an acquisition unit 301, a detection unit 302, a position estimation unit 303, a sound processing unit 304, a determination unit 305, a judgment unit 306, and an output control unit 307, as well as functions for device guidance and device control.

[0116] The acquisition unit 301 acquires detection results from various sensors installed on the construction machine 100. For example, the acquisition unit 301 acquires image information representing the imaging results from the imaging device S6. Furthermore, the acquisition unit 301 acquires sound signals representing sounds generated around the construction machine 100 from the external sound collection device M1.

[0117] The detection unit 302 detects people in the vicinity of the construction machine 100 based on the image information acquired by the acquisition unit 301. Any known method may be used for detecting people. For example, a determination may be made as to whether a feature extracted from the image information is closer than or equal to a predetermined value to a feature representing a person.

[0118] When a person is detected by the detection unit 302, the position inference unit 303 infers the position of the person in the actual space, for example, the direction and distance of the person with reference to the construction machine 100. For example, the position inference unit 303 can infer the direction and distance of the person based on the position and size of the person captured in the image information. In addition, when a LiDAR, a range image camera, or a millimeter wave radar is installed in addition to the camera S6 as an object detection device, the direction and distance of the person can be inferred based on the detection result based on the object detection device. The example in which the position inference unit 303 involved in this embodiment obtains the direction and distance of the person with reference to the construction machine 100 as the position information of the person existing around the construction machine 100 is explained, but the position information of the person is not limited to the direction and distance of the person with reference to the construction machine 100. For example, information showing the position coordinates of the person in the world coordinate system can also be obtained.

[0119] In this embodiment, the detection of a person and the estimation of the person's position are not limited to the above-mentioned method, but a learned model may be used to detect a person and estimate the person's position (for example, direction and distance).

[0120] For the sound signal acquired by the acquisition unit 301, the sound processing unit 304 performs an emphasizing process on the frequency band of human voice while performing a reducing process on frequency bands other than the frequency band of human voice. As these processes, well-known methods can be used, for example, noise elimination processing can be performed, and band-pass filters can be used to suppress frequency bands other than the frequency band of human voice. Specifically, as a noise elimination process, the sound processing unit 304 performs the following processing: based on the sound signals from multiple external sound collecting devices M1, the sound generated from the construction machine 100 such as the engine 11 is suppressed. Since the sound generated from the construction machine 100 is suppressed, it is easy for the operator to hear the sound generated from the surrounding environment of the construction machine 100. The operator can easily grasp the surrounding conditions of the construction machine 100, thereby achieving improved safety.

[0121] The determination unit 305 determines whether a person is speaking based on the audio signal processed by the audio processing unit 304. For example, the determination unit 305 determines whether the volume of the audio signal in the frequency band from which the human voice is extracted is greater than a predetermined threshold value, and determines whether a person is speaking based on the determination result.

[0122] Moreover, when it is determined that there is a person speaking, the determination unit 305 determines the speaking direction based on the volume of each sound signal of the multiple external sound collection devices M1. The speaking direction is, for example, the direction in which the person speaking is located. For example, the determination unit 305 determines the direction in which the external sound collection device M1 that detects the sound with the loudest volume among the four directions of front, back, left, and right is configured as the speaking direction. In addition, this embodiment shows an example of a method for determining a direction, but is not limited to this method. For example, there is another method in which the speaking direction is determined based on the phase shift and volume difference in the sound signals of the multiple external sound collection devices M1.

[0123] The determination unit 306 determines whether the utterance direction specified by the identification unit 305 matches the direction in which the person is estimated by the position estimation unit 303 .

[0124] The output control unit 307 outputs information to one or more of the display device D1 and the internal sound output device SP2. For example, if a person near the construction machine 100 speaks, the output control unit 307 is configured to notify the user of the direction of the sound produced by the person's speech, based on the direction of the sound detected by the external sound collection device M1. This notification is made when the direction of the person detected based on the image information matches the direction of the speech.

[0125] For example, when the determination unit 306 determines that the directions match, the output control unit 307 notifies the detected direction, and when the determination unit 306 determines that the directions do not match (are different), the output control unit 307 suppresses notification of the detected direction.

[0126] Next, refer to Figure 7 , an example of the display screen 42 outputted to the display device D1 by the output control unit 307 will be described. Figure 7 1 is a diagram showing an example of a display screen 42 displayed on the display device D1 according to the present embodiment.

[0127] The output control unit 307 of this embodiment generates a display screen 42 based on the image information input from the camera S6, the sound signal input from the external sound collection device M1, and various information detected by various sensors installed in the construction machine 100. The various information detected by the various sensors includes the rotation angle, information indicating the area where a person is captured in the image information, position information of a person existing in the vicinity of the construction machine 100 (information indicating the direction and distance of the detected person), the person speaking based on the sound signal input from the external sound collection device M1, and the detection results of the various sensors.

[0128] The display screen 42 includes, under control of the output control unit 307, a date and time display area 42a, a travel mode display area 42b, an attachment display area 42c, a fuel consumption rate display area 42d, an engine control status display area 42e, a coolant temperature display area 42g, a fuel level display area 42h, a speed level display area 42i, a urea solution level display area 42j, a hydraulic oil temperature display area 42k, a status display area 1421, a first image display area 1422, a second image display area 1423, and a third image display area 1424. The display screen 42 may also include other display areas.

[0129] The travel mode display area 42b, the attachment display area 42c, the engine control status display area 42e, and the speed level display area 42i are areas that display information related to the set state of the construction machine 100, namely, set state information. The fuel consumption rate display area 42d, the engine operating time display area, the cooling water temperature display area 42g, the remaining fuel level display area 42h, the remaining urea water level display area 42j, and the operating oil temperature display area 42k are areas that display information indicating the operating state of the construction machine 100 based on the detection results of various sensors, namely, operating state information.

[0130] The date and time display area 42a displays the current date and time. The walking mode display area 42b displays the current walking mode. The accessory display area 42c displays images of currently worn accessories. The fuel consumption rate display area 42d displays fuel consumption rate information calculated by the controller 30. The fuel consumption rate display area 42d includes an average fuel consumption rate display area 42d1 that displays lifetime average fuel consumption rate or segment average fuel consumption rate, and an instantaneous fuel consumption rate display area 42d2 that displays instantaneous fuel consumption rate.

[0131] The engine control status display area 42e displays the control status of the engine 11. The engine operating time display area displays the cumulative operating time of the engine 11. The cooling water temperature display area 42g displays the current temperature of the engine cooling water. The remaining fuel level display area 42h displays the remaining fuel level in the fuel tank.

[0132] The speed level display area 42i is an area that graphically displays the current speed level set using the control dial (not shown). The number representing the selected speed level is displayed in the speed level display area 42i. "1" displayed in the speed level display area 42i indicates that the selected speed level is "Level 1." The number "n" displayed in the speed level display area 42i indicates that the selected speed level is "Level n." "n" is a natural number. As the operator rotates the control dial (not shown), the number displayed in the speed level display area 42i changes.

[0133] The urea water remaining amount display area 42j is an area for displaying the remaining amount of urea water stored in the urea water tank using an image, and the hydraulic oil temperature display area 42k is an area for displaying the temperature of the hydraulic oil in the hydraulic oil tank.

[0134] exist Figure 7 In the screen shown, the first image display area 1422, the second image display area 1423, and the third image display area 1424 are areas for displaying image information captured by the camera device S6. The first image display area 1422 displays the right image. The second image display area 1423 displays the rear image. The third image display area 1424 displays the left image. The right image represents the space to the right of the construction machine 100, the rear image represents the space behind the construction machine 100, and the left image represents the space to the left of the construction machine 100.

[0135] The first image display area 1422 is displayed on the right with respect to the status display area 1421. The second image display area 1423 is displayed below with respect to the status display area 1421. The third image display area 1424 is displayed on the left with respect to the status display area 1421. In addition, the bottom of the display screen 42 corresponds to the rear of the construction machine 100. That is, the display screen 42 displays the image information captured by the camera S6 in the direction in which the camera S6 is capturing, with respect to the status display area 1421. In addition, this embodiment shows an example of the configuration of image information and is not limited to this configuration. For example, the image display area can also be configured regardless of the direction of the capture. In this embodiment, the image information captured in the capturing direction is displayed with respect to the status display area 1421, so that when the operator refers to the image information, he or she can intuitively recognize in which direction the image information represents the situation. Therefore, safety can be improved.

[0136] When the position and distance of a person are determined by the position estimation unit 303, the output control unit 307 superimposes a frame indicating the detected person on the area determined by the determined position and distance for one or more of the right image, the rear image, and the left image.

[0137] As a result, a frame 1422b is displayed in the right image of the first image display area 1422 so as to surround the person 1422a, and a frame 1424b is displayed in the left image of the third image display area 1424 so as to surround the person 1424a.

[0138] The status display area 1421 is an area for displaying information indicating the positional relationship between the construction machine 100 and a person detected around the construction machine 100 .

[0139] The status display area 1421 is a display area that displays the actual space centered on the construction machine 100 at a predetermined scale ratio. In the status display area 1421, a construction machine icon (an example of display information) 1421b indicating the presence of the construction machine 100 is arranged at the center of the area.

[0140] In addition to the construction machine icon 1421b representing the construction machine 100, the status display area 1421 also displays a direction display icon 1421a indicating the moving direction of the construction machine 100 and icons indicating people detected from the periphery of the construction machine 100 (in the Figure 7 In the example of the status display area 1421, the area other than the construction machine icon 1421b, the direction display icon 1421a, the human detection icons 1421e, 1421f, 1421g, and the area 1421h (in other words, the background) can be, for example, an area represented by a single color (for example, black).

[0141] The construction machine icon 1421b is an icon formed by combining an image representing the upper revolving body 3 and an image representing the lower propelling body 1 in accordance with the positional relationship between the upper revolving body 3 and the lower propelling body 1 based on the revolving angle.

[0142] The direction display icon 1421a is a triangle that indicates the direction in which the construction machine 100 will travel when the travel pole is tilted forward. While this embodiment illustrates an example of an icon indicating the direction in which the construction machine 100 will travel when the travel pole is tilted forward, any shape may be used as long as it indicates the direction in which the construction machine 100 can travel.

[0143] The person detection icons 1421e, 1421f, and 1421g are icons representing people detected based on image information captured by the imaging device S6. Specifically, the person detection icons 1421e, 1421f, and 1421g are positioned based on the person's position and direction estimated by the position estimation unit 303. For example, the person detection icons 1421e, 1421f, and 1421g are positioned relative to the construction machine 100 at a position corresponding to the direction and distance of the person indicated by the estimation result multiplied by a predetermined scale ratio.

[0144] In this way, the positional relationship between the construction machine icon 1421b and the human detection icons 1421e, 1421f, and 1421g corresponds to the positional relationship between the construction machine 100 and the human beings existing around the construction machine 100 in the real space.

[0145] Furthermore, the operator can refer to the state display area 1421 to estimate how the positional relationship between the construction machine 100 and people present in the surrounding area will change when the construction machine 100 is moved.

[0146] Furthermore, a first circular area 1421 c and a second circular area 1421 d , which are determined based on the distance from the construction machine 100 with the construction machine 100 as a reference, are displayed in the state display area 1421 .

[0147] The first circular region 1421 c is information indicating the current swing range of the attachment AT of the construction machine 100 based on the detection results of the boom angle sensor S1 , the arm angle sensor S2 , and the bucket angle sensor S3 .

[0148] The second circular area 1421d is information indicating the range in which the attachment AT is rotated when the attachment AT is extended to the maximum in the horizontal direction.

[0149] The display screen 42 according to this embodiment displays a first circular area 1421c and a second circular area 1421d. Specifically, the positional relationship between the first and second circular areas 1421c, 1421d, and the human detection icons 1421e, 1421f, and 1421g allows identification of the positional relationship between the person and the accessory AT. This makes it easier for the operator to avoid contact between the person and the accessory AT, thereby improving safety.

[0150] Furthermore, the output control unit 307 displays the human detection icons 1421e, 1421f, and 1421g in different ways depending on whether they are included in the first circular area 1421c and the second circular area 1421d.

[0151] For example, the human detection icon 1421e existing in the first circular area 1421c is displayed in red, for example. Furthermore, the human detection icon 1421f existing outside the first circular area 1421c and in the second circular area 1421d is displayed in yellow, for example. Furthermore, the human detection icon 1421g existing outside the second circular area 1421d is displayed in blue, for example. The output control unit 307 involved in this embodiment makes the display method different depending on whether it is included in the first circular area 1421c and the second circular area 1421d. In other words, the color of the human detection icon changes according to the distance from the construction machine 100. In this way, the display device D1 can remind the operator to pay attention according to the distance between the construction machine 100 and the person by displaying the human detection icon whose color changes according to the distance. Therefore, it is possible to achieve improved safety.

[0152] By referring to the state display area 1421 , the operator can recognize how the positional relationship between the attachment AT of the construction machine 100 and surrounding people changes when turning the construction machine 100 .

[0153] Furthermore, the output control unit 307 makes the color of the frame 1422b of the right image of the first image display area 1422 consistent with the color of the human detection icon 1421e, and makes the color of the frame 1424b of the rear image of the third image display area 1424 consistent with the color of the human detection icon 1421f.

[0154] Specifically, frames representing the detected persons are displayed in the right and left images on the image display unit 42, and the correspondence between the persons represented by the frames and the person detection icons is displayed so that the operator can identify the status of the persons shown in the status display area 1421 by referring to the right and left images.

[0155] Furthermore, the output control unit 307 displays an area 1421h indicating the direction of speech determined by the determination unit 305 in the status display area 1421. Area 1421h of the status display area 1421 is displayed in a color different from the background (other areas) of the status display area 1421. That is, the output control unit 307 involved in this embodiment displays the area 1421h indicating the direction in which the sound generated by the human speech is detected (an example of information) in a manner different from the other areas (an example of information) on the display device D1, using the construction machine icon 1421b as a reference. Thus, the operator can identify the direction of speech by referring to the status display area 1421. Furthermore, the operator can infer who is talking from this direction. For example, the operator can infer that the person corresponding to the human detection icon 1421f included in the area 1421h is talking. Therefore, the operator can infer the person talking, making it easier to communicate with the person talking. This reduces the burden during operation.

[0156] Furthermore, the utterance direction identified by the identification unit 305 and the direction in which the person is located identified by the position estimation unit 303 may slightly deviate from each other.

[0157] Figure 8 This is a diagram illustrating an example of correction of the utterance direction in the output control unit 307 according to the present embodiment. Figure 8 (A) is an example in which the determination unit 305 detects the sound generated by a person's speech from the left direction among the four directions. At this time, the output control unit 307 displays the area 1502 corresponding to the left direction in a different color from other areas. Figure 8 , the position estimation unit 303 estimates that a person exists in the direction and distance indicated by the person detection icon 1501.

[0158] The utterance direction determined by the determination unit 305 may be shifted depending on the surrounding environment. The output control unit 307 according to the present embodiment corrects the area indicating the utterance direction based on the actual position of the person.

[0159] Output control unit 307 corrects the region representing the direction of speech based on the direction of the person detected based on the image information, as determined by position estimation unit 303. The basis for correction can be any basis. For example, even if the region representing the direction determined by determination unit 305 does not include the direction of the person estimated by position estimation unit 303, if it is closer than a specified basis (e.g., within 10 degrees), output control unit 307 corrects the region to include the direction of the person estimated by position estimation unit 303.

[0160] Figure 8 The middle (B) shows the status display area 1421 after the output control unit 307 corrects the area. Figure 8 The area 1503 shown in (B) is corrected to include the human detection icon 1421f. Therefore, the operator can Figure 8 The state display area 1421 in the middle (B) is used to identify the person who is talking and to communicate with the person. In this way, in this embodiment, even when the speaking direction determined by the external sound collection device M1 is misaligned, the output control unit 307 corrects the speaking direction according to the direction of the person detected based on the image information, thereby being able to prompt the speaking direction without making the operator feel uncomfortable. Therefore, when the construction machine 100 needs to be moved according to the content of the speech, the operator can understand in which direction the content of the speech is based and can understand in which direction the construction machine 100 should be moved, thereby improving convenience.

[0161] Figure 7 This is an example of the display screen 42 displayed by the output control unit 307 , and the utterance direction may be indicated in other forms.

[0162] Figure 9 42A is a diagram showing an example of a display screen 42A displayed by the display device D1 according to this embodiment. Figure 7 The same symbols are assigned to the same displays, and descriptions thereof are omitted.

[0163] exist Figure 9 In the illustrated screen, a first image display area 1422 , a second image display area 1423 , and a third image display area 1424 are areas for displaying image information captured by the imaging device S6 .

[0164] Then, output control unit 307 displays information indicating the utterance direction determined by determination unit 305 in the outer frame of at least one of first image display area 1422 , second image display area 1423 , and third image display area 1424 .

[0165] exist Figure 9 In the example shown, the outer frame 1424FL of the third image display area 1424 is displayed thicker than the outer frame 1422FL of the first image display area 1422 and the outer frame 1423FL of the second image display area 1423. Furthermore, the outer frame 1424FL is displayed in a different color than the outer frames 1422FL and 1423FL.

[0166] The operator can identify the direction of the speaker by referring to first image display area 1422, second image display area 1423, and third image display area 1424. Specifically, because outer frame 1424FL is displayed differently from the other outer frames, the operator can infer that the person captured in third image display area 1424, surrounded by outer frame 1424FL, is the speaker. This allows the operator to infer the speaker, making it easier to communicate with the speaker.

[0167] If the speaker's direction is forward, the speaker is not captured in the first image display area 1422, the second image display area 1423, and the third image display area 1424. In this case, the output control unit 307 may display information on the display device D1 that switches to a screen showing the front due to the sound being heard from the front. Alternatively, the output control unit 307 may automatically switch the display screen to include an image of the space in front of the construction machine 100 captured by the camera S6F.

[0168] exist Figure 9 In the example shown, the output control unit 307 displays image information captured by the camera S6 on the display device D1, and also displays the outer frame of the image information captured in the direction in which the sound generated by a person's speech is detected differently. This allows the operator to identify the direction of the speech and the current situation in that direction. This allows the operator to infer the speaker and easily communicate with them.

[0169] Figures 7 to 9 The display screens 42 and 42A shown are one form of display showing the periphery of the construction machine 100, but other forms may be used to show the periphery of the construction machine 100. As a method of displaying the periphery of the construction machine 100, an overhead image may be used.

[0170] Figure 10 42B is a diagram showing an example of a display screen 42B displayed by the display device D1 according to this embodiment. Figure 7 The same symbols are assigned to the same displays, and descriptions thereof are omitted.

[0171] Figure 10The bird's-eye view image 1700 shown in the screen is image information showing the surroundings of the construction machine 100 from above, based on the construction machine icon 1701. Bird's-eye view image 1700 is generated based on image information from each of the four cameras S6F, S6B, S6L, and S6R. The output control unit 307 of this embodiment synthesizes image information captured from the front, rear, right, and left, and converts it into an image viewed from directly above the construction machine 100, thereby generating bird's-eye view image 1700.

[0172] exist Figure 10 In the example shown, the output control unit 307 superimposes and displays a thick line 1704 indicating the area where the speaker is located on the outer frame 1702 of the bird's-eye view image 1700. The thick line 1704 may be a different color from the outer frame 1702 of the bird's-eye view image 1700.

[0173] By referring to the bird's-eye view image 1700, the operator can recognize the left direction indicated by the thick line 1704 as the direction of speech, with reference to the construction machine 100. Therefore, the operator can infer that the person 1703 captured in the bird's-eye view image 1700 is talking.

[0174] Therefore, with the controller 30 according to this embodiment, the operator can guess who is speaking, making it easier to communicate with the person who is speaking. This reduces the burden on the operator during operation.

[0175] exist Figure 10 In the example shown, only the overhead image 1700 is displayed, but the method is not limited to displaying only the overhead image 1700. The overhead image 1700 may be displayed in combination with the above-mentioned Figure 8 、 Figure 9 The output control unit 307 may display one or more of the first image display area 1422 for the right image, the second image display area 1423 for the rear image, and the third image display area 1424 for the left image in combination. For example, the output control unit 307 may output a display screen that combines the bird's-eye view image 1700, the first image display area 1422, and the second image display area 1423.

[0176] Next, the processing steps executed by the controller 30 according to the present embodiment will be described. Figure 11 This is a flowchart showing a processing procedure for displaying the utterance direction by the controller 30 according to the present embodiment.

[0177] First, the acquisition unit 301 acquires image information of the periphery of the construction machine 100 from the imaging device S6 ( S1801 ).

[0178] The detection unit 302 detects a person existing around the construction machine 100 based on the image information acquired by the acquisition unit 301 ( S1802 ).

[0179] When a person is detected by the detection unit 302 , the position estimation unit 303 estimates the direction and distance of the person with reference to the construction machine 100 ( S1803 ).

[0180] Next, the acquisition unit 301 acquires sound signals representing sounds generated around the construction machine 100 from each of the plurality of external sound collecting devices M1 ( S1804 ).

[0181] For each sound signal of the external sound collecting device M1 acquired by the acquisition unit 301 , the sound processing unit 304 performs a process of emphasizing the frequency band of human voice and a process of reducing the frequency band other than the frequency band of human voice (noise removal process) ( S1805 ).

[0182] The identification unit 305 identifies the utterance direction based on the audio signal processed by the audio processing unit 304 ( S1806 ).

[0183] The determination unit 306 determines whether the utterance direction specified by the specification unit 305 corresponds to the direction in which the person is estimated by the position estimation unit 303 ( S1807 ).

[0184] When the determination unit 306 determines that there is no coincidence ( S1807 : No), the display of the utterance direction is suppressed, and the process proceeds to S1809 .

[0185] On the other hand, when the determination unit 306 determines that the two directions are consistent (S1807: Yes), the output control unit 307 displays the utterance direction (S1808). Figure 7 Display screen 42, Figure 9 Display screen 42A and Figure 10 Displayed in any of the display screen 42B.

[0186] Then, the output control unit 307 outputs the audio signal processed by the audio processing unit 304 ( S1809 ).

[0187] In this embodiment, by performing the above-described processing steps, the operator can recognize the direction of speech in the vicinity of the construction machine 100. The operator can estimate the speaker by visually recognizing the direction of speech, thereby facilitating communication with the speaker.

[0188] Furthermore, in the present embodiment, by performing emphasis processing on the frequency band of human voice and outputting a sound signal from which noise has been removed, the operator can easily understand what the person present around the construction machine 100 has said.

[0189] This embodiment shows an example of the configuration of the external sound collecting device M1, but is not limited to this configuration example. For example, one or more microphone arrays can be set on the upper rotating body 3. The microphone array has multiple microphones, and the differences (time difference, phase difference) of the sounds observed by the multiple microphones are processed / analyzed, thereby being able to identify the direction of the sound. In this way, when a directional microphone array is provided, the direction of the sound can be detected based on the sound detected by each microphone included in the microphone array.

[0190] In this embodiment, an example in which four external sound collecting devices M1 are installed is described. However, this embodiment does not limit the number of external sound collecting devices M1 installed on the construction machine 100, and the number may be three or less or five or more.

[0191] (Variation)

[0192] In this embodiment, the example of displaying the direction on the display device D1 is described as a notification method. However, the above embodiment is not limited to the method of displaying the direction on the display device D1. This variation is an example of providing multiple internal sound output devices SP2 within the cab 10. For example, the internal sound output devices SP2 are installed in the cab 10 in the front, rear, left, and right directions relative to the operator's seat.

[0193] The determination unit 305 of this variation determines the direction of speech based on the sound signals from the four external sound collection devices M1, and the output control unit 307 controls the output of sound from the internal sound output device SP2 corresponding to the direction of speech, among the multiple internal sound output devices SP2. In this variation, the operator can identify the direction of speech from the direction of the sound output. Furthermore, the construction machine 100 of this variation does not necessarily need to be equipped with the camera S6.

[0194] This modification shows one way of the notification method, and other ways are also possible. For example, the cab 10 may be provided with a plurality of light source units, and the output control unit 307 controls the lighting of the light source unit corresponding to the direction of speech among the plurality of light source units. As another example, the plurality of operating levers of the operating device 26 may be respectively provided with a vibration mechanism, and the output control unit 307 controls the vibration mechanism corresponding to the direction existing in the direction of speech among the plurality of operating levers to vibrate. In this modification, a right operating lever and a left operating lever are provided, so that the direction of speech can be intuitively identified by being limited to the left and right directions. In this modification, the operator can identify the direction of speech by performing the above-mentioned notification method. Therefore, the operator can infer the person who is speaking by visually recognizing the direction of speech, and thus it is easy to communicate with the person who is speaking.

[0195] The notification method is not limited to a single method and may be a combination of multiple methods. For example, a combination of two or more of the following may be used: displaying the direction on the display device D1, outputting a sound from the internal sound output device SP2 corresponding to the direction, lighting a light source corresponding to the direction, and vibrating each of the plurality of operating levers of the operating device 26.

[0196] (Second embodiment)

[0197] Next, refer to Figure 12 , another structural example of the construction machine 100 involved in the second embodiment is described. Figure 12 It is a plan view of another configuration example of the construction machine 100 according to the second embodiment. Figure 12 The construction machine 100 shown is Figure 1 The construction machine 100 shown is different in that the external sound output device SP1 is composed of four speakers (front speaker SP1F, left speaker SP1L, right speaker SP1R and rear speaker SP1B). Figure 1 In the illustrated construction machine 100 , the external sound output device SP1 is composed of a single omnidirectional speaker provided on the cab 10 .

[0198] Through this structure, such as Figure 12 The construction machine 100 shown, for example, can output sound to the worker WK located in front of the construction machine 100 instead of outputting sound to the worker WK located to the left, right and rear of the construction machine 100 by setting the front speaker SP1F to be on (a state capable of outputting sound) and setting the left speaker SP1L, the right speaker SP1R and the rear speaker SP1B to be off (a state unable to output sound).

[0199] And, in Figure 12In the example shown, a front camera S6F and a front microphone M1F are positioned adjacent to the front speaker SP1F, with a front light bar G1F attached to the housing of the front microphone M1F. Furthermore, a left camera S6L and a left microphone M1L are positioned adjacent to the left speaker SP1L, with a left light bar G1L attached to the housing of the left microphone M1L. Furthermore, a right camera S6R and a right microphone M1R are positioned adjacent to the right speaker SP1R, with a right light bar G1R attached to the housing of the right microphone M1R. Furthermore, a rear camera S6B and a rear microphone M1B are positioned adjacent to the rear speaker SP1B, with a rear light bar G1B attached to the housing of the rear microphone M1B.

[0200] At this time, the construction machine 100 may turn on the light bar corresponding to the speaker set to on (a state capable of emitting light), and turn off the light bar corresponding to the speaker set to off (a state incapable of emitting light).

[0201] The external sound output device SP1 may also be composed of one or more parametric speakers. A parametric speaker is a speaker that uses ultrasound and can selectively transmit sound to a person within a specific narrow range. A parametric speaker can transmit sound to any location.

[0202] exist Figure 12 In the construction machine 100 shown, the controller 30 can detect a worker WK located around the construction machine 100 based on the image captured by the camera S6 and determine the position of the worker WK. Furthermore, if there are multiple workers WK around the construction machine 100, the controller 30 can distinguish between the intended conversation partner (the worker WK emitting sound) and the non-intended conversation partner (the worker WK not emitting sound) based on the output of the four external sound collection devices M1. Furthermore, the controller 30 can also distinguish between the intended conversation partner (the worker WK facing the construction machine 100) and the non-intended conversation partner (the worker WK not facing the construction machine 100) based on the image captured by the camera S6. Furthermore, the controller 30 can also turn on the speaker and light bar facing the worker WK. For example, if there is a worker WK (emitting sound) behind the construction machine 100, the controller 30 can turn on the rear speaker SP1B while keeping the front speaker SP1F, left speaker SP1L, and right speaker SP1R in the off state. At this time, the controller 30 can turn on the rear light bar G1B while keeping the front light bar G1F, the left light bar G1L and the right light bar G1R in the off state. Figures 1 to 5 The construction machine 100 is shown as being implemented.

[0203] With this configuration, the controller 30 can output sound in the direction where the worker WK is present, rather than in the direction where the worker WK is not present. Therefore, the worker WK can easily recognize whether he or she is a conversation partner or a non-conversation partner.

[0204] (Third embodiment)

[0205] In the third embodiment, a case where an operator performs remote control of the construction machine 100 will be described.

[0206] Next, refer to Figure 13 , a configuration example of an operating system (an example of a control system) SYS according to a third embodiment will be described. Figure 13 1 is a schematic diagram showing a configuration example of an operating system (an example of a control system for a construction machine) SYS of the construction machine 100. Figure 13 As shown, the operating system SYS includes a construction machine 100, a remote operation room RC and a management center MC. Figure 13 In the figure, the detailed structure of the construction machine 100 is omitted. This is because Figure 13 The construction machine 100 shown has Figure 1 or Figure 10 The construction machine 100 shown has the same structure.

[0207] The construction machine 100, the remote operation room RC, and the management center MC are connected to each other in a manner that enables data transmission and reception via a communication network NW. Alternatively, the construction machine 100, the remote operation room RC, and the management center MC can be connected to each other in a manner that enables direct data transmission and reception without going through the communication network NW. In the illustrated example, the construction machine 100 transmits information related to the work site to the remote operation room RC. This allows the remote operator RO in the remote operation room RC to understand the status of the work site based on the information received from the construction machine 100.

[0208] For example, the construction machine 100 transmits image information captured by the imaging device S6 and a sound signal representing the sound collected by the external sound collecting device M1 to the remote operation room RC.

[0209] The construction machine 100 is equipped with sensors that can three-dimensionally identify the position and shape of objects at the work site. For example, the construction machine 100 is equipped with a space recognition device. Therefore, the construction machine 100 can transmit the results of three-dimensional measurement of the work site to the remote control room RC.

[0210] The space recognition device is a device for identifying the space surrounding the construction machine 100. In the illustrated example, the space recognition device is a LiDAR. The LiDAR, for example, measures the distance between each of the one million or more points within the monitoring range and the LiDAR. Furthermore, any space recognition device capable of measuring the distance to an object may be used. For example, the space recognition device may be a stereo camera or a combination of an imaging device S6 and a distance measuring device such as a millimeter-wave radar.

[0211] The operating system SYS may include one or more construction machines 100. In the case of multiple construction machines 100, the remote operator RO of a specific construction machine 100 can obtain not only information related to the work site obtained by the specific construction machine 100, but also information related to the work site obtained by one or more other construction machines 100.

[0212] The remote control room RC is equipped with a communication device T2, a remote controller R40, an operating device R42, an operating sensor R43, a display device D1E, an internal sound collection device M2E, and an internal sound output device SP2E. Furthermore, the remote control room RC has an operator's seat DS where a remote operator RO, who remotely operates the construction machine 100, sits.

[0213] The communication device T2 is configured to be able to communicate with the communication device T1 mounted on the construction machine 100 .

[0214] The remote controller R40 is a computing device that performs various operations. In this embodiment, the remote controller R40 is composed of a microcomputer including a CPU and a memory. Furthermore, the various functions of the remote controller R40 are realized by the CPU executing programs stored in the memory.

[0215] The display device D1E is a device capable of displaying various information. The display device D1E displays images based on information transmitted from the construction machine 100, allowing the remote operator RO in the remote control room RC to visually identify the surroundings of the construction machine 100. In the illustrated example, the display device D1E is a liquid crystal display. Alternatively, the display device D1E may be a display or projector that enables naked-eye stereoscopic vision, or may be VR glasses, for example.

[0216] The internal sound collecting device M2E is a device for collecting sounds generated in the remote operation room RC. In the example shown in the figure, the internal sound collecting device M2E is a room microphone and is configured to pick up sounds uttered by the remote operator RO in the remote operation room RC.

[0217] The internal sound output device SP2E is a device capable of outputting various sound information. The internal sound output device SP2E outputs sound based on information transmitted from the construction machine 100 so that the remote operator RO in the remote operation room RC can hear the sound produced at the work site. The internal sound output device SP2E can be configured to output sound captured by an external sound collection device M1 installed on the outside of the cab 10, or it can be configured to output sound captured by an internal sound collection device M2 installed on the inside of the cab 10. In this case, the internal sound collection device M2 can be located at a position corresponding to the position of the ears of the operator sitting in the driver's seat 50 in the cab 10. The internal sound output device SP2E can be a set-type device such as a speaker, or a wearable device such as earphones or headphones. The speaker can be a monaural speaker, a stereo speaker, or a surround sound speaker. Furthermore, the speaker can be an omnidirectional speaker or a directional speaker. The wearable device can be a device with a noise cancellation function, a device with a spatial audio function (stereo function), or a device with a bone conduction function.

[0218] An operation sensor R43 for detecting the operation content of the operation device R42 is provided in the operation device R42. The operation sensor R43 is, for example, a tilt sensor for detecting the tilt angle of the operation lever or an angle sensor for detecting the swing angle of the operation lever around the swing axis. The operation sensor R43 may also be composed of other sensors such as a pressure sensor, a current sensor, a voltage sensor or a distance sensor. The operation sensor R43 outputs information related to the detected operation content of the operation device R42 to the remote controller R40. The remote controller R40 generates an operation signal based on the received information and sends the generated operation signal to the construction machine 100. The operation sensor R43 may also be configured to generate an operation signal. In this case, the operation sensor R43 may output the operation signal to the communication device T2 without passing through the remote controller R40. With this structure, the remote operator RO can remotely operate the construction machine 100 from the remote operation room RC.

[0219] The remote controller R40 , similar to the controller 30 of the first embodiment, includes an acquisition unit 301 , a detection unit 302 , a position estimation unit 303 , a sound processing unit 304 , a determination unit 305 , a determination unit 306 , and an output control unit 307 .

[0220] The acquisition unit 301 of the remote controller R40 acquires a sound signal representing the sound collected by the external sound collection device M1 from the construction machine 100 via the communication device T2. The acquisition unit 301 also acquires a sound signal representing the sound collected by the internal sound collection device M2E in the remote control room RC.

[0221] The output control unit 307 of the remote controller R40 can control the output of sound based on the sound signal acquired from the external sound collecting device M1 by the acquisition unit 301 from the internal sound output device SP2E of the remote operation room RC via the communication device T2. This allows the remote operator RO to hear the voices of people around the construction machine 100.

[0222] The output control unit 307 of the remote controller R40 can control the output of sound based on the sound signal acquired by the acquisition unit 301 from the internal sound collection device M2E of the remote operation room RC from the external sound output device SP1 of the construction machine 100 via the communication device T2. This allows people around the construction machine 100 to hear the voice of the remote operator RO.

[0223] The detection unit 302 , the position estimation unit 303 , the sound processing unit 304 , the identification unit 305 , and the determination unit 306 of the remote controller R40 perform the same control as that of the controller 30 in the above-described embodiment.

[0224] The remote controller R40 has this structure, and thus, similarly to the controller 30 described above, it can determine the direction of speech based on the acquired sound signal. Figure 7 The display screen 42 shown, Figure 9 The display screen 42A or Figure 10 The display screen 42B shown is displayed on the display device DR.

[0225] The management center MC is a facility equipped with various devices for managing the remote operation of the construction machine 100 by the remote operator RO located in the remote operation room RC. In the illustrated example, the management center MC is located at a location separate from the construction machine 100 work site and the remote operation room RC. Furthermore, the management center MC is equipped with a management device 200, an internal sound collection device M2C, and an internal sound output device SP2C.

[0226] The management device 200 is an example of a control device, such as a server computer (so-called cloud server) or an edge server. Typically, the management device 200 is a fixed terminal device, but it can also be a portable terminal device (such as a laptop computer, tablet computer, or smartphone). The management device 200 can perform the same control as the remote controller R40.

[0227] Through this structure, the manager at the management center MC can, for example, hear the sound emitted at the work site by using the sound collecting device (external sound collecting device M1 or internal sound collecting device M2) and the internal sound output device SP2C installed on the construction machine 100. Furthermore, the manager at the management center MC can, for example, hear the sound emitted in the remote operation room RC by using the internal sound collecting device M2E and the internal sound output device SP2C installed in the remote operation room RC. Furthermore, the manager at the management center MC can, for example, transmit the sound emitted by himself to the worker WK located around the construction machine 100 by using the internal sound collecting device M2C and the external sound output device SP1 installed on the construction machine 100. Furthermore, the manager at the management center MC can, for example, transmit the sound emitted by himself to the remote operator RO located in the remote operation room RC by using the internal sound collecting device M2C and the internal sound output device SP2E installed in the remote operation room RC.

[0228] Furthermore, the management device 200 has the same structure as the remote controller R40. Therefore, the management device 200 is similar to the above-mentioned embodiment. Figure 7 The display screen 42 shown, Figure 9 The display screen 42A or Figure 10 The display screen 42B shown is displayed on the display device DR.

[0229] In this embodiment, the internal sound collection device M2C and the internal sound output device SP2C perform the same functions as the internal sound collection device M2E and the internal sound output device SP2E. The management device 200 transmits and receives sound signals to and from the construction machine 100 or the remote control room RC. This enables a two-way conversation between the administrator at the management center MC and the remote operator RO in the remote control room RC or people (including the worker WK) near the construction machine 100. The specific controls for achieving this two-way conversation are the same as those in the aforementioned embodiment and are omitted for clarity.

[0230] Thus, in this embodiment, a two-way conversation is achieved between the remote operator RO at the operator's seat DS and people around the construction machine 100. Similarly, a two-way conversation is achieved between the manager managing the work site in the management device 200 and people around the construction machine 100. Therefore, even in remote operation and remote management, communication with people around the construction machine 100 is facilitated.

[0231] Furthermore, in this embodiment, the remote operator RO at the operator's seat DS and the manager at the management center MC can confirm the direction of the speaker around the construction machine 100. Therefore, even when performing remote operation or remote management, communication with people around the construction machine 100 is easy.

[0232] <Function>

[0233] In the above-described embodiment and variations, the operator can identify the direction of the speaker's speech from the surrounding area of ​​the construction machine 100. This allows the operator to easily communicate with the speaker. Furthermore, the operator can understand the speaker's surroundings and thus recognize the speaker's intention. The operator can determine how to best operate the construction machine 100 based on the speaker's intention. Therefore, in the embodiment and variations, the operating efficiency of the construction machine 100 can be improved.

[0234] While the above describes embodiments of the construction machinery and control system of the present invention, the present invention is not limited to the aforementioned embodiments. Various changes, modifications, substitutions, additions, deletions, and combinations are possible within the scope of the technical proposal. Such changes, modifications, substitutions, additions, deletions, and combinations are naturally within the technical scope of the present invention.

Claims

1. A construction machine comprising: Lower walking body; An upper rotating body is mounted on the lower traveling body in a freely rotatable manner; a sound collecting device, provided on the upper rotating body, and arranged in a manner capable of detecting the direction of sounds around the construction machine; and The control device is configured to notify a person present in the vicinity of the construction machine of a direction of speech based on the direction of the sound detected by the sound collecting device when the person present in the vicinity of the construction machine speaks.

2. The construction machine according to claim 1, wherein: As a notification, the control device performs any one or more of displaying the speech direction on a display device, outputting the sound from a sound output device corresponding to the speech direction among a plurality of sound output devices arranged on the inner side of the cab, lighting a light source portion corresponding to the speech direction among a plurality of light source portions, and vibrating an operating device corresponding to the speech direction among a plurality of operating devices.

3. The construction machine according to claim 2, wherein: The control device displays the information indicating the utterance direction on the display device in a display format different from that of information indicating other directions, based on display information indicating the construction machine.

4. The construction machine according to claim 3, wherein: A camera device is further provided on the upper rotating body. The control device displays a bird's-eye view image of the surroundings of the construction machine from above the construction machine based on the image information obtained by the camera device, and displays the information indicating the speech direction in a different display method from the information indicating other directions based on the display information indicating the construction machine contained in the bird's-eye view image.

5. The construction machine according to claim 2, wherein: A plurality of camera devices are further provided on the upper rotating body. The control device displays the image information captured by the plurality of imaging devices on the display device with reference to the construction machine, and displays the image information captured in the utterance direction in a different manner.

6. The construction machine according to claim 1, wherein: A camera device is further provided on the upper rotating body. The control device is configured to correct the speech direction based on the direction of the person detected based on image information captured by the imaging device.

7. The construction machine according to claim 1, wherein: A camera device is further provided on the upper rotating body. The control device suppresses notification of the direction when the direction of the person detected based on the image information captured by the imaging device is different from the utterance direction.

8. The construction machine according to claim 1, wherein: The sound collecting device is provided with a plurality of The plurality of sound collecting devices are installed at different positions of the upper slewing body so as to be able to detect directions of sounds around the construction machine.

9. The construction machine according to claim 8, wherein: The control device suppresses the sound generated by the construction machine based on the sound signals input from the plurality of sound collecting devices.

10. A control system for a construction machine, comprising: A construction machine comprising: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; and a sound collecting device provided on the upper rotating body and capable of detecting the direction of sounds around the construction machine; and The control device is configured to notify a person present in the vicinity of the construction machine of a direction of speech based on the direction of the sound detected by the sound collecting device when the person present in the vicinity of the construction machine speaks.

Images