Work machine, and control system of work machine

Abstract

To provide a work machine capable of facilitating communication with a person in the surrounding area.SOLUTION: A work machine in an embodiment includes: a lower running body; an upper rotating body that is rotatably mounted on the lower running body; a sound collection device that is placed on the upper rotating body so as to detect the direction of sounds around the work machine; a control unit that is configured so as to, when there are people around the work machine who are engaging in verbal communication, notify the direction of verbal communication based on the direction of the sound detected by the sound collection device.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a work machine and a control system for the work machine. [Background technology]

[0002] Conventionally, workers have often been working around excavators. Therefore, a technique for detecting people around an excavator based on an image captured by an imaging device mounted on an upper rotating body is known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2023-63993 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, when a worker is working with a work machine, he or she may be spoken to by a person nearby the work machine. Sound detected by a microphone attached to the work machine is output to the operator of the work machine from a speaker. This allows the operator to understand what has been said. However, because the operator of the work machine does not know the direction in which the person speaking is located, he or she may not know how to respond.

[0005] One aspect of the present invention is to notify the direction of speech, thereby making it easier to identify the person who spoke and to communicate with the person who spoke. [Means for solving the problem]

[0006] A work machine according to one aspect of the present invention comprises a lower running body, an upper rotating body mounted on the lower running body so as to be freely rotatable, a sound collection device provided on the upper rotating body and capable of detecting the direction of sound around the work machine, and a control device configured to notify the direction of speech based on the direction of sound detected by the sound collection device when a person in the vicinity of the work machine is speaking. [Effects of the Invention]

[0007] According to one aspect of the present invention, communication with the person who spoke is facilitated. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a side view of a work machine according to a first embodiment. [Figure 2] 1 is a top view of a work machine according to a first embodiment. [Figure 3] 2 is a diagram showing an example of the configuration of an external sound collecting device and an information transmitting device attached to the work machine of FIG. 1. [Figure 4] 1 is a diagram schematically illustrating an example of the configuration of a work machine according to a first embodiment. [Figure 5] FIG. 2 is a top view of the interior of the operator's cab according to the first embodiment. [Figure 6] 1 is a conceptual diagram showing a two-way conversation between an operator on board a work machine according to a first embodiment and a worker present in the vicinity of the work machine. FIG. [Figure 7] FIG. 2 is a diagram showing an example of a display screen displayed by the display device according to the first embodiment. [Figure 8] 5A to 5C are diagrams illustrating an example of correction of the speaking direction in the output control unit according to the first embodiment. [Figure 9] FIG. 2 is a diagram showing an example of a display screen displayed by the display device according to the first embodiment. [Figure 10] FIG. 2 is a diagram showing an example of a display screen displayed by the display device according to the first embodiment. [Figure 11]5 is a flowchart showing a processing procedure for the controller according to the first embodiment to display the speaking direction. [Figure 12] FIG. 10 is a top view of another configuration example of a work machine according to a second embodiment. [Figure 13] FIG. 10 is a schematic diagram showing an example of the configuration of a remote support system for a work machine according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments described below are merely examples and do not limit the invention, and all features and combinations described in the embodiments are not necessarily essential to the invention. In addition, identical or corresponding components in each drawing are designated by identical or corresponding reference numerals, and descriptions thereof may be omitted.

[0010] The work machine 100 according to the embodiment of the present disclosure is a shovel. The work machine 100 may be a machine other than a shovel, such as a crane, an asphalt finisher, or a forklift. In the illustrated example, the shovel serving as the work machine 100 is an excavator equipped with a bucket 6 as an end attachment, but the work machine 100 may also be an applied machine such as a forestry machine equipped with an end attachment other than the bucket 6.

[0011] In the following description, work machine 100 is an excavator equipped with a bucket 6 as an end attachment, but the present invention is not limited to excavators. Work machine 100 may also be an application machine such as a forestry machine equipped with an end attachment other than a bucket 6.

[0012] (First embodiment) First, an overview of a work machine 100 will be described with reference to Figures 1 and 2. Figure 1 is a top view of the work machine 100, and Figure 2 is a side view of the work machine 100.

[0013] The work machine 100 comprises a lower traveling body 1, an upper rotating body 3 that is mounted on the lower traveling body 1 so as to be rotatable via a rotating mechanism 2, an attachment AT for performing various tasks, and a cab 10. The front of the work machine 100 (upper rotating body 3) corresponds to the direction in which the attachment AT extends relative to the upper rotating body 3 when the work machine 100 is viewed from directly above along the rotation axis of the upper rotating body 3 (top view). Furthermore, the left and right sides of the work machine 100 (upper rotating body 3) correspond to the left and right sides, respectively, as seen from the operator seated in the driver's seat in the cab 10.

[0014] The lower traveling body 1 includes, for example, a pair of left and right crawlers 1C. Specifically, the crawlers 1C include a left crawler 1CL and a right crawler 1CR. The left crawler 1CL is driven by a left traveling hydraulic motor 2ML, and the right crawler 1CR is driven by a right traveling hydraulic motor 2MR. The left traveling hydraulic motor 2ML is a traveling drive unit that drives the left crawler 1CL as a driven part, and can rotate the left crawler 1CL. The right traveling hydraulic motor 2MR is a traveling drive unit that drives the right crawler 1CR as a driven part, and can rotate the right crawler 1CR. Note that the traveling drive units may be electric motors.

[0015] The upper rotating body 3 rotates relative to the lower traveling body 1 when the rotating mechanism 2 is driven by the hydraulic swing motor 2A. The hydraulic swing motor 2A is a swing drive unit that drives the upper rotating body 3 as a driven unit, and can change the orientation of the upper rotating body 3. The swing drive unit may be an electric motor.

[0016] A boom 4 is rotatably attached to the center of the front of the upper rotating body 3, an arm 5 is rotatably attached to the tip of the boom 4, and a bucket 6 is rotatably attached to the tip of the arm 5. In the illustrated example, the boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of an attachment AT. The boom 4, arm 5, and bucket 6 are driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.

[0017] The bucket 6 is an example of a work tool (end attachment). The bucket 6 is used, for example, for excavation work. Instead of the bucket 6, another work tool may be attached to the end of the arm 5 depending on the type of work, etc. The other work tool may be, for example, another type of bucket, such as a large bucket, a slope bucket, or a dredging bucket. The other work tool may also be a type of work tool other than a bucket, such as an agitator, a breaker, a grapple, or a lifting magnet.

[0018] The swing hydraulic motor 2A, the left traveling hydraulic motor 2ML, the right traveling hydraulic motor 2MR, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9 are hydraulic actuators driven by hydraulic oil discharged from a hydraulic pump.

[0019] Note that in the work machine 100, all or some of the driven parts, such as the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6, may be electrically driven. In other words, the work machine 100 may be a hybrid excavator, an electric excavator, or the like, in which all or some of the driven parts are driven by electric actuators.

[0020] Additionally, the work machine 100 is equipped with an information transmission device G1, an external sound collection device M1, an imaging device S6, and an external sound output device SP1.

[0021] The imaging device S6 is provided on the upper rotating body 3 or the cab 10, and captures images of the periphery of the work machine 100 to obtain image information showing the periphery of the work machine 100. In the illustrated example, the imaging device S6 includes a front camera S6F, a left camera S6L, a right camera S6R, and a rear camera S6B.

[0022] The front camera S6F is a camera that captures images in front of the work machine 100, and is attached to the exterior of the cab 10, such as on the roof of the cab 10 or on the side of the boom 4. The front camera S6F may also be attached, for example, to the ceiling of the cab 10, i.e., inside the cab 10. The left camera S6L is a camera that captures images to the left of the work machine 100, the right camera S6R is a camera that captures images to the right of the work machine 100, and the rear camera S6B is a camera that captures images to the right of the work machine 100. Specifically, the front camera S6F, left camera S6L, right camera S6R, and rear camera S6B are all monocular wide-angle cameras equipped with an imaging element such as a CCD or CMOS, and output the captured images to the display device D1. Information about the images captured by the imaging device S6 is taken into the controller 30.

[0023] In the illustrated example, the front camera S6F is attached to the roof of the driver's cab 10, the left camera S6L is attached to the left end of the upper surface of the upper rotating body 3, the right camera S6R is attached to the right end of the upper surface of the upper rotating body 3, and the rear camera S6B is attached to the rear end of the upper surface of the upper rotating body 3.

[0024] The imaging device S6 may constitute an object detection device that detects objects in the vicinity of the work machine 100. The object detection device may be constituted by a device other than a camera. For example, the object detection device may be a LiDAR. A LiDAR is a device that can measure the distance between a point cloud of one million or more points within a monitoring range and the LiDAR (laser source). The object detection device may also be another device that can measure the distance to an object, such as a stereo camera, a range imaging camera, or a millimeter-wave radar. When a millimeter-wave radar or the like is used as the object detection device, the object detection device may derive the distance and direction of the object by emitting a number of signals (laser light, etc.) toward the object and receiving the reflected signals. Alternatively, the object detection device may be a combination of two or more types of devices. For example, the object detection device may be a combination of an imaging device and a LiDAR, a combination of an imaging device and a millimeter-wave radar, or a combination of an imaging device and a stereo camera.

[0025] The external sound collector M1 is a device that collects external sounds and is also called a microphone. In the illustrated example, the external sound collector M1 is provided on the upper rotating body 3 or the cab 10, and converts sounds (air vibrations) generated around the work machine 100 into mechanical vibrations, which are then converted into electrical signals. Specifically, the external sound collector M1 includes a front microphone M1F, a left microphone M1L, a right microphone M1R, and a rear microphone M1B.

[0026] The front microphone M1F is a microphone that collects sounds generated in front of the work machine 100, and is attached to the exterior of the cab 10, such as on the roof of the cab 10 or on the side of the boom 4. The front microphone M1F may also be attached to the ceiling of the cab 10, i.e., the interior of the cab 10, for example. The left microphone M1L is a microphone that collects sounds generated to the left of the work machine 100, the right microphone M1R is a microphone that collects sounds generated to the right of the work machine 100, and the rear microphone M1B is a microphone that collects sounds generated behind the work machine 100. Electrical signals generated by the front microphone M1F, left microphone M1L, right microphone M1R, and rear microphone M1B are taken into the controller 30.

[0027] In the illustrated example, the front microphone M1F is attached to the roof of the cab 10, the left microphone M1L is attached to the left end of the upper surface of the upper rotating body 3, the right microphone M1R is attached to the right end of the upper surface of the upper rotating body 3, and the rear microphone M1B is attached to the rear end of the upper surface of the upper rotating body 3. In this way, the four external sound collection devices M1 (the front microphone M1F, the left microphone M1L, the right microphone M1R, and the rear microphone M1B) are provided at different positions on the upper rotating body 3. Therefore, the controller 30 can detect the direction of the sound source based on the difference in the sounds collected by each of the four external sound collection devices M1 (e.g., difference in volume). Furthermore, when an array microphone is used as the external sound collection device M1, the direction of the sound source can be detected based on, for example, a phase shift or difference in volume.

[0028] In the illustrated example, the four external sound collection devices M1 and the four image capture devices S6 are arranged to correspond to one another. Specifically, the front microphone M1F is arranged adjacent to the front camera S6F, the left microphone M1L is arranged adjacent to the left camera S6L, the right microphone M1R is arranged adjacent to the right camera S6R, and the rear microphone M1B is arranged adjacent to the rear camera S6B.

[0029] The external sound output device SP1 is a device that outputs sound toward the periphery of the work machine 100. In the illustrated example, the external sound output device SP1 is an omnidirectional speaker that is configured to output sound uniformly in all directions. However, the external sound output device SP1 may also be a directional speaker that outputs sound toward the front.

[0030] The information transmission device G1 is a device for communicating the status of the work machine 100 to someone outside the work machine 100. In the illustrated example, the information transmission device G1 is provided on the upper rotating body 3 or the operator's cab 10, and is configured to be able to communicate the status of the work machine 100 to an operator around the work machine 100. Specifically, the information transmission device G1 includes a front light bar G1F, a left light bar G1L, a right light bar G1R, and a rear light bar G1B.

[0031] The front light bar G1F is a light-emitting device that can visually convey information to workers and the like in front of the work machine 100, and is attached to the exterior of the cab 10, such as on the roof of the cab 10 or on the side of the boom 4. The front light bar G1F may also be attached, for example, to the ceiling of the cab 10, i.e., inside the cab 10. The left light bar G1L is a light-emitting device that can visually convey information to workers and the like on the left side of the work machine 100, the right light bar G1R is a light-emitting device that can visually convey information to workers and the like on the right side of the work machine 100, and the rear light bar G1B is a light-emitting device that can visually convey information to workers and the like behind the work machine 100. The front light bar G1F, left light bar G1L, right light bar G1R, and rear light bar G1B each emit light in response to an electrical signal from the controller 30. In the illustrated example, the light-emitting devices are LED lights, but they may also be other light-emitting devices, such as halogen lamps. Furthermore, the light emitting device is of a multicolor emission type, but may be of a monochromatic emission type.

[0032] In the illustrated example, the front light bar G1F is attached to the roof of the operator's cab 10, the left light bar G1L is attached to the left end of the upper surface of the upper rotating body 3, the right light bar G1R is attached to the right end of the upper surface of the upper rotating body 3, and the rear light bar G1B is attached to the rear end of the upper surface of the upper rotating body 3. In this way, the four information transmission devices G1 (front light bar G1F, left light bar G1L, right light bar G1R, and rear light bar G1B) are provided in different positions on the upper rotating body 3. Therefore, the controller 30 can communicate the status of the work machine 100 to workers and the like who are respectively located in front of, on the left, right, and rear of the work machine 100 by separately operating each of the four information transmission devices G1.

[0033] In the illustrated example, the four information transmission devices G1 and the four external sound collection devices M1 are arranged to correspond to each other. Specifically, the front light bar G1F is arranged adjacent to the front microphone M1F, the left light bar G1L is arranged adjacent to the left microphone M1L, the right light bar G1R is arranged adjacent to the right microphone M1R, and the rear light bar G1B is arranged adjacent to the rear microphone M1B.

[0034] Fig. 3 is a diagram showing an example configuration of an external sound collection device M1 and an information transmission device G1 attached to a work machine 100. Specifically, Fig. 3 is a perspective view of a left microphone M1L and a left light bar G1L attached to a housing having a substantially rectangular parallelepiped shape. Note that the following description with reference to Fig. 3 relates to the combination of the left microphone M1L and the left light bar G1L, but it also applies to the combination of the front microphone M1F and the front light bar G1F, the combination of the right microphone M1R and the right light bar G1R, and the combination of the rear microphone M1B and the rear light bar G1B.

[0035] As shown in FIG. 3, the left microphone M1L and left light bar G1L are arranged on the left side of a substantially rectangular parallelepiped housing so as to face to the left of the work machine 100. With this arrangement, the left microphone M1L can efficiently collect sounds generated to the left of the work machine 100, and the left light bar G1L can efficiently communicate the status of the work machine 100 to an operator on the left of the work machine 100. For example, the left microphone M1L can pick up the voice uttered by an operator on the left of the work machine 100, and the left light bar G1L can notify the operator that the left microphone M1L has picked up the operator's voice by emitting light in a predetermined color. In this case, an operator on the left of the work machine 100 who speaks into the left microphone M1L can confirm that their voice has reached the left microphone M1L (i.e., the operator of the work machine 100) by seeing the left light bar G1L emitting light in a predetermined color.

[0036] The information transmission device G1 may be provided at the upper part of each of the four side surfaces of the cab 10. For example, the information transmission device G1 may be configured so that a front light bar G1F is attached to the upper part of the front surface of the cab 10, a left light bar G1L is attached to the upper part of the left surface of the cab 10, a right light bar G1R is attached to the upper part of the right surface of the cab 10, and a rear light bar G1B is attached to the upper part of the rear surface of the cab 10. Furthermore, the information transmission device G1 may be a single rotating light such as a Nico Torch attached to the upper surface of the cab 10, or may be a display device such as a liquid crystal display or an organic EL display.

[0037] The controller 30 is an example of a control device, and is configured, for example, by a computer including a CPU, a volatile storage device, a non-volatile storage device, and various input / output interfaces. The controller 30 then realizes various functions, for example, by reading a program from the non-volatile storage device, loading it into the volatile storage device, and having the CPU execute the program. In the illustrated example, the controller 30 is configured to realize various functions to control the work machine 100. The various functions include, for example, a machine guidance function that guides the operator in manually operating the work machine 100. The various functions may also include a contact avoidance function that automatically or autonomously operates or stops the work machine 100 to avoid contact between the work machine 100 and an object that is present within a monitoring range around the work machine 100.

[0038] The boom angle sensor S1 detects the boom angle, which is the rotation angle of the boom 4 relative to the upper rotating body 3. The arm angle sensor S2 detects the arm angle, which is the rotation angle of the arm 5 relative to the boom 4. The bucket angle sensor S3 detects the bucket angle, which is the rotation angle of the bucket 6 relative to the arm 5.

[0039] Each of the boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 may be, for example, a rotary encoder, an acceleration sensor, a six-axis sensor, an IMU (Inertial Measurement Unit), a potentiometer using a variable resistor, a cylinder stroke sensor that detects the stroke amount of a hydraulic cylinder, etc.

[0040] The controller 30 receives a detection signal corresponding to the boom angle from the boom angle sensor S1, a detection signal corresponding to the arm angle from the arm angle sensor S2, and a detection signal corresponding to the bucket angle from the bucket angle sensor S3.

[0041] The machine body tilt sensor S4 detects the tilt state of the machine body (undercarriage 1 or upper rotating body 3) relative to the horizontal plane. The machine body tilt sensor S4 is attached to, for example, the upper rotating body 3, and detects the tilt angle of the work machine 100 (i.e., the upper rotating body 3) around two axes in the forward / backward and left / right directions. The machine body tilt sensor S4 may be, for example, an acceleration sensor, a six-axis sensor, an IMU, or the like. A detection signal corresponding to the tilt angle detected by the machine body tilt sensor S4 is input to the controller 30.

[0042] The rotation sensor S5 outputs information related to the rotation of the upper rotating body 3. The rotation sensor S5 detects, for example, the rotation angular velocity of the upper rotating body 3 relative to the lower traveling body 1. The rotation sensor S5 may also detect the rotation angle. The rotation sensor S5 may be, for example, a gyro sensor, a resolver, or a rotary encoder. A detection signal corresponding to the rotation angle or rotation angular velocity of the upper rotating body 3 detected by the rotation sensor S5 is input to the controller 30.

[0043] The positioning device PS measures the position of the upper rotating body 3. The positioning device PS is, for example, a Global Navigation Satellite System (GNSS) compass, and detects the position and orientation of the upper rotating body 3. A detection signal corresponding to the position and orientation of the upper rotating body 3 is input to the controller 30. The function of detecting the orientation of the upper rotating body 3 may be realized by a direction sensor attached to the upper rotating body 3.

[0044] The cab 10 is a cab in which an operator sits, and is mounted on the front left side of the upper rotating body 3. However, when the work machine 100 is remotely controlled or when the work machine 100 operates by fully automatic driving, the cab 10 may be omitted.

[0045] The communication device T1 communicates with external devices through a communication network including a mobile communication network, a satellite communication network, the Internet, etc. The communication device T1 is, for example, a mobile communication module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), or a satellite communication module for connecting to a satellite communication network.

[0046] In response to operations by an operator seated in the cab 10, the work machine 100 operates actuators to drive driven parts such as the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6.

[0047] Alternatively, the work machine 100 may be configured so that it can be remotely operated from outside the work machine 100. When the work machine 100 is remotely operated, the inside of the cab 10 may be unmanned.

[0048] Furthermore, the work machine 100 may automatically operate the actuators regardless of the operation performed by the operator. This allows the work machine 100 to realize a function of automatically operating at least some of the driven parts, such as the lower traveling structure 1, upper rotating structure 3, boom 4, arm 5, and bucket 6, i.e., a so-called "machine control function."

[0049] Figure 4 is a diagram that schematically shows an example of the configuration of a work machine 100. In Figure 4, the mechanical power transmission system, hydraulic oil lines, pilot lines, and electrical control system are indicated by double lines, thick solid lines, thick dashed lines, and dotted lines, respectively.

[0050] The drive system of the work machine 100 includes an engine 11, a regulator 13, a main pump 14, and a control valve unit 17. The hydraulic drive system of the work machine 100 also includes hydraulic actuators such as a swing hydraulic motor 2A, a left traveling hydraulic motor 2ML, a right traveling hydraulic motor 2MR, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9.

[0051] The engine 11 is an example of a power source for the work machine 100, and is mounted, for example, on the rear of the upper rotating body 3. The power source for the work machine 100 may be a combination of a power source such as a battery or a fuel cell and an electric motor. Specifically, the engine 11 rotates at a constant speed at a preset target rotation speed under direct or indirect control by the controller 30, and drives the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses diesel as fuel. The engine 11 may also be a gasoline engine, a hydrogen engine, or the like.

[0052] The regulator 13 controls the discharge amount of the main pump 14. For example, the regulator 13 controls the discharge amount of the main pump 14 by adjusting the angle (tilting angle) of the swash plate of the main pump 14 in response to a control command from the controller 30.

[0053] The main pump 14 is mounted on the rear of the upper rotating body 3, similar to the engine 11, and supplies hydraulic oil to the control valve unit 17 through a hydraulic oil line. In the illustrated example, the main pump 14 is a variable displacement hydraulic pump.

[0054] The control valve unit 17 is a hydraulic control device that controls the hydraulic system in the work machine 100. In the illustrated example, the control valve unit 17 includes control valves 171 to 176. The control valve unit 17 is configured to selectively supply hydraulic oil discharged by the main pump 14 to one or more hydraulic actuators via the control valves 171 to 176. The control valves 171 to 176 control the flow rate of hydraulic oil flowing from the main pump 14 to the hydraulic actuators, and the flow rate of hydraulic oil flowing from the hydraulic actuators to a hydraulic oil tank. The hydraulic actuators include a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, a left traveling hydraulic motor 2ML, a right traveling hydraulic motor 2MR, and a swing hydraulic motor 2A. Specifically, the control valve 171 corresponds to the left traveling hydraulic motor 2ML, the control valve 172 corresponds to the right traveling hydraulic motor 2MR, and the control valve 173 corresponds to the swing hydraulic motor 2A. Furthermore, the control valve 174 corresponds to the bucket cylinder 9 , the control valve 175 corresponds to the boom cylinder 7 , and the control valve 176 corresponds to the arm cylinder 8 .

[0055] The pilot pump 15 is an example of a pilot pressure generating device, and is configured to supply hydraulic oil to hydraulic control devices via a pilot line. In the illustrated example, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pressure generating device may be realized by the main pump 14. That is, the main pump 14 may have a function of supplying hydraulic oil to various hydraulic control devices via a pilot line, in addition to a function of supplying hydraulic oil to the control valve unit 17 via a hydraulic oil line. In this case, the pilot pump 15 may be omitted.

[0056] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In the example shown in the figure, the discharge pressure sensor 28 outputs the detected value to the controller 30.

[0057] The operating device 26 is a device used by an operator to operate the actuator. The operating device 26 includes, for example, an operating lever and an operating pedal. The actuator may be a hydraulic actuator or an electric actuator.

[0058] The operation sensor 29 is configured to detect the operation content of the operator using the operation device 26. In this embodiment, the operation sensor 29 detects the operation direction and operation amount of the operation device 26 corresponding to each actuator and outputs the detected value to the controller 30. In the illustrated example, the controller 30 can control the opening area of ​​the proportional valve 31 in accordance with the output of the operation sensor 29. The controller 30 then supplies the hydraulic oil discharged from the pilot pump 15 to the pilot ports of the corresponding control valves in the control valve unit 17. The pressure of the hydraulic oil supplied to each pilot port (pilot pressure) is, in principle, a pressure corresponding to the operation direction and operation amount of the operation device 26 corresponding to each hydraulic actuator. In this way, the operation device 26 is configured to supply the hydraulic oil discharged from the pilot pump 15 to the pilot ports of the corresponding control valves in the control valve unit 17.

[0059] Proportional valve 31, which functions as a control valve for machine control, is disposed in a pipe connecting pilot pump 15 and a pilot port of a control valve in control valve unit 17, and is configured so that the flow path area of ​​that pipe can be changed. In the illustrated example, proportional valve 31 operates in response to a control command output by controller 30. Therefore, controller 30 can adjust the pilot pressure acting on the pilot port of the control valve using proportional valve 31, regardless of the operation of operating device 26 by the operator.

[0060] With this configuration, the controller 30 can operate the hydraulic actuator corresponding to a specific operating device 26 even when the specific operating device 26 is not being operated.

[0061] The control system of the work machine 100 includes a controller 30, a display device D1, an input device D2, a talk button KS, an external sound collection device (an example of a sound collection device) M1, an internal sound collection device M2, an external sound output device SP1, an internal sound output device SP2, an external volume dial DL1, an internal volume dial DL2, a switch SW, and a communication device T1.

[0062] The controller 30 is configured to output a control command to the regulator 13 as necessary, thereby changing the discharge rate of the main pump 14.

[0063] Furthermore, the controller 30 may be configured to perform control relating to a machine guidance function that guides (provides guidance for) the manual operation of the work machine 100 by the operator via the operation device 26. Furthermore, the controller 30 may be configured to perform control relating to a machine control function that automatically assists the manual operation of the work machine 100 by the operator via the operation device 26.

[0064] Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers. For example, the machine guidance function and the machine control function may be realized by a dedicated controller (control device).

[0065] The interior of the cab 10 will now be described with reference to Figure 5. Figure 5 is a top view of the interior of the cab 10. The work machine 100 is equipped with a driver's seat 50, an operating device 26, a display device D1, and the like, which are arranged inside the cab 10. A door for getting on and off is provided on the left side of the driver's seat 50. The operator can enter the cab 10 by opening the door for getting on and off.

[0066] The driver's seat 50 is located in the center of the driver's cab 10 in a top view. The driver's seat 50 includes a seat portion 51 on which the operator sits and a backrest 52. The driver's seat 50 is a reclining seat, and the tilt angle of the backrest 52 is adjustable. A left armrest 53L is located on the left side of the driver's seat 50, and a right armrest 53R is located on the right side. The left armrest 53L and the right armrest 53R are rotatably supported by the backrest 52.

[0067] A left console 54L is disposed on the left side of the driver's seat 50, and a right console 54R is disposed on the right side. The left console 54L and the right console 54R extend in the front-to-rear direction. The driver's seat 50 is slidable in the front-to-rear direction. The driver's seat 50 may be configured to be slidable in the front-to-rear direction together with the left console 54L and the right console 54R.

[0068] The left armrest 53L is disposed on the left console 54L. The right armrest 53R is disposed on the right console 54R. The left armrest 53L is disposed so as to cover a portion of the left console 54L in a top view. The right armrest 53R is disposed so as to cover a portion of the right console 54R in a top view.

[0069] The operating device 26 includes a left operating lever 26L, a right operating lever 26R, a left travel pedal 26PL, a right travel pedal 26PR, a left travel lever 26DL, and a right travel lever 26DR.

[0070] The left operation lever 26L is provided in front of the left console 54L. Similarly, the right operation lever 26R is provided in front of the right console 54R. An operator seated in the driver's seat 50 can operate the left operation lever 26L while holding the left operation lever 26L with his left hand, and can operate the right operation lever 26R while holding the right operation lever 26R with his right hand. An operator seated in the driver's seat 50 can operate the left operation lever 26L with his left hand to drive the arm cylinder 8 and the swing hydraulic motor 2A. In addition, an operator seated in the driver's seat 50 can operate the right operation lever 26R with his right hand to drive the boom cylinder 7 and the bucket cylinder 9. The bases of the left operation lever 26L and the right operation lever 26R are covered with lever boots 27.

[0071] The left travel pedal 26PL and the right travel pedal 26PR are located on the floor in front of the driver's seat 50. An operator seated in the driver's seat 50 can operate the left travel pedal 26PL with his left foot to drive the left travel hydraulic motor 2ML. Also, an operator seated in the driver's seat 50 can operate the right travel pedal 26PR with his right foot to drive the right travel hydraulic motor 2MR.

[0072] The left travel lever 26DL and the right travel lever 26DR are located between the left travel pedal 26PL and the right travel pedal 26PR in a top view. The left travel lever 26DL and the right travel lever 26DR extend upward from the floor in front of the driver's seat 50. An operator seated in the driver's seat 50 can drive the left travel hydraulic motor 2ML by gripping and operating the left travel lever 26DL with their left hand, similar to operating the left travel pedal 26PL. Furthermore, an operator seated in the driver's seat 50 can drive the right travel hydraulic motor 2MR by gripping and operating the right travel lever 26DR with their right hand, similar to operating the right travel pedal 26PR. Furthermore, the left travel lever 26DL and the right travel lever 26DR are positioned so that the operator can operate the left travel lever 26DL and the right travel lever 26DR simultaneously with one hand.

[0073] The display device D1 is located on the front right side of the driver's seat 50. The display device D1 displays various types of image information. The display device D1 includes a display screen that displays information such as the work conditions or operating state of the work machine 100. An operator seated in the driver's seat 50 can perform work using the work machine 100 while checking the various types of information displayed on the display device D1. The display device D1 may be provided with an input device D2.

[0074] The input device D2 is provided within reach of the operator seated in the cab 10, accepts various operational inputs from the operator, and outputs signals corresponding to the operational inputs to the controller 30. The input device D2 includes a touch panel mounted on the display of the display device D1 that displays various information images, a knob switch provided at the tip of one or more levers of a plurality of operating levers included in the operating device 26, or a button switch, lever, toggle switch, rotary dial, or the like provided around the display device D1. A signal corresponding to the content of an operation on the input device D2 is taken into the controller 30.

[0075] A gate bar 55 is attached to the front surface of the front end of the left console 54L. The gate bar 55 operates in conjunction with the operation of a gate lock lever GL provided on the left console 54L. The gate bar 55 is attached to a frame inside the left console 54L so that it can be raised and lowered around a left-right axis at its upper end.

[0076] The gate lock lever GL is a mechanical input operation unit for switching between a state in which the work machine 100 can be operated by the operation device 26 (operable state) and a state in which the work machine 100 cannot be operated by the operation device 26 (inoperable state). In the illustrated example, the gate lock lever GL is configured so that the operator can switch between a first operation position that realizes the inoperable state and a second operation position that realizes the operable state. The controller 30 switches between the operable state and the inoperable state depending on the operation state of the gate lock lever GL. In the illustrated example, the controller 30 switches between the operable state and the inoperable state of the work machine 100 by electrically switching between connected and disconnected states of the pilot line depending on the operation state of the gate lock lever GL.

[0077] When the gate lock lever GL is in the second operating position, the gate bar 55 is raised forward (passing prohibited state) so as to prevent the operator from passing through the boarding / exiting door, as shown in Fig. 5. On the other hand, when the gate lock lever GL is in the first operating position, the gate bar 55 is housed inside the left console 54L (passing permitted state) so as not to prevent the operator from passing through the boarding / exiting door.

[0078] With this configuration, the operator cannot operate the work machine 100 unless he or she sets the gate lock lever GL to the second operating position and puts the gate bar 55 in a passage-prohibiting state. Therefore, this configuration can prevent the work machine 100 from moving unintentionally, even if the operator inadvertently touches the operating device 26 when getting on or off the machine. Therefore, this configuration can improve the safety of the work machine 100.

[0079] Furthermore, the work machine 100 may be configured so that it can accept a predetermined operation to start the engine 11 only when the gate lock lever GL is in the second operating position and the gate bar 55 is in a pass-prohibiting state. In other words, the work machine 100 may be configured so that it cannot start the engine 11 when the gate lock lever GL is in the first operating position and the gate bar 55 is in a pass-permitting state.

[0080] A switch SW is installed on the right console 54R. A window console 56 is installed on the right side of the right console 54R. The window console 56 extends over the entire length of the driver's cab 10 in the fore-and-aft direction and is installed parallel to the right console 54R. A display device D1 is installed in front of the window console 56. An external volume dial DL1, an internal volume dial DL2, an internal sound collection device M2, etc. are installed on the window console 56.

[0081] Furthermore, a radio tuner 57 and the like are installed on the window console 56. The radio tuner 57 and the like may be installed on the console 54L or the console 54R.

[0082] The radio tuner 57 is provided with a switch. Pressing and rotating the switch allows the radio to be turned on and off and the volume to be adjusted. Furthermore, the operation for controlling the radio tuner 57 is not limited to operating a physically provided switch, but may also be an operation using a touch panel (of the input device D2) provided on the display device D1. For example, the controller 30 may be able to control the radio tuner 57 to be on and off and adjust the volume by accepting an operation on the touch panel. The volume of the radio tuner 57 may be adjusted independently of or integrally with the volume of the external audio (the operator's voice output into the vehicle).

[0083] The internal sound collection device M2 is a device that collects sounds generated inside the cab 10. In the illustrated example, the internal sound collection device M2 is an indoor microphone that is configured to be able to pick up the voice of an operator inside the cab 10.

[0084] The horn button HS is a button that is operated by the operator of the work machine 100 to sound the horn. In the illustrated example, the horn button HS is a knob switch provided at the tip of the left operation lever 26L.

[0085] The speech button KS is a button that the operator of the work machine 100 operates when he or she wants to speak to a worker around the work machine 100. In the illustrated example, the speech button KS is a knob switch provided at the tip of the right operating lever 26R.

[0086] The internal sound output device SP2 is a device that outputs sound to an operator in the cab 10 and is provided in the cab 10. The internal sound output device SP2 converts an electrical signal input from the controller 30 into a physical sound (air vibration) and outputs the sound. The internal sound output device SP2 may be provided in any position, for example, near the display device D1, near the input device D2, or near the door of the cab 10. In the illustrated example, the internal sound output device SP2 includes a left interior speaker SP2L attached to the upper left corner of the rear wall of the cab 10 and a right interior speaker SP2R attached to the upper right corner of the rear wall of the cab 10. The internal sound output device SP2 may be headphones or earphones worn by the operator. In this case, the headphones or earphones are connected to the controller 30 so as to be able to communicate with the controller 30 via, for example, Bluetooth (registered trademark) or the like.

[0087] The external volume dial DL1 is configured to adjust the volume of the sound output by the external sound output device SP1. The volume of the sound output by each external sound output device SP1 may be additionally configured to be adjustable using a device other than the external volume dial DL1, such as a touch panel attached to the display device D1.

[0088] The external volume dial DL1 may be configured to be infinitely rotatable in both the clockwise and counterclockwise directions, in order to accommodate a case where volume adjustment using the external volume dial DL1 and volume adjustment using a device other than the external volume dial DL1 are used in combination.

[0089] The internal volume dial DL2 is configured to adjust the volume of the sound output by the internal sound output device SP2. The volume of the sound output by each internal sound output device SP2 may be additionally configured to be adjustable using a device other than the internal volume dial DL2, such as a touch panel attached to the display device D1.

[0090] The internal volume dial DL2 may be configured to be infinitely rotatable in both the clockwise and counterclockwise directions, in order to accommodate a case where volume adjustment using the internal volume dial DL2 and volume adjustment using a device other than the internal volume dial DL2 are used in combination.

[0091] The storage device 35 is provided, for example, in the cab 10, and stores various types of information under the control of the controller 30. The storage device 35 is, for example, a non-volatile storage medium such as a semiconductor memory. The storage device 35 stores information for suppressing sound output from the internal sound output device SP2. The storage device 35 may store data relating to a target construction plane that is acquired via the communication device T1 or the like, or that is set via the input device D2 or the like. The target construction plane may be set (saved) by the operator of the work machine 100, or may be set by a construction manager or the like.

[0092] The switch SW is a switch for switching whether or not to operate a sound output function that outputs sound collected by the external sound collection device M1 from the internal sound output device SP2. In the illustrated example, the switch SW is provided on the top surface of the right console 54R. However, the switch SW may be one of the input devices D2, may be realized by a touch panel provided on the display device D1, or may be a knob switch.

[0093] The conversation function is a function for enabling a conversation between an operator OP of the work machine 100 and a worker WK located around the work machine 100, as shown in FIG. 6. FIG. 6 is a perspective view of the work machine 100 on which the operator OP is riding and the worker WK located at the front left of the work machine 100. FIG. 6 shows the operator OP's voice being collected by the internal sound collection device M2 and output from the external sound output device SP1, and the worker WK's voice being collected by the external sound collection device M1 and output from the internal sound output device SP2. In the work machine 100 shown in FIG. 6, an information transmission device G1 is provided at the top of each of the four side surfaces of the cab 10. The front light bar G1F provided at the top of the front of the cab 10 emits green light, and the left light bar G1L provided at the top of the left side of the cab 10 emits white light. In FIG. 6, the front light bar G1F emitting green light has a dot pattern attached. When the worker WK sees the front light bar G1F emitting green light, he or she can recognize that his or her voice is being detected by the front microphone M1F. Note that, for clarity, other devices such as the imaging device S6 are not shown in Figure 6.

[0094] The operating state of the conversation function includes an ON state (the state shown in FIG. 6) in which conversation between the operator OP and the worker WK is possible, and an OFF state in which conversation between the operator OP and the worker WK is not possible. However, the operating state of the conversation function may additionally include at least one of an audible state (with respect to the operator OP) in which the operator OP can hear the worker WK's voice but the worker WK cannot hear the operator OP's voice, and an utterance enabled state (with respect to the operator OP) in which the worker WK can hear the operator OP's voice but the operator OP cannot hear the worker WK's voice.

[0095] Specifically, when the switch SW is operated to switch the operation state of the conversation function to the ON state, the external sound collection device M1, the external sound output device SP1, the internal sound collection device M2, and the internal sound output device SP2 become available. Conversely, when the switch SW is operated to switch the operation state of the conversation function to the OFF state, the external sound collection device M1, the external sound output device SP1, the internal sound collection device M2, and the internal sound output device SP2 become unavailable. Furthermore, when the switch SW is operated to switch the operation state of the conversation function to the audible state, the external sound collection device M1 and the internal sound output device SP2 become available. Furthermore, when the switch SW is operated to switch the operation state of the conversation function to the speech-enabled state, the external sound output device SP1 and the internal sound collection device M2 become available. In the illustrated example, the operator OP can use the external sound output device SP1 to talk to the worker WK by pressing the speech button KS and speaking when the internal sound collection device M2 is available.

[0096] By controlling the controller 30 according to this embodiment, the internal sound output device SP2 provided inside the cab 10 outputs the sound collected by the external sound collection device M1, and the external sound output device SP1 provided outside the cab 10 outputs the sound collected by the internal sound collection device M2. The controller 30 can switch between the sound output from the internal sound output device SP2 and the sound output from the external sound output device SP1 in response to an operation from the operator OP, thereby realizing two-way conversation. The controller 30 according to this embodiment controls the sound output in response to an operation, thereby preventing simultaneous output of sound from the internal sound output device SP2 and the external sound output device SP1, thereby preventing the occurrence of feedback and the like. Note that this embodiment is not limited to the method of performing this switching control, and sound may be output simultaneously from the external sound output device SP1 and the internal sound output device SP2.

[0097] When the controller 30 simultaneously outputs the sound collected by the external sound collection device M1 from the internal sound output device SP2 installed inside the cab 10 and outputs the sound collected by the internal sound collection device M2 from the external sound output device SP1 installed outside the cab 10, the sound output from the sound output devices SP1 and SP2 may be collected by the sound collection devices M2 and M1, resulting in an echo (reverberation) similar to an "echo." Therefore, the controller 30 may execute an echo cancellation function on the sound represented by the sound signal acquired from the external sound collection device M1 or the internal sound collection device M2. The echo cancellation function may be any function capable of eliminating echoes. For example, an echo suppressor method that prevents one person's voice from being picked up while the other person's voice is being heard may be used, or an echo canceller method that removes echoes (reverberation) collected by the sound collection devices M2 and M1 may be used. Furthermore, the echo cancellation function may be any method, not just the well-known method described above.

[0098] [Controller function configuration] The following describes a configuration for the controller 30 to perform control based on a detected person. The controller 30 includes an acquisition unit 301, a detection unit 302, a position estimation unit 303, a sound processing unit 304, an identification unit 305, a determination unit 306, and an output control unit 307 as machine guidance and machine control functions.

[0099] The acquisition unit 301 acquires detection results from various sensors provided on the work machine 100. For example, the acquisition unit 301 acquires image information indicating the imaging results from the imaging device S6. The acquisition unit 301 also acquires sound signals representing sounds generated around the work machine 100 from the external sound collection device M1.

[0100] The detection unit 302 performs processing to detect people present in the vicinity of the work machine 100 from the image information acquired by the acquisition unit 301. Any method may be used for the person detection processing, regardless of whether it is a well-known method. For example, it may be determined whether or not the feature amount extracted from the image information is similar to the feature amount indicating a person by a predetermined value or more.

[0101] When a person is detected by the detection unit 302, the position estimation unit 303 estimates the position of the person in real space, for example, the direction and distance of the person relative to the work machine 100. For example, the position estimation unit 303 may estimate the direction and distance of the person based on the position and size of the person in the image information. Furthermore, if a LiDAR, a range imaging camera, or a millimeter-wave radar is installed as an object detection device in addition to the imaging device S6, the direction and distance of the person may be estimated from the detection results of the object detection device. An example will be described in which the position estimation unit 303 according to the present embodiment acquires the direction and distance of the person relative to the work machine 100 as position information of a person present in the vicinity of the work machine 100, but the position information of the person is not limited to the direction and distance of the person relative to the work machine 100, and for example, information indicating the position coordinates of the person in a world coordinate system may be acquired.

[0102] In this embodiment, the detection of a person and the estimation of the person's position are not limited to the above-mentioned method, and a trained model may be used to detect a person and estimate the person's position (e.g., direction and distance).

[0103] The sound processing unit 304 performs processing to emphasize the human voice band for the sound signals acquired by the acquisition unit 301, and performs processing to reduce bands other than the human voice band. These processing operations may be performed using well-known methods, such as noise canceling processing, or suppressing bands other than the human voice band using a bandpass filter or the like. Specifically, as noise canceling processing, the sound processing unit 304 performs processing to suppress sounds generated from the work machine 100, such as the engine 11, based on sound signals from multiple external sound collection devices M1. Because the sounds generated from the work machine 100 are suppressed, the operator can more easily hear sounds generated from the environment around the work machine 100. Because it becomes easier for the operator to grasp the situation around the work machine 100, improved safety can be achieved.

[0104] The identification unit 305 identifies whether or not a person has spoken, based on the sound signal processed by the sound processing unit 304. For example, the identification unit 305 determines whether or not the sound signal from which the human voice band has been extracted has a volume equal to or greater than a predetermined threshold, and identifies whether or not a person has spoken, based on the determination result.

[0105] Furthermore, when the identification unit 305 identifies the presence of a person who has spoken, it identifies the speaking direction based on the volume of each of the sound signals from the multiple external sound collection devices M1. The speaking direction is, for example, the direction in which the person who spoke is present. For example, the identification unit 305 identifies, as the speaking direction, the direction in which the external sound collection device M1 that detected the loudest sound among the four directions (front, back, left, right), is located. Note that this embodiment illustrates an example of a method for identifying a direction, and is not limited to this method. For example, there is also a method for identifying the speaking direction based on the phase shift and volume difference in the sound signals from the multiple external sound collection devices M1.

[0106] The determining unit 306 determines whether or not the speaking direction identified by the identifying unit 305 and the direction in which the person is present estimated by the position estimating unit 303 match.

[0107] The output control unit 307 outputs information to one or more of the display device D1 and the internal sound output device SP2. For example, when a person present in the vicinity of the work machine 100 is speaking, the output control unit 307 is configured to notify the direction in which the sound of the person's speech was detected, based on the direction of the sound detected by the external sound collection device M1. This notification is made when the direction of the person detected from the image information corresponds to the direction of the speech.

[0108] For example, when the determination unit 306 determines that the directions match, the output control unit 307 notifies the detected direction, and when the determination unit 306 determines that the directions do not match (are different), the output control unit 307 suppresses the notification of the detected direction.

[0109] Next, an example of the display screen 42 that the output control unit 307 outputs to the display device D1 will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the display screen 42 that is displayed by the display device D1 according to this embodiment.

[0110] The output control unit 307 according to this embodiment generates a display screen 42 based on image information input from the imaging device S6, sound signals input from the external sound collection device M1, and various types of information detected by various sensors and the like provided on the work machine 100. The various types of information detected by the various sensors and the like include the turning angle, information indicating the area in which a person is captured in the image information, position information of people present in the vicinity of the work machine 100 (information indicating the direction and distance of the detected person), the person who spoke based on the sound signals input from the external sound collection device M1, and the detection results of the various sensors.

[0111] The display screen 42, under control of the output control unit 307, includes a date and time display area 42a, a driving mode display area 42b, an attachment display area 42c, a fuel efficiency display area 42d, an engine control status display area 42e, a coolant temperature display area 42g, a remaining fuel amount display area 42h, an RPM level display area 42i, a urea water remaining amount display area 42j, a hydraulic oil temperature display area 42k, a status display area 1421, a first image display area 1422, a second image display area 1423, and a third image display area 1424. The display screen 42 may include other display areas.

[0112] The travel mode display area 42b, the attachment display area 42c, the engine control state display area 42e, and the rotation speed level display area 42i are areas that display setting state information, which is information related to the setting state of the work machine 100. The fuel efficiency display area 42d, the engine operating time display area, the coolant temperature display area 42g, the remaining fuel amount display area 42h, the remaining urea water amount display area 42j, and the hydraulic oil temperature display area 42k are areas that display operating state information, which is information that represents the operating state of the work machine 100 based on the detection results of various sensors.

[0113] The date and time display area 42a is an area that displays the current date and time. The driving mode display area 42b is an area that displays the current driving mode. The attachment display area 42c is an area that displays an image representing the currently attached attachment. The fuel efficiency display area 42d is an area that displays fuel efficiency information calculated by the controller 30. The fuel efficiency display area 42d includes an average fuel efficiency display area 42d1 that displays lifetime average fuel efficiency or section average fuel efficiency, and an instantaneous fuel efficiency display area 42d2 that displays instantaneous fuel efficiency.

[0114] The engine control status display area 42e is an area that displays the control status of the engine 11. The engine operating time display area is an area that displays the accumulated operating time of the engine 11. The coolant temperature display area 42g is an area that displays the current temperature state of the engine coolant. The remaining fuel amount display area 42h is an area that displays the remaining amount of fuel stored in the fuel tank.

[0115] The rotation speed level display area 42i is an area that displays the current level set by a dial (not shown) as an image. A number indicating the selected level is displayed in the rotation speed level display area 42i. The number "1" displayed in the rotation speed level display area 42i indicates that the selected rotation speed level is the "first level." The number "n" displayed in the rotation speed level display area 42i indicates that the selected rotation speed level is the "nth level." "n" is a natural number. When the operator rotates the dial (not shown), the number displayed in the rotation speed level display area 42i changes.

[0116] The urea water remaining amount display area 42j is an area that displays an image of the remaining amount of urea water stored in the urea water tank. The hydraulic oil temperature display area 42k is an area that displays the temperature of the hydraulic oil in the hydraulic oil tank.

[0117] 7, the first image display area 1422, the second image display area 1423, and the third image display area 1424 are areas that display image information captured by the imaging device S6. The first image display area 1422 displays a right image. The second image display area 1423 displays a rearward image. The third image display area 1424 displays a left image. The right image is an image that displays the space to the right of the work machine 100, the rearward image is an image that displays the space behind the work machine 100, and the left image is an image that displays the space to the left of the work machine 100.

[0118] The first image display area 1422 is displayed to the right with the status display area 1421 as the reference. The second image display area 1423 is displayed below with the status display area 1421 as the reference. The third image display area 1424 is displayed to the left with the status display area 1421 as the reference. The bottom of the display screen 42 corresponds to the rear of the work machine 100. In other words, the display screen 42 displays image information captured by the imaging device S6 in the direction in which the image was captured with the status display area 1421 as the reference. Note that this embodiment shows an example of the arrangement of image information, and is not limited to this arrangement. For example, the image display area may be arranged regardless of the direction in which the image was captured. In this embodiment, image information captured in the direction in which the image was captured is displayed with the status display area 1421 as the reference. Therefore, when the operator refers to the image information, the operator can intuitively recognize in which direction the situation in the image information is represented. Therefore, improved safety can be achieved.

[0119] When the position and distance of a person are identified by the position estimation unit 303, the output control unit 307 superimposes a frame indicating that a person has been detected in an area identified by the identified position and distance on one or more of the right image, rear image, and left image.

[0120] As a result, a frame 1422b is displayed in the right image of the first image display area 1422 so as to surround the person 1422a, and a frame 1424b is displayed in the left image of the third image display area 1424 so as to surround the person 1424a.

[0121] The status display area 1421 is an area that displays information that indicates the positional relationship between the work machine 100 and a person detected in the vicinity of the work machine 100.

[0122] The status display area 1421 is a display area that displays, at a predetermined scale, real space with the work machine 100 at its center. In the status display area 1421, a work machine icon (an example of display information) 1421b indicating the presence of the work machine 100 is placed in the center of the area.

[0123] In addition to a work machine icon 1421b representing the work machine 100, the status display area 1421 simultaneously displays a direction display icon 1421a indicating the direction of movement when the work machine 100 moves forward, and icons representing people detected in the vicinity of the work machine 100 (person detection icons 1421e, 1421f, 1421g in the example of FIG. 7). The area of ​​the status display area 1421 other than the work machine icon 1421b, direction display icon 1421a, person detection icons 1421e, 1421f, 1421g, and area 1421h (in other words, the background) may be an area displayed in a single color (for example, black).

[0124] The work machine icon 1421b is an icon that combines an image showing the upper rotating body 3 and an image showing the lower traveling body 1 in accordance with the positional relationship between the upper rotating body 3 and the lower traveling body 1 based on the rotation angle.

[0125] The direction display icon 1421a is triangular in shape and indicates the direction in which the work machine 100 will travel when the travel lever is tilted forward. Note that this embodiment shows only one example of an icon that indicates the direction in which the work machine 100 will travel when the travel lever is tilted forward, and any shape may be used as long as it indicates the direction in which the work machine 100 can travel.

[0126] The person detection icons 1421e, 1421f, and 1421g are icons that represent people detected from image information captured by the imaging device S6. Specifically, the person detection icons 1421e, 1421f, and 1421g are placed based on the results of estimation of the position and direction of people by the position estimation unit 303. For example, the person detection icons 1421e, 1421f, and 1421g are placed at positions obtained by multiplying the direction and distance to the presence of people, which are indicated by the estimation results, by a predetermined scale, with the work machine 100 as the reference.

[0127] In this way, the positional relationship between the work machine icon 1421b and the person detection icons 1421e, 1421f, 1421g corresponds to the positional relationship between the work machine 100 and the people present around the work machine 100 in real space.

[0128] Furthermore, by referring to the status display area 1421, the operator can infer how the positional relationship between the work machine 100 and people present in the vicinity will change when the work machine 100 is moved.

[0129] Furthermore, in the status display area 1421, a first circular area 1421c and a second circular area 1421d are displayed, which are determined according to the distance from the work machine 100, with the work machine 100 as the reference point.

[0130] The first circular area 1421c is information indicating the range in which the attachment AT of the work machine 100 currently swings, based on the detection results of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3.

[0131] The second circular area 1421d is information indicating the range in which the attachment AT can turn when the attachment AT is extended to its maximum extent in the horizontal direction.

[0132] The display screen 42 according to this embodiment displays a first circular area 1421c and a second circular area 1421d. That is, the positional relationship between the person and the attachment AT can be recognized based on the positional relationship between the first circular area 1421c and the second circular area 1421d and the person detection icons 1421e, 1421f, and 1421g. This makes it easier for the operator to operate the attachment AT in a way that prevents the person from coming into contact with it, thereby improving safety.

[0133] Furthermore, the output control unit 307 changes the display mode of the person detection icons 1421e, 1421f, and 1421g depending on whether they are included in the first circular area 1421c and the second circular area 1421d.

[0134] For example, a person detection icon 1421e that exists within the first circular area 1421c is displayed in red, for example. A person detection icon 1421f that exists outside the first circular area 1421c but within the second circular area 1421d is displayed in yellow, for example. A person detection icon 1421g that exists outside the second circular area 1421d is displayed in blue, for example. The output control unit 307 according to this embodiment changes the display mode depending on whether the person is included in the first circular area 1421c and the second circular area 1421d; in other words, it changes the color of the person detection icon depending on the distance from the work machine 100. In this way, the display device D1 can alert the operator to the distance between the work machine 100 and a person by displaying a person detection icon whose color changes depending on the distance. Therefore, improved safety can be achieved.

[0135] By referring to the status display area 1421, the operator can recognize how the positional relationship between the attachment AT of the work machine 100 and people present in the vicinity changes when the work machine 100 is turned, thereby achieving improved safety.

[0136] In addition, the output control unit 307 matches the color of the frame 1422b of the right image in the first image display area 1422 with the color of the person detection icon 1421e, and also matches the color of the frame 1424b of the rear image in the third image display area 1424 with the color of the person detection icon 1421f.

[0137] That is, on the display screen 42, frames showing detected people are displayed in the right and left images, and the correspondence between the people shown in the frames and the person detection icons is displayed so that the operator can recognize the status of the people shown in the status display area 1421 by referring to the right and left images.

[0138] Furthermore, the output control unit 307 displays an area 1421h in the status display area 1421 that indicates the direction of speech identified by the identification unit 305. The area 1421h in the status display area 1421 is displayed in a color different from the background (other areas) of the status display area 1421. That is, the output control unit 307 according to this embodiment displays an area (an example of information) 1421h that indicates the direction in which a human speech sound was detected on the display device D1 in a different display mode from the other areas (examples of information) using the work machine icon 1421b as a reference. This allows the operator to recognize the direction of speech by referring to the status display area 1421. Then, the operator can infer who the person speaking is from that direction. For example, the operator can infer that the person corresponding to the person detection icon 1421f included in the area 1421h is speaking to the operator. Therefore, the operator can infer who the person speaking to is, which makes it easier for the operator to communicate with the person speaking to him / her. This reduces the burden on the operator during work.

[0139] Furthermore, there may be a case where the speaking direction identified by the identification unit 305 and the direction in which the person is present identified by the position estimation unit 303 are slightly different.

[0140] 8A and 8B are diagrams illustrating an example of speech direction correction in the output control unit 307 according to this embodiment. FIG. 8A shows an example in which the identification unit 305 detects a sound caused by a person speaking from the left direction among four directions. In this case, the output control unit 307 displays an area 1502 corresponding to the left direction in a color different from the other areas. However, in FIG. 8, the position estimation unit 303 estimates that a person is present in the direction and at the distance indicated by the person detection icon 1501.

[0141] The speaking direction identified by the identification unit 305 may deviate depending on the surrounding environment. The output control unit 307 according to this embodiment corrects the area indicating the speaking direction according to the actual position of the person.

[0142] The output control unit 307 corrects the area indicating the speaking direction based on the direction of the person detected from the image information identified by the position estimation unit 303. Any criterion may be used for the correction. For example, even if the area indicating the direction identified by the identification unit 305 does not include the direction of the person estimated by the position estimation unit 303, if the area is closer than a predetermined criterion (for example, within 10 degrees), the output control unit 307 corrects the area so that the direction of the person estimated by the position estimation unit 303 is included.

[0143] FIG. 8(B) shows the status display area 1421 after the output control unit 307 corrects the area. The area 1503 shown in FIG. 8(B) has been corrected to include the person detection icon 1421f. Therefore, by referring to the status display area 1421 in FIG. 8(B), the operator can recognize the person speaking and communicate with that person. In this embodiment, even if the speaking direction identified by the external sound collection device M1 is misaligned, the output control unit 307 corrects the speaking direction based on the direction of the person detected from the image information, so that the speaking direction can be presented to the operator without causing discomfort. Therefore, when operation of the work machine 100 is required based on the content of the speech, the operator can understand which direction the content of the speech is based on and in which direction the work machine 100 should be operated, thereby achieving improved convenience.

[0144] FIG. 7 shows an example of the display screen 42 displayed by the output control unit 307, and the speaking direction may be displayed in other ways.

[0145] 9 is a diagram showing an example of a display screen 42A displayed by the display device D1 according to this embodiment. Note that the same displays as those in FIG. 7 are assigned the same reference numerals, and descriptions thereof will be omitted.

[0146] Of the screens shown in FIG. 9, a first image display area 1422, a second image display area 1423, and a third image display area 1424 are areas for displaying image information captured by the imaging device S6.

[0147] Furthermore, the output control unit 307 displays information indicating the speaking direction identified by the identification unit 305 in the outer frame of one or more of the first image display area 1422, the second image display area 1423, and the third image display area 1424.

[0148] 9, the outer frame 1424FL of the third image display region 1424 is rendered thicker than the outer frame 1422FL of the first image display region 1422 and the outer frame 1423FL of the second image display region 1423. Furthermore, the outer frame 1424FL is rendered in a different color than the outer frames 1422FL and 1422FL.

[0149] The operator can recognize the direction of speech by referring to the first image display area 1422, the second image display area 1423, and the third image display area 1424. In other words, because the display mode of the outer frame 1424FL is different from that of the other outer frames, the operator can infer that the person appearing in the third image display area 1424 surrounded by the outer frame 1424FL has spoken. Therefore, the operator can infer who has spoken, making it easier to communicate with the person who has spoken.

[0150] If the speaking direction is forward, the person who spoke will not be shown in any of the first image display area 1422, second image display area 1423, and third image display area 1424. In this case, the output control unit 307 can display a message on the display device D1 indicating that a voice is being heard from the front and that the screen should be switched to one that shows the front. Another method is for the output control unit 307 to automatically switch to a display screen that includes an image of the space in front of the work machine 100, captured by the camera S6F.

[0151] 9, the output control unit 307 displays image information captured by the imaging device S6 on the display device D1, and displays the image information captured in a direction from which a person's speech sound was detected in a different manner. Therefore, the operator can recognize the direction of the speech and the current situation in that direction. Therefore, the operator can easily guess who is speaking and communicate with that person.

[0152] 7 to 9 show one mode of display showing the periphery of the work machine 100, and the periphery of the work machine 100 may be displayed in other modes. As a method of displaying the periphery of the work machine 100, an overhead image may be used.

[0153] 10 is a diagram showing an example of a display screen 42B displayed by the display device D1 according to this embodiment. Note that the same displays as those in FIG. 7 are assigned the same reference numerals, and descriptions thereof will be omitted.

[0154] 10, overhead image 1700 is image information that shows the area around the work machine 100 from above the work machine 100, with a work machine icon 1701 as the reference. The overhead image 1700 is image information generated based on image information from each of the four cameras S6F, S6B, S6L, and S6R. The output control unit 307 according to this embodiment synthesizes image information captured in front, image information captured in rear, image information captured to the right, and image information captured to the left, and then converts this into an image seen from directly above the work machine 100 to generate overhead image 1700.

[0155] 10, the output control unit 307 displays a thick line 1704 indicating the area where the person who spoke is located, superimposed on an outer frame 1702 of the overhead image 1700. The thick line 1704 may be a different color from the outer frame 1702 of the overhead image 1700.

[0156] By referring to the overhead image 1700, the operator can recognize that the direction of the speech is to the left, indicated by the thick line 1704, with the work machine 100 as the reference point. Therefore, the operator can infer that the person 1703 shown in the overhead image 1700 is speaking to him.

[0157] Therefore, the controller 30 according to the present embodiment allows the operator to guess who is speaking, making it easier to communicate with the person who is speaking to them, thereby reducing the burden on the operator during work.

[0158] 10 shows an example in which only overhead image 1700 is displayed, but the present invention is not limited to a mode in which only overhead image 1700 is displayed, and overhead image 1700 may be displayed in combination with one or more of first image display area 1422 of the right image, second image display area 1423 of the rear image, and third image display area 1424 of the left image, as shown in Figures 8 and 9. For example, there is a method in which output control unit 307 outputs a display screen that combines overhead image 1700, first image display area 1422, and second image display area 1423.

[0159] Next, a description will be given of the processing procedure executed by the controller 30 according to this embodiment. Fig. 11 is a flowchart showing the processing procedure for the controller 30 according to this embodiment to display the speaking direction.

[0160] First, the acquisition unit 301 acquires image information of the surroundings of the work machine 100 from the imaging device S6 (S1801).

[0161] The detection unit 302 detects people present in the vicinity of the work machine 100 from the image information acquired by the acquisition unit 301 (S1802).

[0162] When a person is detected by the detection unit 302, the position estimation unit 303 estimates the direction and distance of the person relative to the work machine 100 (S1803).

[0163] Next, the acquisition unit 301 acquires, from each of the multiple external sound collection devices M1, a sound signal representing a sound generated around the work machine 100 (S1804).

[0164] The sound processing unit 304 performs an emphasis process on the human voice band for each sound signal of the external sound collection device M1 acquired by the acquisition unit 301, and performs a reduction process (noise removal process) on bands other than the human voice band (S1805).

[0165] The identifying unit 305 identifies the speaking direction based on the sound signal processed by the sound processing unit 304 (S1806).

[0166] The determining unit 306 determines whether or not the speaking direction identified by the identifying unit 305 corresponds to the direction in which the person is present estimated by the position estimating unit 303 (S1807).

[0167] If the determining unit 306 determines that they do not match (S1807: NO), the display of the speaking direction is suppressed, and the process proceeds to S1809.

[0168] On the other hand, if the determination unit 306 determines that there is a match (S1807: YES), the output control unit 307 displays the speaking direction (S1808). For example, the output control unit 307 displays the speaking direction in one of the display screen 42 in Fig. 7, the display screen 42A in Fig. 9, and the display screen 42B in Fig. 10.

[0169] Furthermore, the output control unit 307 outputs the sound signal processed by the sound processing unit 304 (S1809).

[0170] In this embodiment, by performing the above-described processing procedure, the operator can recognize the direction of speech in the vicinity of the work machine 100. By visually checking the direction of speech, the operator can guess who is speaking, making it easier for the operator to communicate with the person who spoke.

[0171] Furthermore, in this embodiment, by emphasizing the human voice band and outputting a sound signal with noise removed, it becomes easier for the operator to understand what people around the work machine 100 are saying.

[0172] This embodiment shows an example of the arrangement of the external sound collection device M1, and is not limited to this arrangement example. For example, one or more microphone arrays may be provided on the upper rotating body 3. A microphone array has multiple microphones, and the direction from which the sound is coming can be recognized by processing and analyzing the differences (time difference, phase difference) in the sounds observed by the multiple microphones. When one directional microphone array is provided in this way, the direction of the sound can be detected from the sounds detected by each of the microphones included in the microphone array.

[0173] In this embodiment, an example in which four external sound collection devices M1 are provided has been described. However, this embodiment does not limit the number of external sound collection devices M1 attached to the work machine 100, and the number may be three or less, or five or more.

[0174] (Variation) In this embodiment, an example of a notification method in which a direction is displayed on the display device D1 has been described. However, the above-described embodiment does not limit the notification method to a method in which a direction is displayed on the display device D1. This modified example is an example in which a plurality of internal sound output devices SP2 are provided in the driver's cab 10. The internal sound output devices SP2 are provided, for example, in the driver's cab 10, in front, behind, left, and right of the operator's seat.

[0175] The identification unit 305 according to this modification identifies the speaking direction based on sound signals from the four external sound collection devices M1, and the output control unit 307 controls the output of sound from one of the multiple internal sound output devices SP2 that corresponds to the speaking direction. In this modification, the operator can recognize the speaking direction from the direction from which the sound is output. Note that the work machine 100 according to this modification does not need to be provided with an imaging device S6.

[0176] This modification shows one example of the notification method, and other examples are also possible. For example, the cab 10 may be provided with multiple light source units, and the output control unit 307 may control the lighting of one of the multiple light source units corresponding to the speaking direction. As another example, each of the multiple operating levers of the operating device 26 may be provided with a vibration mechanism, and the output control unit 307 may control the vibration of one of the operating levers corresponding to the speaking direction. In this modification, right and left operating levers are provided, so that the speaking direction can be intuitively recognized only in the left and right directions. In this modification, the above-mentioned notification method is used to allow the operator to recognize the speaking direction. Therefore, the operator can guess the speaking person by visually checking the speaking direction, which makes it easier to communicate with the speaking person.

[0177] The notification method is not limited to one mode, and multiple modes may be combined. For example, two or more of the following may be combined: displaying the direction on the display device D1, outputting a sound from the internal sound output device SP2 corresponding to the direction, lighting up the light source corresponding to the direction, and vibrating each of the multiple operating levers of the operating device 26.

[0178] (Second embodiment) Next, another configuration example of the work machine 100 according to the second embodiment will be described with reference to Fig. 12. Fig. 12 is a top view of another configuration example of the work machine 100 according to the second embodiment. The work machine 100 shown in Fig. 12 differs from the work machine 100 shown in Fig. 1 in that the external sound output device SP1 is configured from four speakers (front speaker SP1F, left speaker SP1L, right speaker SP1R, and rear speaker SP1B). In the work machine 100 shown in Fig. 1, the external sound output device SP1 is configured from a single omnidirectional speaker provided above the cab 10.

[0179] With this configuration, the work machine 100 shown in FIG. 12 can output sound toward the worker WK in front of the work machine 100 without outputting sound toward the workers WK to the left, right, and rear of the work machine 100, for example, by turning on the front speaker SP1F (a state in which sound can be output) and turning off the left speaker SP1L, the right speaker SP1R, and the rear speaker SP1B (a state in which sound cannot be output).

[0180] 12, a front camera S6F and a front microphone M1F are provided adjacent to the front speaker SP1F, and a front light bar G1F is provided on the housing of the front microphone M1F. A left camera S6L and a left microphone M1L are provided adjacent to the left speaker SP1L, and a left light bar G1L is provided on the housing of the left microphone M1L. A right camera S6R and a right microphone M1R are provided adjacent to the right speaker SP1R, and a right light bar G1R is provided on the housing of the right microphone M1R. A rear camera S6B and a rear microphone M1B are provided adjacent to the rear speaker SP1B, and a rear light bar G1B is provided on the housing of the rear microphone M1B.

[0181] In this case, the work machine 100 may turn on the light bar corresponding to the speaker that has been turned on (in a state capable of emitting light), and turn off the light bar corresponding to the speaker that has been turned off (in a state unable to emit light).

[0182] The external sound output device SP1 may be configured with one or more parametric speakers. A parametric speaker is a speaker that uses ultrasonic waves and can selectively transmit sound to people within a specific narrow range. A parametric speaker can transmit sound toward any position.

[0183] In the work machine 100 shown in Fig. 12, the controller 30 may detect a worker WK around the work machine 100 based on an image captured by the imaging device S6, and identify the position of the worker WK. Furthermore, if there are multiple workers WK around the work machine 100, the controller 30 may distinguish between a conversation target person (a worker WK who is speaking) and a non-conversation target person (a worker WK who is not speaking) based on the output of the four external sound collection devices M1. The controller 30 may also distinguish between a conversation target person (a worker WK facing the work machine 100) and a non-conversation target person (a worker WK who is not facing the work machine 100) based on an image captured by the imaging device S6. The controller 30 may then turn on a speaker and a light bar facing the worker WK. For example, if there is a worker WK (speaking) behind the work machine 100, the controller 30 may turn on the rear speaker SP1B while keeping the front speaker SP1F, left speaker SP1L, and right speaker SP1R off. In this case, the controller 30 may turn on the rear light bar G1B while keeping the front light bar G1F, left light bar G1L, and right light bar G1R off. Note that such a function may be implemented in the work machine 100 shown in Figures 1 to 5.

[0184] With this configuration, the controller 30 can output sound in the direction where the worker WK is located without outputting sound in the direction where the worker WK is not present, so the worker WK can easily recognize whether he or she is considered a conversation target or non-target.

[0185] (Third embodiment) In the third embodiment, a case where an operator remotely controls a work machine 100 will be described.

[0186] Next, an example configuration of an operation system (an example of a control system) SYS according to the third embodiment will be described with reference to Fig. 13. Fig. 13 is a schematic diagram showing an example configuration of an operation system (an example of a control system for a work machine) SYS for a work machine 100. As shown in Fig. 13, the operation system SYS includes a work machine 100, a remote control room RC, and a management center MC. Note that the detailed configuration of the work machine 100 is omitted from Fig. 13. This is because the work machine 100 shown in Fig. 13 has the same configuration as the work machine 100 shown in Fig. 1 or Fig. 10.

[0187] The work machine 100, remote control room RC, and management center MC are connected to one another so that they can send and receive data via a communication network NW. Note that the work machine 100, remote control room RC, and management center MC may also be connected to one another so that they can send and receive data directly to one another without going through the communication network NW. In the illustrated example, the work machine 100 transmits information about the work site to the remote control room RC. This allows the remote operator RO in the remote control room RC to understand the situation at the work site based on the information from the work machine 100.

[0188] For example, the work machine 100 transmits image information captured by the imaging device S6 and sound signals representing sounds collected by the external sound collection device M1 to the remote control room RC.

[0189] The work machine 100 is equipped with a sensor that can three-dimensionally recognize the position and shape of objects present at the work site. For example, the work machine 100 is equipped with a spatial recognition device. Therefore, the work machine 100 can transmit the results of three-dimensionally measuring the work site to the remote control room RC.

[0190] The spatial recognition device is a device for recognizing the space around the work machine 100. In the illustrated example, the spatial recognition device is a LiDAR. The LiDAR measures the distance between the LiDAR and each of one million or more points within a monitoring range, for example. The spatial recognition device may be any device that can measure the distance to an object. For example, the spatial recognition device may be a stereo camera, or may be a combination of the image capture device S6 and a distance measuring device such as millimeter wave radar.

[0191] The operation system SYS may include one or more work machines 100. When multiple work machines 100 are included, the remote operator RO of a specific work machine 100 can obtain information about the work site obtained by that specific work machine 100, as well as information about the work site obtained by one or more other work machines 100.

[0192] The remote control room RC is equipped with a communication device T2, a remote controller R40, an operation device R42, an operation sensor R43, a display device D1E, an internal sound collection device M2E, and an internal sound output device SP2E. The remote control room RC also is equipped with an operation seat DS where a remote operator RO who remotely operates the work machine 100 sits.

[0193] The communication device T2 is configured to be able to communicate with the communication device T1 attached to the work machine 100.

[0194] The remote controller R40 is a computing device that executes various calculations. In this embodiment, the remote controller R40 is configured as a microcomputer including a CPU and memory. The various functions of the remote controller R40 are realized by the CPU executing programs stored in the memory.

[0195] The display device D1E is a device capable of displaying various types of information. The display device D1E displays images based on information transmitted from the work machine 100 so that the remote operator RO in the remote control room RC can visually recognize the surroundings of the work machine 100. In the illustrated example, the display device D1E is a liquid crystal display. Note that the display device D1E may be a display or projector that realizes naked-eye stereoscopic vision, or may be VR goggles or the like.

[0196] The internal sound collection device M2E is a device that collects sounds generated within the remote control room RC. In the illustrated example, the internal sound collection device M2E is an indoor microphone that is configured to be able to pick up the voice of the remote operator RO within the remote control room RC.

[0197] The internal sound output device SP2E is a device capable of outputting various types of sound information. The internal sound output device SP2E outputs sound based on information transmitted from the work machine 100 so that the remote operator RO in the remote control room RC can hear sounds generated at the work site. The internal sound output device SP2E may be configured, for example, to output sound captured by an external sound collector M1 attached to the outside of the cab 10, or may be configured to output sound captured by an internal sound collector M2 attached to the inside of the cab 10. In this case, the internal sound collector M2 may be provided at a position corresponding to the ear position of the operator seated in the cab 10. The internal sound output device SP2E may be a stationary device such as a speaker, or may be a wearable device such as earphones or headphones. The speaker may be a monaural speaker, a stereo speaker, or a surround speaker. The speaker may also be an omnidirectional speaker or a directional speaker. The wearable device may have a noise canceling function, a spatial audio function (stereophonic sound function), or a bone conduction function.

[0198] The operation device R42 is provided with an operation sensor R43 for detecting the operation content of the operation device R42. The operation sensor R43 is, for example, an inclination sensor that detects the inclination angle of the operation lever, or an angle sensor that detects the swing angle of the operation lever around the swing axis. The operation sensor R43 may be configured with other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor R43 outputs information related to the detected operation content of the operation device R42 to the remote controller R40. The remote controller R40 generates an operation signal based on the received information and transmits the generated operation signal to the work machine 100. The operation sensor R43 may be configured to generate the operation signal. In this case, the operation sensor R43 may output the operation signal to the communication device T2 without passing through the remote controller R40. With this configuration, the remote operator RO can remotely operate the work machine 100 from the remote control room RC.

[0199] The remote controller R40, like the controller 30 of the first embodiment, includes an acquisition unit 301, a detection unit 302, a position estimation unit 303, a sound processing unit 304, an identification unit 305, a judgment unit 306, and an output control unit 307.

[0200] The acquisition unit 301 of the remote controller R40 acquires, via the communication device T2, a sound signal representing sound collected by the external sound collection device M1 from the work machine 100. Furthermore, the acquisition unit 301 acquires a sound signal representing sound collected by the internal sound collection device M2E of the remote control room RC.

[0201] The output control section 307 of the remote controller R40 enables control to output, via the communication device T2, from the internal sound output device SP2E of the remote control room RC, a sound based on a sound signal acquired from the external sound collection device M1 by the acquisition section 301. This allows the remote operator RO to hear the voices of people present around the work machine 100.

[0202] The output control section 307 of the remote controller R40 enables control to output sound based on the sound signal acquired by the acquisition section 301 from the internal sound collection device M2E of the remote control room RC from the external sound output device SP1 of the work machine 100 via the communication device T2. This allows people present around the work machine 100 to hear the voice of the remote operator RO.

[0203] The detection unit 302, position estimation unit 303, sound processing unit 304, identification unit 305, and determination unit 306 of the remote controller R40 perform the same control as the controller 30 of the above-described embodiment.

[0204] The remote controller R40, having this configuration, identifies the speaking direction based on the acquired sound signal, similar to the above-described controller 30. Then, similar to the above-described embodiment, the remote controller R40 displays the display screen 42 shown in Fig. 7, the display screen 42A shown in Fig. 9, or the display screen 42B shown in Fig. 10 on the display device DR.

[0205] The management center MC is a facility where various devices are installed to manage the remote operation of the work machine 100 by the remote operator RO in the remote operation room RC. In the illustrated example, the management center MC is installed in a location away from both the work site of the work machine 100 and the remote operation room RC. The management center MC is also equipped with a management device 200, an internal sound collection device M2C, and an internal sound output device SP2C.

[0206] The management device 200 is an example of a control device, and is, for example, a server computer (a so-called cloud server) or an edge server. The management device 200 is typically a fixed terminal device, but may also be a portable terminal device (for example, a laptop computer, a tablet, or a smartphone). The management device 200 can perform the same control as the remote controller R40.

[0207] With this configuration, a manager at the management center MC can, for example, use a sound collection device (external sound collection device M1 or internal sound collection device M2) attached to the work machine 100 and the internal sound output device SP2C to listen to sounds emitted at the work site. The manager at the management center MC can also, for example, use an internal sound collection device M2E and internal sound output device SP2C provided in the remote control room RC to listen to sounds emitted in the remote control room RC. The manager at the management center MC can, for example, use the internal sound collection device M2C and the external sound output device SP1 attached to the work machine 100 to communicate the voice he or she makes to a worker WK who is around the work machine 100. The manager at the management center MC can, for example, use the internal sound collection device M2C and the internal sound output device SP2E provided in the remote control room RC to communicate the voice he or she makes to a remote operator RO in the remote control room RC.

[0208] The management device 200 also has the same configuration as the remote controller R40. As in the above-described embodiment, the management device 200 displays the display screen 42 shown in FIG. 7, the display screen 42A shown in FIG. 9, or the display screen 42B shown in FIG. 10 on the display device DR.

[0209] In this embodiment, the internal sound collection device M2C and the internal sound output device SP2C realize the same functions as the internal sound collection device M2E and the internal sound output device SP2E, and the management device 200 transmits and receives sound signals to and from the work machine 100 or the remote control room RC. Therefore, the manager at the management center MC realizes two-way conversation with the remote operator RO in the remote control room RC or people (including the worker WK) present in the vicinity of the work machine 100. The specific control for realizing this two-way conversation is the same as in the above-mentioned embodiments, so a description thereof will be omitted.

[0210] As a result, in this embodiment, two-way conversation is realized between the remote operator RO present at the operator's seat DS and people present in the vicinity of the work machine 100. Similarly, two-way conversation is realized between the manager who manages the work site using the management device 200 and people present in the vicinity of the work machine 100. Therefore, even when remote operation is being performed or when remote management is being performed, it becomes easy to communicate with people present in the vicinity of the work machine 100.

[0211] Furthermore, in this embodiment, the remote operator RO present at the operator's seat DS and the manager at the management center MC can confirm the direction of speech in the vicinity of the work machine 100. Therefore, even when remote operation or remote management is being performed, it becomes easy to communicate with people present in the vicinity of the work machine 100.

[0212] <effect> In the above-described embodiment and modified examples, the operator can recognize the direction of the speech in the vicinity of the work machine 100. Therefore, the operator can easily communicate with the person who spoke. Furthermore, because the operator can grasp the situation around the person who spoke, the operator can recognize the intention of the speech. The operator can understand how to operate the work machine 100 in line with the intention of the speech. Therefore, in the embodiment and modified examples, the work efficiency of the work machine 100 can be improved.

[0213] While the embodiments of the work machine and work machine control system according to the present invention have been described above, the present invention is not limited to the above-described embodiments. Various changes, modifications, substitutions, additions, deletions, and combinations are possible within the scope of the claims. Naturally, these also fall within the technical scope of the present invention. [Explanation of symbols]

[0214] 100 Work Machinery 1 Undercarriage 2. Swivel mechanism 3 Upper rotating body 4. Boom 5 Arm 6 buckets S6 imaging device G1 Information Transmission Device M1 External sound collection device M2, M2C, M2E internal sound collector SP1 External Sound Output Device SP2, SP2C, SP2E internal sound output device SW switch KS Speak button DL1 External Volume Dial DL2 internal volume dial T1, T2 communications equipment 30 Controllers 301 Acquisition Department 302 Detection unit 303 Position estimation section 304 Sound Processing Unit 305 Specific section 306 Judgment section 307 Output control section RC remote control room R40 Remote Controller 200 Management device

Claims

1. a lower running body; an upper rotating body rotatably mounted on the lower traveling body; a sound collection device that is provided on the upper rotating body and is capable of detecting the direction of sound around the work machine; a control device configured to, when a person present in the vicinity of the work machine is speaking, report the direction of the speech based on the direction of the sound detected by the sound collection device; A work machine comprising:

2. The control device performs, as the notification, one or more of the following: displaying the speaking direction on a display device; outputting the sound from a sound output device corresponding to the speaking direction among a plurality of sound output devices provided inside the driver's cab; lighting up a light source unit corresponding to the speaking direction among a plurality of light source units; and vibrating an operation device corresponding to the speaking direction among a plurality of operation devices.

2. The work machine according to claim 1.

3. the control device displays the information indicating the speaking direction on the display device in a display manner different from that of information indicating other directions, using display information indicating the work machine as a reference.

3. The work machine according to claim 2.

4. An imaging device is further provided on the upper rotating body, the control device displays an overhead image showing the periphery of the work machine from above the work machine based on image information captured by the imaging device, and displays information indicating the speaking direction in a different display manner from information indicating other directions, using display information indicating the work machine included in the overhead image as a reference.

4. The work machine according to claim 3.

5. A plurality of imaging devices are further provided on the upper rotating body, the control device displays the image information captured by the plurality of imaging devices on the display device with the work machine as a reference, and varies the display mode of the image information captured in the speaking direction.

3. The work machine according to claim 2.

6. An imaging device is further provided on the upper rotating body, the control device is configured to correct the speaking direction based on a direction of a person detected from image information captured by the imaging device.

2. The work machine according to claim 1.

7. An imaging device is further provided on the upper rotating body, the control device suppresses notification of the direction when the direction of the person detected from the image information captured by the imaging device differs from the speaking direction.

2. The work machine according to claim 1.

8. A plurality of the sound collection devices are provided, Each of the plurality of sound collection devices is provided at a different position on the upper rotating body so as to be able to detect the direction of sound around the work machine.

2. The work machine according to claim 1.

9. the control device suppresses the sound generated by the work machine based on the sound signals input from the plurality of sound collection devices.

9. A work machine according to claim 8.

10. a work machine including a lower traveling body, an upper rotating body rotatably mounted on the lower traveling body, and a sound collecting device provided on the upper rotating body and capable of detecting the direction of sound around the work machine; a control device configured to, when a person present in the vicinity of the work machine is speaking, report the direction of the speech based on the direction of the sound detected by the sound collection device; A work machine control system comprising:

Images