Work machine, remote control system of work machine, and management system of work machine

Abstract

To provide a work machine capable of reducing the workload for an operator.SOLUTION: The work machine includes: an upper rotating body; a lower running body; an operator cab provided on the upper rotating body; an external sound pickup unit placed outside the operator cab; an internal audio output unit that is placed inside the operator cab to output the sounds picked up by the external sound pickup unit; and a control unit that identifies the types of sounds captured by the external sound pickup unit and notifies the same to the operator of the work machine.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a work machine, a work machine remote control system, and a work machine management system. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a technique is known in which a sound collector disposed outside the driver's cab collects sounds around the driver's seat. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-167680 Summary of the Invention [Problem to be solved by the invention]

[0004] Sounds change constantly, so if an operator in the vehicle tries to recognize sounds around the driver's seat, they must constantly listen carefully to the collected sounds, which places a heavy burden on the operator.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to reduce the burden on the operator of a work machine. [Means for solving the problem]

[0006] A work machine according to an embodiment of the present invention is a work machine having an upper rotating body, a lower running body, a cab provided on the upper rotating body, an external sound collection device located outside the cab, an internal sound output device located inside the cab and outputting sound collected by the external sound collection device, and a control device that identifies the type of sound collected by the external sound collection device and notifies the operator of the work machine of information indicating the identified type of sound.

[0007] A remote control system for a work machine according to an embodiment of the present invention is a remote control system for a work machine that includes an external sound collection device that collects sounds around the work machine, an external device that remotely controls the work machine, a sound output device that outputs sounds toward an operator of the external device, and a control device that identifies the type of sound detected based on the sound collected by the external sound collection device and notifies the operator of the external device of information indicating the identified type of sound.

[0008] A work machine management system according to an embodiment of the present invention is a work machine management system having a work machine and a management device that manages the work machine, wherein the work machine has an upper rotating body, a lower running body, a cab provided on the upper rotating body, an external sound collection device located outside the cab, an internal sound output device located inside the cab and outputting sound collected by the external sound collection device, and a control device that transmits log information acquired during the period when sound is collected by the external sound collection device to the management device, and the management device stores the log information received from the work machine in a storage device. [Effects of the Invention]

[0009] The burden on the operator of the work machine can be reduced. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a side view of a work machine 100. [Figure 2] FIG. 1 is a top view of a work machine 100. [Figure 3] 2 is a diagram showing an example of the configuration of an external sound collection device M1 and an information transmission device G1 attached to a work machine 100. FIG. [Figure 4] 1 is a diagram illustrating an example of the configuration of a work machine 100. FIG. [Figure 5] FIG. 2 is a top view of the interior of the operator's cab 10. [Figure 6] 1 is a perspective view of a work machine 100 with an operator OP on board and a worker WK positioned to the left and front of the work machine 100. FIG. [Figure 7] 4 is a flowchart illustrating the operation of the controller 30 of the work machine 100. [Figure 8] 10A and 10B are diagrams illustrating a display example of a display device D1. [Figure 9] FIG. 10 is a top view of another example configuration of the work machine 100. [Figure 10] FIG. 1 is a schematic diagram showing an example of the configuration of an operation system SYS. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The embodiments described below are illustrative and do not limit the invention. Not all features and combinations thereof in the embodiments of the present disclosure are necessarily essential to the invention. In addition, identical or corresponding components in each drawing are designated by identical or corresponding reference numerals, and redundant description may be omitted.

[0012] The work machine 100 according to the embodiment of the present disclosure may be a machine other than a work machine, such as a crane, an asphalt finisher, or a forklift. In the illustrated example, the work machine 100 is an excavator equipped with a bucket 6 as an end attachment, but it may also be an applied machine such as a forestry machine equipped with an end attachment other than the bucket 6.

[0013] First, an overview of the work machine 100 will be described with reference to Figures 1 and 2. Figure 1 is a top view of the work machine 100, and Figure 2 is a side view of the work machine 100.

[0014] +X in Fig. 1 represents one direction of the X axis that constitutes a three-dimensional Cartesian coordinate system, and -X represents the other direction of the X axis. In Fig. 2, +Y represents one direction of the Y axis that constitutes a three-dimensional Cartesian coordinate system, and -Y represents the other direction of the Y axis. In Fig. 1, +Z represents one direction of the Z axis that constitutes a three-dimensional Cartesian coordinate system, and -Z represents the other direction of the Z axis. In Fig. 1, the +X side of the work machine 100 corresponds to the front side of the work machine 100, and the -X side of the work machine 100 corresponds to the rear side of the work machine 100. Furthermore, the +Y side of the work machine 100 corresponds to the left side of the work machine 100, and the -Y side of the work machine 100 corresponds to the right side of the work machine 100. Furthermore, the +Z side of the work machine 100 corresponds to the top side of the work machine 100, and the -Z side of the work machine 100 corresponds to the bottom side of the work machine 100. The same applies to the other figures.

[0015] The work machine 100 comprises a lower traveling body 1, an upper rotating body 3 that is mounted on the lower traveling body 1 so as to be rotatable via a rotating mechanism 2, an attachment AT for performing various tasks, and a driver's cab 10. The driver's cab 10 is also called a cabin or cab. The front side of the work machine 100 (upper rotating body 3) corresponds to the side of the upper rotating body 3 to which the attachment AT is attached, when the work machine 100 is viewed from directly above along the rotation axis of the upper rotating body 3. Furthermore, the left, right, and rear sides of the work machine 100 (upper rotating body 3) correspond to the left, right, and rear sides, respectively, as viewed from the perspective of an operator seated in the driver's seat in the driver's cab 10.

[0016] The lower traveling body 1 includes, for example, a pair of left and right crawlers 1C. Specifically, the crawlers 1C include a left crawler 1CL and a right crawler 1CR. The left crawler 1CL is driven by a left traveling hydraulic motor 2ML, and the right crawler 1CR is driven by a right traveling hydraulic motor 2MR. The left traveling hydraulic motor 2ML is a traveling drive unit that drives the left crawler 1CL as a driven part, and can rotate the left crawler 1CL. The right traveling hydraulic motor 2MR is a traveling drive unit that drives the right crawler 1CR as a driven part, and can rotate the right crawler 1CR. Note that the traveling drive units may be electric motors.

[0017] The upper rotating body 3 rotates relative to the lower traveling body 1 when the rotating mechanism 2 is driven by the hydraulic swing motor 2A. The hydraulic swing motor 2A is a swing drive unit that drives the upper rotating body 3 as a driven unit, and can change the orientation of the upper rotating body 3. The swing drive unit may be an electric motor.

[0018] A boom 4 is rotatably attached to the center of the front of the upper rotating body 3, an arm 5 is rotatably attached to the tip of the boom 4, and a bucket 6 is rotatably attached to the tip of the arm 5. In the illustrated example, the boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of an attachment AT. The boom 4, arm 5, and bucket 6 are driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.

[0019] The bucket 6 is an example of a work tool (end attachment). The bucket 6 is used, for example, for excavation work. Instead of the bucket 6, another work tool may be attached to the end of the arm 5 depending on the type of work, etc. The other work tool may be, for example, another type of bucket, such as a large bucket, a slope bucket, or a dredging bucket. The other work tool may also be a type of work tool other than a bucket, such as an agitator, a breaker, a grapple, or a lifting magnet.

[0020] The swing hydraulic motor 2A, the left traveling hydraulic motor 2ML, the right traveling hydraulic motor 2MR, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9 are hydraulic actuators driven by hydraulic oil discharged from a hydraulic pump.

[0021] Note that in work machine 100, all or some of the driven parts, such as lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6, may be electrically driven. In other words, work machine 100 may be a hybrid work machine, an electric work machine, or the like, in which all or some of the driven parts are driven by electric actuators.

[0022] Additionally, the work machine 100 is equipped with an information transmission device G1, an external sound collection device M1, an imaging device S6, and an external sound output device SP1.

[0023] The imaging device S6 is provided on the upper rotating body 3 or the cab 10, and captures images of the periphery of the work machine 100 to obtain image information showing the periphery of the work machine 100. In the illustrated example, the imaging device S6 includes a front camera S6F, a left camera S6L, a right camera S6R, and a rear camera S6B.

[0024] The front camera S6F is a camera that captures images in front of the work machine 100, and is attached to the outside of the cab 10, such as on the roof of the cab 10 or on the side of the boom 4. The front camera S6F may also be attached to the inside of the cab 10, for example, on the ceiling of the cab 10. The left camera S6L is a camera that captures images to the left of the work machine 100, the right camera S6R is a camera that captures images to the right of the work machine 100, and the rear camera S6B is a camera that captures images to the right of the work machine 100. Specifically, the front camera S6F, left camera S6L, right camera S6R, and rear camera S6B are all monocular wide-angle cameras equipped with an imaging element such as a CCD or CMOS, and output the captured images to the display device D1. Information about the images captured by the imaging device S6 is taken into the controller 30.

[0025] In the illustrated example, the front camera S6F is attached to the roof of the driver's cab 10, the left camera S6L is attached to the left end of the upper surface of the upper rotating body 3, the right camera S6R is attached to the right end of the upper surface of the upper rotating body 3, and the rear camera S6B is attached to the rear end of the upper surface of the upper rotating body 3.

[0026] The imaging device S6 may constitute an object detection device that detects objects around the work machine 100. Examples of objects include people, animals, vehicles, construction machinery, buildings, and holes. The object detection device may be configured to be able to distinguish between people and non-human objects when detecting them. In other words, the object detection device may be configured to function as a person detection device. The object detection device may be constituted by a device other than a camera. For example, the object detection device may be a LiDAR. A LiDAR is a device that can measure the distance between the LiDAR (laser source) and a point cloud of, for example, one million or more points within a monitoring range. The object detection device may also be another device that can measure the distance to an object, such as a stereo camera, a range imaging camera, or a millimeter-wave radar. When a millimeter-wave radar or the like is used as the object detection device, the object detection device may emit a number of signals (such as laser light) toward the object and receive the reflected signals to derive the distance and direction of the object. Alternatively, the object detection device may be a combination of two or more types of devices. For example, the object detection device may be a combination of an imaging device and a LiDAR, a combination of an imaging device and a millimeter wave radar, or a combination of an imaging device and a stereo camera.

[0027] The external sound collection device M1 is a device that collects external sounds and is also called a microphone or a microphone. In the illustrated example, the external sound collection device M1 is provided on the upper rotating body 3 or the cab 10, and converts sounds (air vibrations) generated around the work machine 100 into mechanical vibrations, which are then converted into electrical signals. Specifically, the external sound collection device M1 is configured to be able to pick up the voices of workers around the work machine 100, and includes a front microphone M1F, a left microphone M1L, a right microphone M1R, and a rear microphone M1B.

[0028] The front microphone M1F is a microphone that collects sounds generated in front of the work machine 100, and is attached to the outside of the cab 10, such as on the roof of the cab 10 or on the side of the boom 4. The front microphone M1F may also be attached to the inside of the cab 10, for example, on the ceiling of the cab 10. The left microphone M1L is a microphone that collects sounds generated to the left of the work machine 100, the right microphone M1R is a microphone that collects sounds generated to the right of the work machine 100, and the rear microphone M1B is a microphone that collects sounds generated behind the work machine 100. The electrical signals generated by the front microphone M1F, left microphone M1L, right microphone M1R, and rear microphone M1B are taken into the controller 30.

[0029] In the illustrated example, the front microphone M1F is attached to the roof of the cab 10, the left microphone M1L is attached to the left end of the upper surface of the upper rotating body 3, the right microphone M1R is attached to the right end of the upper surface of the upper rotating body 3, and the rear microphone M1B is attached to the rear end of the upper surface of the upper rotating body 3. In this way, the four external sound collection devices M1 (the front microphone M1F, the left microphone M1L, the right microphone M1R, and the rear microphone M1B) are provided at different positions on the upper rotating body 3. Therefore, the controller 30 can detect the direction of the sound source based on the difference in the sounds collected by each of the four external sound collection devices M1 (e.g., difference in volume). Furthermore, when an array microphone is used as the external sound collection device M1, the direction of the sound source can be detected based on, for example, a phase shift or difference in volume.

[0030] In the illustrated example, the four external sound collection devices M1 and the four image capture devices S6 are arranged to correspond to one another. Specifically, the front microphone M1F is arranged adjacent to the front camera S6F, the left microphone M1L is arranged adjacent to the left camera S6L, the right microphone M1R is arranged adjacent to the right camera S6R, and the rear microphone M1B is arranged adjacent to the rear camera S6B.

[0031] The external sound output device SP1 is a device that outputs sound toward the periphery of the work machine 100. In the illustrated example, the external sound output device SP1 is an omnidirectional speaker that is configured to output sound uniformly in all directions. However, the external sound output device SP1 may also be a directional speaker that outputs sound toward a specific direction, such as the front.

[0032] The information transmission device G1 is a device for communicating the status of the work machine 100 to someone outside the work machine 100. In the illustrated example, the information transmission device G1 is provided on the upper rotating body 3 or the operator's cab 10, and is configured to be able to communicate the status of the work machine 100 to an operator around the work machine 100. Specifically, the information transmission device G1 is a light-emitting device, and includes a front light bar G1F, a left light bar G1L, a right light bar G1R, and a rear light bar G1B.

[0033] The front light bar G1F is a light-emitting device that can visually convey information to workers and the like in front of the work machine 100, and is attached to the outside of the cab 10, such as on the roof of the cab 10 or on the side of the boom 4. The front light bar G1F may also be attached to the inside of the cab 10, for example, on the ceiling of the cab 10. The left light bar G1L is a light-emitting device that can visually convey information to workers and the like on the left side of the work machine 100, the right light bar G1R is a light-emitting device that can visually convey information to workers and the like on the right side of the work machine 100, and the rear light bar G1B is a light-emitting device that can visually convey information to workers and the like behind the work machine 100. The front light bar G1F, left light bar G1L, right light bar G1R, and rear light bar G1B each emit light in response to an electrical signal from the controller 30. In the illustrated example, the light-emitting devices are LED lights, but they may also be other light-emitting devices, such as halogen lamps. Furthermore, the light emitting device is of a multicolor emission type, but may be of a monochromatic emission type.

[0034] In the illustrated example, the front light bar G1F is attached to the roof of the operator's cab 10, the left light bar G1L is attached to the left end of the upper surface of the upper rotating body 3, the right light bar G1R is attached to the right end of the upper surface of the upper rotating body 3, and the rear light bar G1B is attached to the rear end of the upper surface of the upper rotating body 3. In this way, the four information transmission devices G1 (front light bar G1F, left light bar G1L, right light bar G1R, and rear light bar G1B) are provided in different positions on the upper rotating body 3. Therefore, the controller 30 can communicate the status of the work machine 100 to workers and the like who are respectively located in front of, on the left, right, and rear of the work machine 100 by separately operating each of the four information transmission devices G1.

[0035] In the illustrated example, the four information transmission devices G1 and the four external sound collection devices M1 are arranged to correspond to each other. Specifically, the front light bar G1F is arranged adjacent to the front microphone M1F, the left light bar G1L is arranged adjacent to the left microphone M1L, the right light bar G1R is arranged adjacent to the right microphone M1R, and the rear light bar G1B is arranged adjacent to the rear microphone M1B.

[0036] Fig. 3 is a diagram showing an example configuration of an external sound collection device M1 and an information transmission device G1 attached to a work machine 100. Specifically, Fig. 3 is a perspective view of a left microphone M1L and a left light bar G1L attached to a housing having a substantially rectangular parallelepiped shape. Note that the following description with reference to Fig. 3 relates to the combination of the left microphone M1L and the left light bar G1L, but it also applies to the combination of the front microphone M1F and the front light bar G1F, the combination of the right microphone M1R and the right light bar G1R, and the combination of the rear microphone M1B and the rear light bar G1B.

[0037] As shown in FIG. 3, the left microphone M1L and left light bar G1L are arranged on the left side of a substantially rectangular parallelepiped housing so as to face to the left of the work machine 100. With this arrangement, the left microphone M1L can efficiently collect sounds generated to the left of the work machine 100, and the left light bar G1L can efficiently communicate the status of the work machine 100 to an operator on the left of the work machine 100. For example, the left microphone M1L can pick up the voice uttered by an operator on the left of the work machine 100, and the left light bar G1L can notify the operator that the left microphone M1L has picked up the operator's voice by emitting light in a predetermined color. In this case, an operator on the left of the work machine 100 who speaks into the left microphone M1L can confirm that their voice has reached the left microphone M1L (i.e., the operator of the work machine 100) by seeing the left light bar G1L emitting light in a predetermined color.

[0038] The information transmission device G1 may be provided at the upper part of each of the four side surfaces of the cab 10 (see FIG. 6). For example, the information transmission device G1 may be configured so that a front light bar G1F is attached to the upper part of the front surface of the cab 10, a left light bar G1L is attached to the upper part of the left surface of the cab 10, a right light bar G1R is attached to the upper part of the right surface of the cab 10, and a rear light bar G1B is attached to the upper part of the rear surface of the cab 10. The information transmission device G1 may also be a single rotating light such as a Nico Torch attached to the upper surface of the cab 10, or a display device such as a liquid crystal display or an organic EL display.

[0039] The controller 30 is an example of a control device, and is configured, for example, by a computer including a CPU, a volatile storage device, a non-volatile storage device, and various input / output interfaces. The controller 30 then realizes various functions, for example, by reading a program from the non-volatile storage device, loading it into the volatile storage device, and having the CPU execute the program. In the illustrated example, the controller 30 is configured to realize various functions to control the work machine 100. The various functions include, for example, a machine guidance function that guides the operator in manually operating the work machine 100. The various functions may also include a contact avoidance function that automatically or autonomously operates or stops the work machine 100 to avoid contact between the work machine 100 and an object that is present within a monitoring range around the work machine 100.

[0040] The boom angle sensor S1 detects the boom angle, which is the rotation angle of the boom 4 relative to the upper rotating body 3. The arm angle sensor S2 detects the arm angle, which is the rotation angle of the arm 5 relative to the boom 4. The bucket angle sensor S3 detects the bucket angle, which is the rotation angle of the bucket 6 relative to the arm 5.

[0041] Each of the boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 may be, for example, a rotary encoder, an acceleration sensor, a six-axis sensor, an IMU (Inertial Measurement Unit), a potentiometer using a variable resistor, a cylinder stroke sensor that detects the stroke amount of a hydraulic cylinder, etc.

[0042] The controller 30 receives a detection signal corresponding to the boom angle from the boom angle sensor S1, a detection signal corresponding to the arm angle from the arm angle sensor S2, and a detection signal corresponding to the bucket angle from the bucket angle sensor S3.

[0043] The machine body tilt sensor S4 detects the tilt state of the machine body (undercarriage 1 or upper rotating body 3) relative to the horizontal plane. The machine body tilt sensor S4 is attached to, for example, the upper rotating body 3, and detects the tilt angle of the work machine 100 (i.e., the upper rotating body 3) around two axes in the forward / backward and left / right directions. The machine body tilt sensor S4 may be, for example, an acceleration sensor, a six-axis sensor, an IMU, or the like. A detection signal corresponding to the tilt angle detected by the machine body tilt sensor S4 is input to the controller 30.

[0044] The rotation sensor S5 outputs information related to the rotation of the upper rotating body 3. The rotation sensor S5 detects, for example, the rotation angular velocity of the upper rotating body 3 relative to the lower traveling body 1. The rotation sensor S5 may also detect the rotation angle. The rotation sensor S5 may be, for example, a gyro sensor, a resolver, or a rotary encoder. A detection signal corresponding to the rotation angle or rotation angular velocity of the upper rotating body 3 detected by the rotation sensor S5 is input to the controller 30.

[0045] The positioning device PS measures the position of the upper rotating body 3. The positioning device PS is, for example, a Global Navigation Satellite System (GNSS) compass, and detects the position and orientation of the upper rotating body 3. A detection signal corresponding to the position and orientation of the upper rotating body 3 is input to the controller 30. The function of detecting the orientation of the upper rotating body 3 may be realized by a direction sensor attached to the upper rotating body 3.

[0046] The operator's cab 10 is a compartment space in which an operator rides, and is provided on the front left side of the upper rotating body 3. However, when the work machine 100 is remotely controlled or when the work machine 100 operates by fully automatic driving, the operator's cab 10 may be omitted.

[0047] The communication device T1 communicates with an external device through a communication network including a mobile communication network, a satellite communication network, the Internet, etc. The communication device T1 is, for example, a mobile communication module compatible with a mobile communication standard such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation), a communication module compatible with a short-range wireless communication standard such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or a satellite communication module for connecting to a satellite communication network.

[0048] In response to operations by an operator seated in the cab 10, the work machine 100 operates actuators to drive driven parts such as the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6.

[0049] Alternatively, the work machine 100 may be configured so that it can be remotely operated from outside the work machine 100. When the work machine 100 is remotely operated, the inside of the cab 10 may be unmanned.

[0050] Furthermore, the work machine 100 may automatically operate the actuators regardless of the operation performed by the operator. This allows the work machine 100 to realize a function of automatically operating at least some of the driven parts, such as the lower traveling structure 1, upper rotating structure 3, boom 4, arm 5, and bucket 6, i.e., a so-called "machine control function."

[0051] Figure 4 is a diagram that schematically shows an example of the configuration of a work machine 100. In Figure 4, the mechanical power transmission system, hydraulic oil lines, pilot lines, and electrical control system are indicated by double lines, thick solid lines, thick dashed lines, and dotted lines, respectively.

[0052] The drive system of the work machine 100 includes an engine 11, a regulator 13, a main pump 14, and a control valve unit 17. The hydraulic drive system of the work machine 100 also includes hydraulic actuators such as a swing hydraulic motor 2A, a left traveling hydraulic motor 2ML, a right traveling hydraulic motor 2MR, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9.

[0053] The engine 11 is an example of a power source for the work machine 100, and is mounted, for example, on the rear of the upper rotating body 3. The power source for the work machine 100 may be a combination of a power source such as a battery or a fuel cell and an electric motor. Specifically, the engine 11 rotates at a constant speed at a preset target rotation speed under direct or indirect control by the controller 30, and drives the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses diesel as fuel. The engine 11 may also be a gasoline engine, a hydrogen engine, or the like.

[0054] The regulator 13 controls the discharge amount of the main pump 14. For example, the regulator 13 controls the discharge amount of the main pump 14 by adjusting the angle (tilting angle) of the swash plate of the main pump 14 in response to a control command from the controller 30.

[0055] The main pump 14 is mounted on the rear of the upper rotating body 3, similar to the engine 11, and supplies hydraulic oil to the control valve unit 17 through a hydraulic oil line. In the illustrated example, the main pump 14 is a variable displacement hydraulic pump.

[0056] The control valve unit 17 is one of the hydraulic control devices that controls the hydraulic system of the work machine 100. In the illustrated example, the control valve unit 17 includes control valves 171-176. The control valve unit 17 is configured to selectively supply hydraulic oil discharged by the main pump 14 to one or more hydraulic actuators via the control valves 171-176. The control valves 171-176 control the flow rate of hydraulic oil flowing from the main pump 14 to the hydraulic actuators, and the flow rate of hydraulic oil flowing from the hydraulic actuators to a hydraulic oil tank. The hydraulic actuators include a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, a left traveling hydraulic motor 2ML, a right traveling hydraulic motor 2MR, and a swing hydraulic motor 2A. Specifically, the control valve 171 corresponds to the left traveling hydraulic motor 2ML, the control valve 172 corresponds to the right traveling hydraulic motor 2MR, and the control valve 173 corresponds to the swing hydraulic motor 2A. Furthermore, the control valve 174 corresponds to the bucket cylinder 9 , the control valve 175 corresponds to the boom cylinder 7 , and the control valve 176 corresponds to the arm cylinder 8 .

[0057] The pilot pump 15 is an example of a pilot pressure generating device, and is configured to be able to supply hydraulic oil to hydraulic control devices via a pilot line. In the illustrated example, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pressure generating device may also be realized by the main pump 14. That is, the main pump 14 may have a function to supply hydraulic oil to various hydraulic control devices via pilot lines, in addition to a function to supply hydraulic oil to the control valve unit 17 via a hydraulic oil line. In this case, the pilot pump 15 may be omitted.

[0058] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In the illustrated example, the discharge pressure sensor 28 outputs the detected value to the controller 30.

[0059] The operating device 26 is a device used by an operator to operate the actuator. The operating device 26 includes, for example, an operating lever and an operating pedal. The actuator may be a hydraulic actuator or an electric actuator.

[0060] The operation sensor 29 is configured to detect the operation content of the operator using the operation device 26. In this embodiment, the operation sensor 29 detects the operation direction and operation amount of the operation device 26 corresponding to each actuator and outputs the detected value to the controller 30. In the illustrated example, the controller 30 can control the opening area of ​​the valve 31 in accordance with the output of the operation sensor 29. The controller 30 then supplies the hydraulic oil discharged from the pilot pump 15 to the pilot ports of the corresponding control valves in the control valve unit 17. The pressure of the hydraulic oil supplied to each pilot port (pilot pressure) is, in principle, a pressure corresponding to the operation direction and operation amount of the operation device 26 corresponding to each hydraulic actuator. In this way, the operation device 26 is configured to supply the hydraulic oil discharged from the pilot pump 15 to the pilot ports of the corresponding control valves in the control valve unit 17.

[0061] Valve 31, which functions as a control valve for machine control, is disposed in a pipe connecting pilot pump 15 and a pilot port of a control valve in control valve unit 17, and is configured so that the flow path area of ​​the pipe can be changed. In the illustrated example, valve 31 is a solenoid valve that operates in response to a control command output by controller 30. Therefore, controller 30 can use valve 31 to adjust the pilot pressure acting on the pilot port of the control valve, regardless of the operation of operating device 26 by the operator.

[0062] With this configuration, the controller 30 can operate the hydraulic actuator corresponding to a specific operating device 26 even when the specific operating device 26 is not being operated.

[0063] 4, the control system of the work machine 100 includes a controller 30, a display device D1, an input device D2, a horn button HS, a talk button KS, an external sound collection device M1, an internal sound collection device M2, an external sound output device SP1, an internal sound output device SP2, an external volume dial DL1, an internal volume dial DL2, a switch SW, and a communication device T1. The display device D1 of this embodiment also has a control unit 40a that controls the screen displayed on the image display unit 42, which will be described later. The control unit 40a is a computer equipped with a CPU, RAM, NVRAM, ROM, etc. The control unit 40a reads programs corresponding to each functional element from the ROM, loads them into the RAM, and causes the CPU to execute the corresponding processing.

[0064] The controller 30 is configured to output a control command to the regulator 13 as necessary, thereby changing the discharge rate of the main pump 14. The controller 30 may also be configured to perform control related to a machine guidance function that guides (provides guidance for) the manual operation of the work machine 100 by an operator via the operation device 26. The controller 30 may also be configured to perform control related to a machine control function that automatically assists the manual operation of the work machine 100 by an operator via the operation device 26. Some of the functions of the controller 30 may be realized by another controller (control device). In other words, the functions of the controller 30 may be realized in a distributed manner by multiple controllers. For example, the machine guidance function and the machine control function may be realized by dedicated controllers (control devices).

[0065] Furthermore, the controller 30 of this embodiment detects the occurrence of an event from sound data collected by the external sound collection device M1 and the internal sound collection device M2, the detection results of the object detection device, the operation of the work machine 100, etc. An event in this embodiment is a predetermined occurrence. Furthermore, when the controller 30 detects the occurrence of an event, it causes the display device D1 to display event information including the type of event and the direction in which the event occurred.

[0066] In this embodiment, by doing this, sound data collection starts automatically, and when an event is detected, the type of event and the direction in which the event occurred are displayed on the display device D1. This allows the operator of the work machine 100 to visually recognize the type and direction of the event that has occurred. Therefore, according to this embodiment, the operator of the work machine 100 can concentrate on operating the work machine 100 rather than listening to surrounding sounds, reducing the burden on the operator during work.

[0067] In addition, the controller 30 stores log information including image data acquired by the imaging device S6 during the period when the event is occurring, sound data collected during the period when the event is occurring, and operation information of the work machine 100 acquired during the period when the event is occurring in association with the event information in the storage device.

[0068] The image data included in the log information may be image data showing an image of the surroundings of the work machine 100, or may be image data showing an image of the interior of the cab 10 of the work machine 100. Furthermore, the image data included in the log information may be still image data or video data.

[0069] The following further describes the functions of the controller 30. The controller 30 of this embodiment has a sound collection control unit 301, a sound analysis unit 302, an event detection unit 303, an event identification unit 304, an association unit 305, a display control unit 306, and an information storage unit 307.

[0070] The sound collection control unit 301 controls the external sound collection device M1 and the internal sound collection device M2. Specifically, the sound collection control unit 301 controls the start and end of sound collection using at least one of the external sound collection device M1 and the internal sound collection device M2. In other words, the sound collection control unit 301 starts sound collection at a predetermined start timing and ends sound collection at a predetermined end timing.

[0071] The predetermined start timing in this embodiment may be, for example, the timing when the work machine 100 is keyed on, or the timing when the operation state of the conversation function, which will be described later, is turned on. In this case, the predetermined end timing may be the timing when the work machine 100 is keyed off, or the timing when the operation state of the conversation function is turned off.

[0072] Furthermore, the predetermined start timing in this embodiment may be the timing when a specific operation is started on the work machine 100. The specific operation may be, for example, an operation on the attachment AT of the work machine 100 or an operation to cause the work machine 100 to travel. In this case, the predetermined end timing may be the timing when the specific operation on the work machine 100 has ended.

[0073] Furthermore, the predetermined start timing in this embodiment may be the timing when the work machine 100 is placed in a specific operation mode. The specific operation mode may be, for example, a crane mode or an operation mode that executes a payload function that calculates the load inside the bucket 6. In this case, the predetermined end timing may be the timing when a specific operation on the work machine 100 is completed.

[0074] The predetermined start timing may be the timing at which an event is detected by the event detection unit 303. In this case, the predetermined end timing may be the timing at which the event detected by the event detection unit 303 ends, or may be the timing at which a certain time has elapsed since the event ended.

[0075] That is, the sound collection control unit 301 of this embodiment may start collecting sound when an event is detected, or may start collecting sound at a predetermined start timing that is set in advance, separate from the detection of an event. Details of events will be described later.

[0076] The sound analysis unit 302 analyzes sound data collected by the external sound collection device M1 and the internal sound collection device M2. Specifically, the sound analysis unit 302 identifies the direction of the sound source that generated the sound and the type of sound based on the collected sound data. The direction of the sound source is identified based on the sound data collected by the front microphone M1F, left microphone M1L, right microphone M1R, and rear microphone M1B. Note that if an array microphone is used as the external sound collection device M1, the direction of the sound source can be identified by the array microphone.

[0077] Furthermore, for example, when the sound data includes different types of sounds generated from multiple sound sources, the sound analysis unit 302 may separate the sound data by sound type. Furthermore, when the sound data is speech, the sound analysis unit 302 of this embodiment may perform voice recognition processing on the sound data and convert the sound data into text data.

[0078] The event detection unit 303 detects the occurrence of an event according to the analysis result of the sound analysis unit 302. The event detection unit 303 also detects the end of an event according to the analysis result of the sound analysis unit 302. Specifically, the event detection unit 303 may detect the end of an event when an event that was detected as an event is no longer detected from the sound data.

[0079] Here, an event in this embodiment will be described.

[0080] An event in this embodiment may be, for example, an utterance by the operator of the work machine 100 in the cab 10 or by a worker in the vicinity of the work machine 100.

[0081] In this embodiment, by detecting the speech of the operator of the work machine 100 and surrounding workers as events, it is possible to obtain sound data that includes the conversation history of the operator of the work machine 100 and surrounding workers.

[0082] Furthermore, the event in this embodiment may be the timing when the operation state of a conversation function, which will be described later, is turned on.

[0083] In this embodiment, by detecting an event in this way when it becomes possible to have a conversation with the operator of the work machine 100 or surrounding workers, sound data including the history of conversations between the operator of the work machine 100 and surrounding workers can be acquired and stored in a storage device.

[0084] In addition, the event in this embodiment may be the appearance of a specific keyword in the utterance. The specific keyword may be, for example, a word that is associated with a dangerous situation, and may be predetermined and stored in the controller 30.

[0085] In this embodiment, by doing this, it is possible to detect the possibility of a dangerous situation occurring, and further, it is possible to obtain sound data including a conversation history when there is a possibility of a dangerous situation occurring.

[0086] Furthermore, the event in this embodiment may be the key being turned on for the work machine 100. In this embodiment, by doing this, it is possible to acquire all sound data while the work machine 100 is in operation.

[0087] Furthermore, the event of this embodiment may be a sound that satisfies a specific condition. A sound that satisfies the specific condition may be, for example, a sound whose volume is equal to or greater than a predetermined volume, or a sound that is sufficiently loud compared to the volume of the sound collected immediately before. A sufficiently loud sound may be, for example, a sound that is louder than the volume of the sound collected immediately before by a predetermined amount or more. A sound that satisfies the specific condition may be, for example, a sound that notifies of danger. A sound that notifies of danger may be, for example, a horn or a siren.

[0088] In this embodiment, by doing so, it is possible to detect the occurrence of an event requiring attention around the work machine 100, and also to acquire sound data at that time.

[0089] Furthermore, a sound that satisfies a specific condition may be, for example, an operating sound that is inconsistent with the operation of the work machine 100. In this embodiment, by doing so, it is possible to detect the possibility of an operation error on the work machine 100, and obtain sound data at that time.

[0090] Furthermore, a sound that satisfies a specific condition may be, for example, a sound that notifies of an abnormality in the work machine 100. Sounds that notify of an abnormality in the work machine 100 include, for example, a sound that indicates an oil temperature abnormality. Sounds that notify of an abnormality in the work machine 100 may be stored in advance by the event detection unit 303. In this embodiment, by doing so, it is possible to detect that an abnormality may have occurred in the work machine 100, and to acquire sound data at that time.

[0091] Furthermore, the event in this embodiment may be the detection of an object by an object detection device around the work machine 100. Specifically, for example, the event detection unit 303 may detect the occurrence of an event when a person is detected by the object detection device.

[0092] In this embodiment, by doing so, sound data can be acquired when an object approaches the work machine 100.

[0093] The event identification unit 304 identifies the type of event according to the content of the event detected by the event detection unit 303 .

[0094] Specifically, for example, if the detected event is "utterance," the event identification unit 304 may identify the type of the event as "utterance." Also, if the detected event is "words associated with a dangerous situation," the event identification unit 304 may identify the type of the event as "dangerous situation."

[0095] Furthermore, if the detected event is "utterance," the event identification unit 304 may identify the person speaking. Specifically, for example, the event identification unit 304 may store voice data for each worker in advance. Then, if the detected event is "utterance," the event identification unit 304 may compare the pre-stored voice data for each worker with the collected sound data to identify the person speaking.

[0096] Furthermore, if the detected event is "speech," the event identification unit 304 may identify the person speaking by performing face recognition processing or the like on the image of an imaging device that captures the direction of the sound source obtained by the sound analysis unit 302.

[0097] In this embodiment, when the event is "utterance," it is possible to identify the person who is making the utterance.

[0098] Furthermore, if the detected event is a "sound notifying of danger," the event identification unit 304 may identify the type of event as a "danger notification." Furthermore, if the identified event is the detection of a sound that is inconsistent with the operation content of the work machine 100 or the detection of an object around the work machine 100, the event identification unit 304 may identify the type of event as a "near miss."

[0099] Furthermore, if the identified event is a sound that notifies of an abnormality in the work machine 100, the event identifying unit 304 may identify the type of event as "abnormality detection."

[0100] In this embodiment, event type information that associates the above-mentioned event content with the event type may be stored in advance in the controller 30, and the event identification unit 304 may identify the type of event by referring to the event content detected by the event detection unit 303 and the event type information stored in the controller 30.

[0101] The association unit 305 associates the date and time when an event is detected, the type of the detected event, and the direction in which the event occurred.

[0102] Specifically, if the detected event is an event detected from sound data, the association unit 305 associates the direction of the sound source, the type of event (type of sound), and the date and time when the sound that constitutes the event was detected.

[0103] In other words, when a detected event is an event detected from sound data, the association unit 305 associates the direction of the sound source, information indicating the type of sound, and the date and time when the sound that constitutes the event was detected. The information indicating the type of sound may include information indicating the sound source. Specifically, the information indicating the type of sound may include "speech," "sound notifying of danger," "abnormality detection," etc.

[0104] If the type of event is an utterance, the association unit 305 may associate information that identifies the person who made the utterance with the type of event. In this way, by identifying the person who made the utterance and associating it with the type of event, the operator of the work machine 100 can easily recognize the person who made the utterance.

[0105] Furthermore, if the detected event is an event detected by an object detection device, the association unit 305 associates the direction in which the object was detected with the type of event, assuming that this is the direction in which the event occurred. In other words, the association unit 305 associates the direction in which the object was detected, the type of event, and the date and time when the object was detected. In this embodiment, by doing so, the operator of the work machine 100 can easily ascertain the direction in which the object was detected.

[0106] In the following description, the information associated by the association unit 305 may be referred to as event information. The event information includes information indicating the direction in which the event occurred, the type of the event, and the date and time when the occurrence of the event was detected.

[0107] The display control unit 306 controls the display of the display device D1. More specifically, when an event is detected and the type of the event is identified, the display control unit 306 causes the display device D1 to display a notification indicating that the event has occurred in a display mode according to the type of the event.

[0108] Furthermore, when an event occurs, the display control unit 306 causes the display device D1 to display event information. In other words, the display control unit 306 causes the display device D1 to display the occurrence of an event, the type of the event, and the direction in which the event occurred so that the user can visually recognize the event. A detailed example of the display on the display device D1 will be described later.

[0109] The information storage unit 307 stores collected sound data, image data acquired by the imaging device S6, operation information indicating the operation details of the work machine 100, and the like in a storage device or the like of the work machine 100. Specifically, the information storage unit 307 may store sound data, image data, operation information, and the like acquired during the period from a predetermined start timing to a predetermined end timing in the storage device together with event information. In the following description, the data stored in the storage device by the information storage unit 307 may be referred to as log information.

[0110] In addition, the information storage unit 307 of this embodiment may store in a storage device log information acquired during a period from a time that is a predetermined period before the time when the occurrence of an event was detected by the event detection unit 303 to a time that is a predetermined period after the end of the event detected by the event detection unit 303.

[0111] That is, the log information stored in the storage device by the information storage unit 307 includes sound data collected during a period included in the period from a predetermined start timing to a predetermined end timing. Note that the collection period of the log information (sound data) stored in the storage device does not have to coincide with the period from the predetermined start timing to the predetermined end timing.

[0112] Furthermore, the log information stored in the storage device by the information storage unit 307 may include sound data collected during the period from the time when the occurrence of an event is detected until the end of the event is detected.

[0113] Furthermore, when an event is detected during the period in which log information is collected, the information storage unit 307 stores the event information of the detected event in association with the log information in the storage device.

[0114] That is, when an event is detected, the information storage unit 307 of this embodiment associates log information acquired during the period in which the event occurred with event information of the detected event and stores them in the storage device. At this time, the information storage unit 307 may also associate log information acquired during a period including periods before and after the period in which the event occurred with the event information and store them in the storage device.

[0115] Furthermore, in this embodiment, if the type of event is speech, the text data converted from the sound data may be included as part of the log information. In other words, the log information may include the sound data, image data captured during the period when the sound data was collected, and operation information of the work machine 100 acquired during the period when the sound data was collected, as well as the text data corresponding to the sound data.

[0116] In this embodiment, by storing the event information and the log information in association with each other in the storage device, the function of a drive recorder can be realized.

[0117] Next, the interior of the cab 10 will be described with reference to Fig. 5. Fig. 5 is a top view of the interior of the cab 10. The work machine 100 is equipped with a driver's seat 50, an operating device 26, a display device D1, and the like, which are arranged inside the cab 10. A door for getting on and off is provided on the left side of the driver's seat 50. The operator can enter the cab 10 by opening the door for getting on and off.

[0118] The driver's seat 50 is located in the center of the driver's cab 10 in a top view. The driver's seat 50 includes a seat portion 51 on which the operator sits and a backrest 52. The driver's seat 50 is a reclining seat, and the tilt angle of the backrest 52 is adjustable. A left armrest 53L is located on the left side of the driver's seat 50, and a right armrest 53R is located on the right side. The left armrest 53L and the right armrest 53R are rotatably supported by the backrest 52.

[0119] A left console 54L is disposed on the left side of the driver's seat 50, and a right console 54R is disposed on the right side. The left console 54L and the right console 54R extend in the front-to-rear direction. The driver's seat 50 is slidable in the front-to-rear direction. The driver's seat 50 may be configured to be slidable in the front-to-rear direction together with the left console 54L and the right console 54R.

[0120] The left armrest 53L is disposed on top of the left console 54L. The right armrest 53R is disposed on top of the right console 54R. The left armrest 53L is disposed so as to cover a portion of the left console 54L in a top view. The right armrest 53R is disposed so as to cover a portion of the right console 54R in a top view.

[0121] The operating device 26 includes a left operating lever 26L, a right operating lever 26R, a left traveling pedal 26PL, a right traveling pedal 26PR, a left traveling lever 26DL, and a right traveling lever 26DR. The left operating lever 26L is provided in front of the left console 54L. Similarly, the right operating lever 26R is provided in front of the right console 54R. An operator seated in the driver's seat 50 can operate the left operating lever 26L while holding the left operating lever 26L with his left hand, and can operate the right operating lever 26R while holding the right operating lever 26R with his right hand. The operator seated in the driver's seat 50 can operate the left operating lever 26L with his left hand to drive the arm cylinder 8 and the swing hydraulic motor 2A. The operator seated in the driver's seat 50 can also operate the right operating lever 26R with his right hand to drive the boom cylinder 7 and the bucket cylinder 9. The bases of the left operating lever 26L and the right operating lever 26R are covered with lever boots 27, respectively.

[0122] The left travel pedal 26PL and the right travel pedal 26PR are located on the floor in front of the driver's seat 50. An operator seated in the driver's seat 50 can operate the left travel pedal 26PL with his left foot to drive the left travel hydraulic motor 2ML. Also, an operator seated in the driver's seat 50 can operate the right travel pedal 26PR with his right foot to drive the right travel hydraulic motor 2MR.

[0123] The left travel lever 26DL and the right travel lever 26DR are located between the left travel pedal 26PL and the right travel pedal 26PR in a top view. The left travel lever 26DL and the right travel lever 26DR extend upward from the floor in front of the driver's seat 50. An operator seated in the driver's seat 50 can drive the left travel hydraulic motor 2ML by gripping and operating the left travel lever 26DL with their left hand, similar to operating the left travel pedal 26PL. Furthermore, an operator seated in the driver's seat 50 can drive the right travel hydraulic motor 2MR by gripping and operating the right travel lever 26DR with their right hand, similar to operating the right travel pedal 26PR. Furthermore, the left travel lever 26DL and the right travel lever 26DR are positioned so that the operator can operate the left travel lever 26DL and the right travel lever 26DR simultaneously with one hand.

[0124] The display device D1 is provided in a location that is easily visible to the operator seated in the cab 10, and displays various information images under the control of the controller 30. In the illustrated example, the display device D1 is located on the front right side of the driver's seat 50, and is connected to the controller 30 via a dedicated line. The display device D1 displays various types of image information. The display device D1 includes a display screen that displays information such as the working conditions or operating state of the work machine 100. The operator seated in the driver's seat 50 can perform work using the work machine 100 while checking the various types of information displayed on the display device D1. The display device D1 may be provided with an input device D2.

[0125] The input device D2 is provided within reach of the operator seated in the driver's seat 50, accepts various operational inputs from the operator, and outputs signals corresponding to the operational inputs to the controller 30. The input device D2 includes a touch panel mounted on the display of the display device D1 that displays various information images, a knob switch provided at the tip of one or more levers of a plurality of operating levers included in the operating device 26, or a button switch, lever, toggle switch, rotary dial, or the like provided around the display device D1. A signal corresponding to the content of the operation on the input device D2 is taken into the controller 30.

[0126] A gate bar 55 is attached to the front surface of the front end of the left console 54L. The gate bar 55 operates in conjunction with the operation of a gate lock lever GL provided on the left console 54L. The gate bar 55 is attached to a frame inside the left console 54L so that it can be raised and lowered around an axis at the top end that extends in the left-right direction.

[0127] The gate lock lever GL is a mechanical input operation unit for switching between a state in which the work machine 100 can be operated by the operation device 26 (operable state) and a state in which the work machine 100 cannot be operated by the operation device 26 (inoperable state). In the illustrated example, the gate lock lever GL is configured so that the operator can switch between a first operation position that realizes the inoperable state and a second operation position that realizes the operable state. The controller 30 switches between the operable state and the inoperable state depending on the operation state of the gate lock lever GL. In the illustrated example, the controller 30 switches between the operable state and the inoperable state of the work machine 100 by electrically switching between a connected state and a disconnected state of the pilot line depending on the operation state of the gate lock lever GL.

[0128] When the gate lock lever GL is in the second operating position, the gate bar 55 is raised forward (passing prohibited state) so as to prevent the operator from passing through the boarding / exiting door, as shown in Fig. 5. On the other hand, when the gate lock lever GL is in the first operating position, the gate bar 55 is housed inside the left console 54L (passing permitted state) so as not to prevent the operator from passing through the boarding / exiting door.

[0129] With this configuration, the operator cannot operate the work machine 100 unless he or she sets the gate lock lever GL to the second operating position and puts the gate bar 55 in a passage-prohibiting state. Therefore, this configuration can prevent the work machine 100 from moving unintentionally, even if the operator inadvertently touches the operating device 26 when getting on or off the machine. Therefore, this configuration can improve the safety of the work machine 100.

[0130] Furthermore, the work machine 100 may be configured so that it can accept a predetermined operation to start the engine 11 only when the gate lock lever GL is in the second operating position and the gate bar 55 is in a pass-prohibiting state. In other words, the work machine 100 may be configured so that it cannot start the engine 11 when the gate lock lever GL is in the first operating position and the gate bar 55 is in a pass-permitting state.

[0131] A switch SW is installed on the right console 54R. A window console 56 is installed on the right side of the right console 54R. The window console 56 extends over the entire length of the driver's cab 10 in the fore-and-aft direction and is arranged parallel to the right console 54R. A display device D1 is installed in front of the window console 56. An external volume dial DL1, an internal volume dial DL2, an internal sound collector M2, a radio tuner, etc. are installed on the window console 56. The radio tuner, etc. may be installed on the left console 54L or the right console 54R.

[0132] The internal sound collection device M2 is a device that collects sounds generated inside the cab 10. In the illustrated example, the internal sound collection device M2 is an indoor microphone that is configured to be able to pick up the voice of an operator inside the cab 10.

[0133] The horn button HS is a button that is operated by the operator of the work machine 100 to sound the horn. In the illustrated example, the horn button HS is a knob switch provided at the tip of the left operation lever 26L.

[0134] The speech button KS is a button that the operator of the work machine 100 operates when he or she wants to speak to a worker around the work machine 100. In the illustrated example, the speech button KS is a knob switch provided at the tip of the right operating lever 26R.

[0135] The internal sound output device SP2 is a device that outputs sound to an operator in the cab 10 and is provided in the cab 10. The internal sound output device SP2 converts an electrical signal input from the controller 30 into a physical sound (air vibration) and outputs the sound. The internal sound output device SP2 may be provided in any position, for example, near the display device D1, near the input device D2, or near the boarding door of the cab 10. In the illustrated example, the internal sound output device SP2 includes a left interior speaker SP2L attached to the upper left corner of the rear wall of the cab 10 and a right interior speaker SP2R attached to the upper right corner of the rear wall of the cab 10. The internal sound output device SP2 may be headphones or earphones worn by the operator. In this case, the headphones or earphones are connected to the controller 30 so as to be able to communicate with the controller 30 via, for example, Bluetooth (registered trademark) or the like.

[0136] The external volume dial DL1 is configured to adjust the volume of the sound output by the external sound output device SP1. The volume of the sound output by each external sound output device SP1 may be additionally configured to be adjustable using a device other than the external volume dial DL1, such as a touch panel attached to the display device D1.

[0137] The external volume dial DL1 may be configured to be infinitely rotatable in both the clockwise and counterclockwise directions, in order to accommodate a case where volume adjustment using the external volume dial DL1 and volume adjustment using a device other than the external volume dial DL1 are used in combination.

[0138] The internal volume dial DL2 is configured to adjust the volume of the sound output by the internal sound output device SP2. The volume of the sound output by each internal sound output device SP2 may be additionally configured to be adjustable using a device other than the internal volume dial DL2, such as a touch panel attached to the display device D1.

[0139] The internal volume dial DL2 may be configured to be infinitely rotatable in both the clockwise and counterclockwise directions, in order to accommodate a case where volume adjustment using the internal volume dial DL2 and volume adjustment using a device other than the internal volume dial DL2 are used in combination.

[0140] The switch SW is an example of an operating device for switching the operating state of the conversation function. In the illustrated example, the switch SW is provided on the top surface of the right console 54R. However, the switch SW may be one of the input devices D2, may be realized by a touch panel provided on the display device D1, or may be a knob switch.

[0141] The conversation function is a function for realizing a conversation between an operator OP of the work machine 100 and a worker WK located around the work machine 100 as shown in FIG.

[0142] Fig. 6 is a perspective view of a work machine 100 with an operator OP on board and a worker WK standing on the front left of the work machine 100. Fig. 6 shows how the voice of the operator OP is collected by the internal sound collection device M2 and output from the external sound output device SP1, and how the voice of the worker WK is collected by the external sound collection device M1 and output from the internal sound output device SP2.

[0143] In addition, in the work machine 100 shown in FIG. 6, an information transmission device G1 is provided at the top of each of the four side surfaces of the cab 10. A front light bar G1F provided at the top of the front of the cab 10 emits green light, and a left light bar G1L provided at the top of the left surface of the cab 10 emits white light. In FIG. 6, the front light bar G1F emitting green light has a dot pattern attached. When a worker WK sees the front light bar G1F emitting green light, he or she can recognize that his or her voice is being detected by the front microphone M1F. Note that, for clarity, other devices such as the imaging device S6 are not shown in FIG. 6.

[0144] The operating state of the conversation function includes an ON state (the state shown in FIG. 6) in which conversation between the operator OP and the worker WK is possible, and an OFF state in which conversation between the operator OP and the worker WK is not possible. However, the operating state of the conversation function may additionally include at least one of an audible state (with respect to the operator OP) in which the operator OP can hear the worker WK's voice but the worker WK cannot hear the operator OP's voice, and an utterance enabled state (with respect to the operator OP) in which the worker WK can hear the operator OP's voice but the operator OP cannot hear the worker WK's voice.

[0145] Specifically, when the switch SW is operated to switch the operation state of the conversation function to the ON state, the external sound collection device M1, the external sound output device SP1, the internal sound collection device M2, and the internal sound output device SP2 become available. Conversely, when the switch SW is operated to switch the operation state of the conversation function to the OFF state, the external sound collection device M1, the external sound output device SP1, the internal sound collection device M2, and the internal sound output device SP2 become unavailable.

[0146] Furthermore, when the switch SW is operated to switch the operation state of the conversation function to a listening state, the external sound collection device M1 and the internal sound output device SP2 become available. Furthermore, when the switch SW is operated to switch the operation state of the conversation function to a speaking state, the external sound output device SP1 and the internal sound collection device M2 become available. In the illustrated example, the operator OP can use the external sound output device SP1 to talk to the worker WK by speaking while pressing the speaking button KS when the internal sound collection device M2 is available.

[0147] Next, the operation of the controller 30 of this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart illustrating the operation of the controller 30 of the work machine 100.

[0148] In this embodiment, the controller 30 starts collecting sound by the sound collection control unit 301 at a predetermined start timing (step S701). Subsequently, the controller 30 analyzes the collected sound data by the sound analysis unit 302 (step S702).

[0149] Next, the controller 30 determines whether or not an event has been detected by the event detection unit 303 (step S703). If an event has not been detected in step S703, the controller 30 proceeds to step S706, which will be described later.

[0150] If an event is detected by the event detection unit 303 in step S703, the controller 30 causes the event identification unit 304 to identify the type of the event (step S704).

[0151] Next, the controller 30 causes the association unit 305 to associate the content and type of the detected event with the direction of occurrence of the event obtained by analyzing the sound data in step S702 to generate event information, and displays the event information on the display device D1 (step S705).

[0152] Next, the display control unit 306 of the controller 30 determines whether a certain time has passed since the event information was displayed (step S706). If the certain time has not passed in step S706, the controller 30 waits.

[0153] In step S706, if a certain period of time has elapsed, the event information is hidden (step S707). Note that the certain period of time may be set in advance.

[0154] In this embodiment, the event information is displayed on the display device D1 for a certain period of time after the event is detected. In other words, in this embodiment, the information indicating the type of sound and the direction from which the sound was detected is displayed on the display device D1 for a certain period of time after the sound is detected. Therefore, in this embodiment, if another event is detected between the detection of an event at a certain timing and the passage of a certain period of time, multiple pieces of event information detected at different timings will be displayed on the display device D1.

[0155] In this embodiment, by continuing to display the event information for a certain period of time in this way, it is possible to allow the operator of the work machine 100 to grasp event information that has occurred in the past. Furthermore, in this embodiment, the event information is hidden after the certain period of time has elapsed, so it is possible to prevent a large amount of event information from being displayed on the display device D1, thereby preventing a decrease in visibility.

[0156] Next, the controller 30 determines whether or not it is the predetermined end timing through the sound collection control unit 301 (step S708). If it is the predetermined end timing in step S708, the controller 30 returns to step S702. If it is the predetermined end timing in step S708, the controller 30 causes the sound collection control unit 301 to end sound collection (step S709), and ends the process.

[0157] In this embodiment, when the event information is displayed on the display device D1 in step S705, the information storage unit 307 may start storing log information that associates the collected sound data, the image data acquired by the imaging device S6 during the period when the sound data was collected, and the operation information of the work machine 100 acquired by the imaging device S6 during the period when the sound data was collected with the event information.

[0158] Next, a display example of the display device D1 in this embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram illustrating a display example of the display device D1.

[0159] 8 is displayed on the image display unit 42 of the display device D1 while the work machine 100 is in operation. The image displayed on the image display unit 42 may be controlled by a control unit of the display device D1. The image displayed on the image display unit 42 may also be controlled by the controller 30 or the imaging device S6.

[0160] The display screen 85 shown in Figure 8 shows an example of the display on the display device D1 when, for example, the sound collection control unit 301 starts collecting sound data when the operation state of the conversation function is turned on, and the event detection unit 303 detects the speech of the worker WK as an event.

[0161] The control unit 40a according to this embodiment generates a display screen 85 based on the image information input from the imaging device S6 and various information received from the controller 30. The information received from the controller 30 includes the turning angle, information indicating the area in the image information in which a person is captured, position information of people present in the vicinity of the work machine (information indicating the direction and distance of the detected person), and detection results of various sensors.

[0162] In accordance with control from the control unit 40a, the image display unit 42 displays a display screen 85 including a date and time display area 42a, a driving mode display area 42b, an attachment display area 42c, a fuel efficiency display area 42d, an engine control status display area 42e, an engine operating time display area, a coolant temperature display area 42g, a remaining fuel amount display area 42h, an RPM level display area 42i, a urea water remaining amount display area 42j, a hydraulic oil temperature display area 42k, a work machine status display area 421, a first image display area 422, a second image display area 423, and a log display area 424. The display screen 85 may include other display areas.

[0163] The travel mode display area 42b, the attachment display area 42c, the engine control state display area 42e, and the rotation speed level display area 42i are areas that display setting state information, which is information related to the setting state of the work machine 100. The fuel efficiency display area 42d, the engine operating time display area, the coolant temperature display area 42g, the remaining fuel amount display area 42h, the remaining urea water amount display area 42j, and the hydraulic oil temperature display area 42k are areas that display operating state information, which is information that represents the operating state of the work machine 100 based on the detection results of various sensors.

[0164] The date and time display area 42a is an area that displays the current date and time. The driving mode display area 42b is an area that displays the current driving mode. The attachment display area 42c is an area that displays an image representing the currently attached attachment. The fuel efficiency display area 42d is an area that displays fuel efficiency information calculated by the controller 30. The fuel efficiency display area 42d includes an average fuel efficiency display area 42d1 that displays lifetime average fuel efficiency or section average fuel efficiency, and an instantaneous fuel efficiency display area 42d2 that displays instantaneous fuel efficiency.

[0165] The engine control status display area 42e is an area that displays the control status of the engine 11. The engine operating time display area is an area that displays the accumulated operating time of the engine 11. The coolant temperature display area 42g is an area that displays the current temperature state of the engine coolant. The remaining fuel amount display area 42h is an area that displays the remaining amount of fuel stored in the fuel tank.

[0166] The rotation speed level display area 42i is an area that displays the current level set by the dial 32 as an image. A number indicating the selected level is displayed in the rotation speed level display area 42i. The number "1" displayed in the rotation speed level display area 42i indicates that the selected rotation speed level is the "first level." The number "n" displayed in the rotation speed level display area 42i indicates that the selected rotation speed level is the "nth level." "n" is a natural number. When the operator rotates the dial 32, the number displayed in the rotation speed level display area 42i changes.

[0167] The urea water remaining amount display area 42j is an area that displays the remaining amount of urea water stored in the urea water tank using an image. The hydraulic oil temperature display area 42k is an area that displays the temperature of the hydraulic oil in the hydraulic oil tank.

[0168] The work machine status display area 421 is an area that displays information that indicates the positional relationship between the work machine 100 and a person detected in the vicinity of the work machine 100.

[0169] The work machine status display area (an example of a predetermined display area) 421 is a display area that displays real space at a predetermined scale with the work machine 100 at its center. A work machine icon 421b indicating the presence of the work machine 100 is placed in the center of the work machine status display area (an example of a predetermined display area) 421.

[0170] In the work machine status display area (an example of a predetermined display area) 421, in addition to a work machine icon (an example of first display information) 421b representing the work machine 100, an icon indicating the direction in which the work machine 100 can proceed (direction display icon 421a in the example of FIG. 8) and icons representing people detected around the work machine 100 (person detection icons 421e, 421f, 421g in the example of FIG. 8) are simultaneously displayed. In the work machine status display area 421, the area other than the work machine icon 421b, direction display icon 421a, and person detection icons 421e, 421f, 421g (in other words, the background) may be an area represented in a single color (for example, black).

[0171] The work machine icon (an example of first display information) 421b is an icon that combines an image showing the upper rotating body 3 and an image showing the lower running body 1, in accordance with the positional relationship between the upper rotating body 3 and the lower running body 1 based on the rotation angle.

[0172] The direction display icon (an example of third display information) 421a is triangular in shape and indicates the direction in which the work machine 100 will travel when the travel lever is tilted forward. Note that this embodiment shows only one example of an icon that indicates the direction in which the work machine 100 will travel when the travel lever is tilted forward, and any shape may be used as long as it indicates the direction in which the work machine 100 can travel.

[0173] The person detection icons (an example of second display information) 421e, 421f, 421g are icons that represent people detected from image information captured by the imaging device S6. Specifically, the person detection icons (an example of second display information) 421e, 421f, 421g are arranged based on position information of people received from the controller 30. For example, the person detection icons (an example of second display information) 421e, 421f, 421g are arranged at positions obtained by multiplying the direction and distance to the detected person, with the work machine 100 as the reference, by a predetermined scale.

[0174] In this way, the positional relationship between the work machine icon 421b and the person detection icons 421e, 421f, 421g corresponds to the positional relationship between the work machine 100 and the people present around the work machine 100 in real space.

[0175] Furthermore, in this embodiment, an icon indicating the work machine 100 (for example, work machine icon 421b), an icon indicating the direction in which the work machine 100 can travel (for example, direction display icon 421a), and an icon indicating a person detected around the work machine 100 (for example, person detection icons 421e, 421f, 421g) are displayed in the work machine state display area (an example of a predetermined display area) 421. In other words, the work machine state display area (an example of a predetermined display area) 421 is prevented from displaying objects other than people that are present around the work machine.

[0176] In other words, by referring to the work machine state display area 421, the operator can recognize the current situation of the work machine 100, the direction in which it can proceed, and the positional relationship between the work machine 100 and people present around it.

[0177] Furthermore, by referring to the work machine status display area 421, the operator can infer how the positional relationship between the work machine 100 and people present in the vicinity will change when the work machine 100 is moved. Furthermore, because the display of objects other than the work machine and people is suppressed in the work machine status display area 421, it is possible to prevent the operator from being drawn to other objects and forgetting the presence of people. Therefore, improved safety can be achieved.

[0178] In addition, the work machine status display area (an example of a predetermined display area) 421 displays a first circular area (an example of a first area) 421c and a second circular area (an example of a first area) 421d, which are determined based on the work machine 100 and are determined according to the distance from the work machine 100.

[0179] The first circular area (an example of the first area) 421c and the second circular area (an example of the first area) 421d shown in FIG. 8 are represented as circles that allow the operator to recognize the relative distance from the work machine 100, using the work machine icon 421b as a reference.

[0180] The first circular area 421c is information indicating the range in which the attachment AT of the work machine 100 currently swings, based on the detection results of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3.

[0181] The second circular area 421d is information indicating the range in which the attachment AT can turn when the attachment AT is extended to its maximum extent in the horizontal direction.

[0182] The image display unit 42 according to this embodiment displays a first circular area 421c and a second circular area 421d. That is, the positional relationship between the person and the attachment AT can be recognized based on the positional relationship between the first circular area 421c and the second circular area 421d and the person detection icons 421e, 421f, and 421g. This makes it easier for the operator to operate the attachment AT in a way that prevents the person from coming into contact with it, thereby improving safety.

[0183] Furthermore, the control unit 40a changes the display mode of the person detection icons 421e, 421f, and 421g depending on whether they are included in the first circular area (an example of the first area) 421c and the second circular area (an example of the first area) 421d.

[0184] For example, a person detection icon 421e that exists within the first circular area 421c is displayed in red, for example. A person detection icon 421f that exists outside the first circular area 421c but within the second circular area 421d is displayed in yellow, for example. A person detection icon 421g that exists outside the second circular area 421d is displayed in blue, for example. The control unit 40a according to this embodiment changes the display mode depending on whether the person is included in the first circular area (an example of the first area) 421c or the second circular area (an example of the first area) 421d; in other words, the color of the person detection icon changes depending on the distance from the work machine 100. In this way, the display device D1 can alert the operator to the distance between the work machine 100 and a person by displaying a person detection icon whose color changes depending on the distance. This makes it possible to achieve improved safety.

[0185] The color of the person detection icon in this embodiment is an example and is not limited to this color. For example, the person detection icon may be displayed in grayscale. Furthermore, the color change of the person detection icon in this embodiment is an example of a display mode and is not limited to this color change. For example, the blinking cycle of the person detection icon may be changed, or the contrast or brightness may be changed, depending on whether the person detection icon is included in the first circular area (an example of the first area) 421c or the second circular area (an example of the first area) 421d.

[0186] 8 shows an example in which one person detection icon is displayed in each of the first circular area (an example of the first area) 421c, the second circular area (an example of the first area) 421d that is outside the first circular area (an example of the first area) 421c and is also outside the second circular area (an example of the first area) 421d. However, multiple person detection icons may be displayed in each area depending on the person detection results. Furthermore, when multiple person detection icons exist in each area, the multiple person detection icons present in one area may be displayed in a distinguishable manner.

[0187] For example, when multiple people detection icons exist in each area, each person detection icon may be assigned a unique number and displayed. Furthermore, the shape of each person detection icon may be displayed so that it changes. This display also allows people present in each area to be uniquely recognized. Therefore, even when multiple people are present, the operator can operate the work machine 100 after recognizing each person, thereby achieving improved safety.

[0188] By referring to the work machine state display area 421, the operator can recognize how the positional relationship between the attachment AT of the work machine 100 and people present in the vicinity changes when the work machine 100 is turned, thereby achieving improved safety.

[0189] Note that, although this embodiment describes an example of displaying the first circular area 421c and the second circular area 421d, this embodiment is not limited to an example of displaying the first circular area 421c and the second circular area 421d, and either the first circular area 421c or the second circular area 421d may be displayed.

[0190] Furthermore, the work machine status display area (an example of a predetermined display area) 421 is not limited to a method of displaying a circular area indicating the range in which the attachment AT rotates in order to allow the distance between the work machine 100 and the person to be recognized, but may instead display, for example, a circular area indicating a predetermined distance (for example, 4 m) with the work machine 100 at its center.

[0191] Furthermore, the work machine state display area 421 may display an overhead image generated by combining image information captured by the imaging device S6 provided on the work machine 100.

[0192] The image display unit 42 displays a work machine status display area 421, as well as image information captured by the imaging device S6. By checking the image information along with the work machine status display area 421, the operator can recognize the specific situation around the work machine 100. This makes it possible to achieve improved safety.

[0193] Of the screen shown in FIG. 8, the first image display area 422 and the second image display area 423 are areas that display image information captured by the imaging device S6. The first image display area 422 displays a right-hand image. The second image display area 423 displays a rearward image. The right-hand image is an image that shows the space to the right of the work machine 100, and includes an image 422c of the right end of the top surface of the upper rotating body 3. The right-hand image is a real-viewpoint image generated by the control unit 40a, and is generated based on an image acquired by the right camera S6R. The rearward image is an image that shows the space behind the work machine 100, and includes an image 423c of the counterweight. The rearward image is a real-viewpoint image generated by the control unit 40a, and is generated based on an image acquired by the rear camera S6B.

[0194] The first image display area 422 is displayed to the right of the work machine status display area 421 as a reference. The second image display area 423 is displayed below the work machine status display area 421 as a reference. In this embodiment, the upper side of the image display unit 42 corresponds to the front of the upper rotating body 3. In other words, the second image display area 423 is displayed at a position corresponding to the rear of the work machine status display area 421 as a reference. In other words, the image display unit 42 displays image information captured by the imaging device S6 in the direction in which the image was captured, based on the work machine status display area 421. In this embodiment, image information captured in the direction in which the image was captured is displayed based on the work machine status display area 421. Therefore, when the operator refers to the image information, he or she can intuitively recognize in which direction the image information represents the situation. This makes it possible to achieve improved safety.

[0195] Note that this embodiment shows an example of the arrangement of image information, and is not limited to this arrangement. For example, the first image display area 422 and the second image display area 423 may be arranged regardless of the imaging direction.

[0196] When the controller 30 detects a person in at least one of the right image and the rear image, the controller 30 transmits information indicating the area in which the person is detected to the control unit 40a.

[0197] Then, based on the received information, the control unit 40a superimposes a frame showing information indicating the area where the person is detected on the right image and the rear image in which the person is detected. In other words, the control unit 40a performs a display to identify the image of the detected person in the right image and the rear image.

[0198] In the example shown in the figure, a frame 422b surrounding an image 422a of the detected person is displayed in the right image of the first image display area 422, and a frame 423b surrounding an image 423a of the detected person is displayed in the rear image of the second image display area 423.

[0199] The control unit 40a matches the color of the frame 422b of the right image in the first image display area 422 with the color of the person detection icon 421e, and also matches the color of the frame 423b of the rear image in the second image display area 423 with the color of the person detection icon 421f.

[0200] That is, the image display unit 42 displays a frame indicating a detected person in the right image and the rear image, and also displays a correspondence between the person indicated by the frame and the person detection icon in a manner that allows the person to be recognized. This allows the operator to recognize the situation of the person indicated in the work machine status display area 421 by referring to the right image and the rear image. Therefore, the operator can operate the work machine 100 while taking into consideration the situation of people present around the work machine 100. This can thereby achieve improved safety. Note that this embodiment has described an example in which the color of the frame matches the color of the person detection icon as an example of a display that allows the correspondence to be recognized. However, this embodiment shows only an example of a display that allows the correspondence to be recognized, and is not limited to a method in which the color of the frame matches the color of the person detection icon. For example, the correspondence may be recognized from the blinking cycle of the frame and the person detection icon.

[0201] As described above, the color of the human detection icon changes depending on the distance from the work machine 100. The human detection icon in the work machine status display area 421 and the color of the frame displayed on the right image and rear image are made to match. Therefore, the image display unit 42 displays the frame in different colors (an example of a display mode) on the right image and rear image based on the distance between the work machine 100 and the person. In this embodiment, the color of the frame displayed on the right image and rear image changes depending on the distance from the work machine 100, so the operator can recognize the distance from the work machine 100 by referring to the color of the frame. Therefore, the operator can perform operations according to that distance, thereby achieving improved safety.

[0202] In this embodiment, the control unit 40a displays a display screen 85 on the image display unit 42, allowing the operator to grasp the situation around the work machine 100.

[0203] In the example of FIG. 8, the controller 30 transmits event information of an event detected by the event detection unit 303 to the control unit 40a.

[0204] In the example of FIG. 8, the event information includes the event type being "utterance", the direction in which the event occurred being to the right, and "worker WK" which is information identifying the person who made the utterance.

[0205] Therefore, the control unit 40a causes the display device D1 to visibly display the type of event, the direction from which the event occurred, and the information identifying the person who made the utterance, which are included in the event information. In other words, the control unit 40a causes the display device D1 to visibly display the type of sound, the direction from which the sound occurred, and the information identifying the person who made the utterance.

[0206] In the example of Fig. 8, a balloon image 422d is displayed in the first image display area 422 of the display screen 85, and the balloon image 422d displays the type of event, "utterance," and information identifying the person who made the utterance, "worker WK." Also, in the example of Fig. 8, by displaying the balloon image 422d in the image of the periphery of the work machine 100, the operator of the work machine 100 can easily visually recognize the type of event that has occurred in the periphery of the work machine 100 and the direction in which the event has occurred.

[0207] The direction in which the event occurred may be displayed in the work machine status display area 421. Specifically, the control unit 40a may cause the display mode of the person detection icon corresponding to the person who made the utterance, among the person detection icons displayed in the work machine status display area 421, to differ from that of the other person detection icons.

[0208] In addition, in this embodiment, the display mode of the image corresponding to the person who made the utterance may be made different from the display mode of the images of other people.

[0209] 8, the image corresponding to the person who made the utterance is image 422a. Therefore, in this embodiment, image 422a or a facial portion (part of image 422a) in image 422a may be made to blink or light up. Also, in this embodiment, image 422a or the outline or frame 422b of the facial portion in image 422a may be made to blink or light up.

[0210] In the example of Figure 8, the person who spoke was located to the right of the work machine 100, so the first image display area 422 includes image 422a in the right image of the work machine 100 captured by the right camera S6R.

[0211] In this embodiment, for example, when an utterance by a person located on the left side of the work machine 100 is detected, the control unit 40a may cause an image taken by the left camera S6L capturing an image to the left of the person who made the utterance to be detected to be displayed in the first image display area 422. In other words, the control unit 40a may cause an image taken by the imaging device S6 in the direction in which the utterance was detected to be displayed in the first image display area 422.

[0212] In this embodiment, by doing this, the image of the person who made the utterance can be made to stand out, allowing the operator OP of the work machine 100 to easily recognize that a utterance has been made.

[0213] In this embodiment, text data converted from the sound data may be displayed in the log display area 424. In other words, the control unit 40a may cause the log display area 424 to display the content of the utterance.

[0214] In this embodiment, for example, when balloon image 422d is selected in first image display area 422, text data indicating the content of the utterance may be displayed in log display area 424. Furthermore, the content of the utterance may be displayed together with information indicating the time when the utterance was detected.

[0215] Furthermore, the log display area 424 may display, as a conversation history, the speech content of the workers WK around the work machine 100 and the speech content of the operator OP operating the work machine 100. In this case, the sound collection control unit 301 can use both the external sound collection device M1 and the internal sound collection device M2, and convert the sound data collected by the internal sound collection device M2 into text data.

[0216] In this embodiment, by displaying the content of each of the worker WK's and operator OP's speech in chronological order in the log display area 424, even if the operator OP misses what the worker WK said, he or she can go back through the conversation history and understand what the worker WK said.

[0217] 8, the text data indicating the utterance content is displayed in log display area 424, but this is not limiting. The text data indicating the utterance content may be displayed, for example, inside balloon image 422d. Furthermore, the text data indicating the utterance content may be displayed in a window that pops up near balloon image 422d in response to selection of balloon image 422d.

[0218] 8, the balloon image 422d is displayed in the first image display area 422, but this is not limiting. The balloon image 422d may be displayed near the person detection icon corresponding to the person who made the utterance, among the person detection icons displayed in the work machine state display area 421.

[0219] In this embodiment, the person who made the utterance, the content of the utterance, and the direction from which the utterance was made are displayed on the display device D1 so that the operator OP can visually recognize them. Also, in this embodiment, text data indicating the content of the utterance is displayed on the display device D1 in response to an operation by the operator OP.

[0220] Therefore, according to this embodiment, even if the operator OP misses a speech from a surrounding worker WK while operating the work machine 100, the operator OP can check the content of the speech from the worker WK. In other words, according to this embodiment, it is acceptable for the operator OP to miss a speech. Therefore, according to this embodiment, the burden on the operator OP can be reduced. Furthermore, according to this embodiment, the operator OP can concentrate on operating the work machine 100, thereby improving work efficiency and safety.

[0221] In the example of Figure 8, a case was described in which the type of detected event was "utterance," but in this embodiment, an icon image corresponding to the type of detected event (type of sound) may also be displayed on the display device D1.

[0222] For example, if the type of detected event is a "danger notification," an icon image corresponding to this event type may be displayed so that the operator OP can grasp the direction in which the event occurred.

[0223] Specifically, for example, the control unit 40a may superimpose and display an icon image corresponding to the event type "hazard notification" in the direction of the event occurrence relative to the work machine icon 421b in the work machine status display area 421. In this case, when an icon image is selected, the control unit 40a may display text data or the like indicating the type of event.

[0224] In this embodiment, the controller 30 may store icon images corresponding to the types of event information, and the display control unit 306 may instruct the control unit 40a to display an icon image corresponding to the type of event information.

[0225] In this embodiment, by displaying an icon image according to the type of event in the direction in which the event occurred, the operator OP can easily understand the type and direction of the event that has occurred.

[0226] Next, another example configuration of the work machine 100 will be described with reference to Fig. 9. Fig. 9 is a top view of another example configuration of the work machine 100. The work machine 100 shown in Fig. 9 differs from the work machine 100 shown in Fig. 1 in that the external sound output device SP1 is made up of four speakers (front speaker SP1F, left speaker SP1L, right speaker SP1R, and rear speaker SP1B). In the work machine 100 shown in Fig. 1, the external sound output device SP1 is made up of a single omnidirectional speaker provided above the cab 10.

[0227] With this configuration, the work machine 100 shown in Figure 9 can output sound toward the worker WK in front of the work machine 100 without outputting sound toward the workers WK on the left, right, and rear of the work machine 100, for example, by turning on the front speaker SP1F (a state in which sound can be output) and turning off the left speaker SP1L, the right speaker SP1R, and the rear speaker SP1B (a state in which sound cannot be output).

[0228] 9, a front camera S6F and a front microphone M1F are provided adjacent to the front speaker SP1F, and a front light bar G1F is provided on the housing of the front microphone M1F. A left camera S6L and a left microphone M1L are provided adjacent to the left speaker SP1L, and a left light bar G1L is provided on the housing of the left microphone M1L. A right camera S6R and a right microphone M1R are provided adjacent to the right speaker SP1R, and a right light bar G1R is provided on the housing of the right microphone M1R. A rear camera S6B and a rear microphone M1B are provided adjacent to the rear speaker SP1B, and a rear light bar G1B is provided on the housing of the rear microphone M1B.

[0229] With this configuration, the work machine 100 can turn on (light-emitting state) the light bar corresponding to the speaker that is turned on, and turn off (light-emitting state) the light bar corresponding to the speaker that is turned off.

[0230] The external sound output device SP1 may be configured with one or more parametric speakers. A parametric speaker is a speaker that uses ultrasonic waves and can selectively transmit sound to people within a specific narrow range. A parametric speaker can transmit sound toward any position.

[0231] In the work machine 100 shown in Fig. 9, the controller 30 may detect a worker WK around the work machine 100 based on images captured by the imaging device S6, and identify the position of the worker WK. Furthermore, if there are multiple workers WK around the work machine 100, the controller 30 may distinguish between a conversation target person (a worker WK who is speaking) and a non-conversation target person (a worker WK who is not speaking) based on the output of the four external sound collection devices M1. The controller 30 may also distinguish between a conversation target person (a worker WK facing the work machine 100) and a non-conversation target person (a worker WK who is not facing the work machine 100) based on images captured by the imaging device S6. The controller 30 may then turn on the speaker and light bar facing the worker WK. For example, if there is a worker WK (speaking) behind the work machine 100, the controller 30 may turn on the rear speaker SP1B while keeping the front speaker SP1F, left speaker SP1L, and right speaker SP1R off. In this case, the controller 30 may turn on the rear light bar G1B while keeping the front light bar G1F, left light bar G1L, and right light bar G1R off. Note that this type of function may be implemented in the work machine 100 shown in Figures 1 to 6.

[0232] With this configuration, the controller 30 can output sound in the direction where the worker WK is located without outputting sound in the direction where the worker WK is not present, so the worker WK can easily recognize whether he or she is considered a conversation target or non-target.

[0233] Next, an example configuration of the operation system SYS according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a schematic diagram showing an example configuration of the operation system SYS. As shown in Fig. 10, the operation system SYS includes a work machine 100, a remote control room RC, and a management center MC. Note that the detailed configuration of the work machine 100 is omitted from Fig. 10. This is because the work machine 100 shown in Fig. 10 has the same configuration as the work machine 100 shown in Fig. 1 or Fig. 9.

[0234] The work machine 100, remote control room RC, and management center MC are connected to one another so that they can send and receive data via a communication network NW. Note that the work machine 100, remote control room RC, and management center MC may also be connected to one another so that they can send and receive data directly to one another without going through the communication network NW. In the illustrated example, the work machine 100 transmits information about the work site to the remote control room RC. This allows the remote operator RO in the remote control room RC to understand the situation at the work site based on the information from the work machine 100.

[0235] Furthermore, the work machine 100 of this embodiment may transmit sound data collected by the external sound collection device M1 and image data acquired by the imaging device S6 to the remote control room RC.

[0236] The work machine 100 is equipped with a sensor that can three-dimensionally recognize the position and shape of objects present at the work site. For example, the work machine 100 is equipped with a spatial recognition device. Therefore, the work machine 100 can transmit the results of three-dimensionally measuring the work site to the remote control room RC.

[0237] The spatial recognition device is a device for recognizing the space around the work machine 100. In the illustrated example, the spatial recognition device is a LiDAR. The LiDAR measures the distance between the LiDAR and each of one million or more points within a monitoring range, for example. The spatial recognition device may be any device that can measure the distance to an object. For example, the spatial recognition device may be a stereo camera, or may be a combination of the image capture device S6 and a distance measuring device such as millimeter wave radar.

[0238] The operation system SYS may include one or more work machines 100. When multiple work machines 100 are included, the remote operator RO of a specific work machine 100 can obtain information about the work site obtained by that specific work machine 100, as well as information about the work site obtained by one or more other work machines 100.

[0239] The remote control room RC is equipped with a communication device T2, a remote controller 40, an operation device 37, an operation sensor 43, a display device D1E, an internal sound collection device M2E, and an internal sound output device SP2E. The remote control room RC also is equipped with an operation seat DS where a remote operator RO who remotely operates the work machine 100 sits.

[0240] The communication device T2 is configured to be able to communicate with the communication device T1 attached to the work machine 100.

[0241] The remote controller 40 is a computing device that executes various calculations. In this embodiment, the remote controller 40 is configured as a microcomputer including a CPU and a memory. The various functions of the remote controller 40 are realized by the CPU executing programs stored in the memory.

[0242] More specifically, the remote controller 40 may have a sound collection control unit 301, a sound analysis unit 302, an event detection unit 303, an event identification unit 304, an association unit 305, and a display control unit 306. In this case, the controller 30 of the work machine 100 does not need to be provided with these functions.

[0243] In this case, the remote controller 40 may execute the processing shown in Fig. 7, and when an event is detected, cause the display device D1E to display a screen equivalent to the display screen 85 shown in Fig. 8. In this way, the remote operator RO can easily grasp events that have occurred around the work machine 100.

[0244] The display device D1E is a device capable of displaying various types of information. The display device D1E displays images based on information transmitted from the work machine 100 so that the remote operator RO in the remote control room RC can visually recognize the surroundings of the work machine 100. In the illustrated example, the display device D1E is a liquid crystal display that displays images captured by an imaging device S6 mounted on the work machine 100. Note that the display device D1E may be a display or projector that realizes naked-eye stereoscopic vision, or may be VR goggles or the like.

[0245] The internal sound output device SP2E is a device capable of outputting various types of sound information. The internal sound output device SP2E outputs sound based on information transmitted from the work machine 100 so that the remote operator RO in the remote control room RC can hear sounds emitted at the work site. The internal sound output device SP2E may be configured to output sound captured by an external sound collection device M1 attached to the outside of the cab 10, for example, or may be configured to output sound captured by an internal sound collection device M2 attached to the inside of the cab 10.

[0246] In this case, the internal sound collection device M2 may be provided at a position corresponding to the position of the ears of an operator seated in the driver's seat 50 in the driver's cab 10. The internal sound output device SP2E may be a stationary device such as a speaker, or may be a wearable device such as earphones or headphones. The speaker may be a monaural speaker, a stereo speaker, or a surround speaker. The speaker may also be an omnidirectional speaker or a directional speaker. The wearable device may have a noise canceling function, a spatial audio function (stereophonic function), or a bone conduction function.

[0247] The operating device 37 is provided with an operation sensor 43 for detecting the operation of the operating device 37. The operation sensor 43 is, for example, an inclination sensor that detects the inclination angle of the operating lever, or an angle sensor that detects the swing angle of the operating lever around the swing axis. The operation sensor 43 may be composed of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor.

[0248] The operation sensor 43 outputs information relating to the detected operation content of the operation device 37 to the remote controller 40. The remote controller 40 generates an operation signal based on the received information and transmits the generated operation signal to the work machine 100. The operation sensor 43 may be configured to generate the operation signal. In this case, the operation sensor 43 may output the operation signal to the communication device T2 without going through the remote controller 40. With this configuration, the remote operator RO can remotely operate the work machine 100 from the remote control room RC.

[0249] The management center MC is a facility where various devices are installed to manage the work machine 100 at the work site or the remote operation of the work machine 100 by the remote operator RO in the remote control room RC. In the illustrated example, the management center MC is installed in a location away from both the work site of the work machine 100 and the remote control room RC. The management center MC is also equipped with a management device 200, an internal sound collection device M2C, and an internal sound output device SP2C.

[0250] The management device 200 is an example of a control device, and is, for example, a server computer (a so-called cloud server) or an edge server. The management device 200 is typically a fixed terminal device, but may also be a portable terminal device (for example, a laptop computer, a tablet, or a smartphone).

[0251] With this configuration, a manager at the management center MC can, for example, use a sound collection device (external sound collection device M1 or internal sound collection device M2) attached to the work machine 100 and the internal sound output device SP2C to listen to sounds emitted at the work site. The manager at the management center MC can also, for example, use an internal sound collection device M2E and internal sound output device SP2C provided in the remote control room RC to listen to sounds emitted in the remote control room RC. The manager at the management center MC can, for example, use the internal sound collection device M2C and the external sound output device SP1 attached to the work machine 100 to communicate the voice he or she makes to a worker WK who is around the work machine 100. The manager at the management center MC can, for example, use the internal sound collection device M2C and the internal sound output device SP2 attached to the work machine 100 to communicate the voice he or she makes to an operator OP of the work machine 100. In addition, an administrator at the management center MC can, for example, use an internal sound collection device M2C and an internal sound output device SP2E installed in the remote control room RC to transmit the voice he or she makes to the remote operator RO in the remote control room RC.

[0252] In this embodiment, an utterance by the administrator may also be detected as an event. In this embodiment, when an utterance by the administrator is detected as an event, text data indicating the content of the utterance by the administrator may be included in the log information.

[0253] In FIG. 10, the operation system SYS includes the work machine 100, the remote control room RC, and the management center MC, but in this embodiment, the operation system SYS may also include a work machine management system including the work machine 100 and the management device 200.

[0254] In this case, the management device 200 may have a sound analysis unit 302, an event detection unit 303, an event identification unit 304, a correspondence unit 305, and an information storage unit 307, and the sound collection control unit 301 and the display control unit 306 may be included in the controller 30 of the work machine 100.

[0255] In other words, in the management system for the work machine 100, the work machine 100 may transmit information used to detect events and information to be included in the log information to the management device 200, and the management device 200 may perform processing such as analysis of sound data, detection of events, and association of the event type with the direction in which the event occurred.

[0256] In this case, the management device 200 may cause the display device D1 of the work machine 100 to display a confirmation screen for obtaining consent for transmitting the information used to detect an event and the information to be included in the log information to the management device 200. When an operation indicating consent is performed on the confirmation screen, the work machine 100 may transmit to the management device 200 the information used to detect an event and the information to be included in the log information.

[0257] Furthermore, the management device 200 can notify the user of the management device 200 (the manager of the work machine 100) whether or not an operation indicating consent has been performed on the work machine 100.

[0258] When an event is detected using various information acquired from the work machine 100, the management device 200 may transmit event information to the work machine 100 and display the event information on the display device D1 of the work machine 100. The management device 200 may also acquire sound data via the remote controller 40 and display the event information on the display device D1E in the remote control room RC.

[0259] Furthermore, the management device 200 may store the event information and log information in association with each other in the storage device using the information storage unit 307. Note that the management device 200 may store the event information and log information in the storage device in association with the machine identification number that identifies the work machine 100.

[0260] In this embodiment, by doing this, even when there are multiple work machines 100 managed by the management device 200, event information and log information can be stored in the storage device for each work machine 100.

[0261] Therefore, according to this embodiment, the manager of the work machine 100 can ascertain, for each work machine 100, what events have occurred during work using the work machine 100 and what conversations have taken place between the worker WK and the operator OP. Furthermore, in this embodiment, by having the management device 200 perform processes such as analyzing sound data and detecting events, the processing load on the controller 30 of the work machine 100 can be reduced.

[0262] In the above description, the management device 200 includes the sound analysis unit 302, the event detection unit 303, the event identification unit 304, the association unit 305, and the information storage unit 307, but is not limited to this.

[0263] The management device 200 may have, for example, only the information storage unit 307. In this case, the work machine 100 only needs to transmit event information and log information to the management device 200. In this way, the communication load between the work machine 100 and the management device 200 can be reduced.

[0264] As such, the work machine 100 of this embodiment has an upper rotating body 1, a lower running body 2, a cab 10 provided on the upper rotating body 1, an external sound collection device M1 arranged outside the cab 10, an internal sound output device M2 arranged inside the cab 10 and outputting the sound collected by the external sound collection device M1, and a controller 30 that identifies the type of sound collected by the external sound collection device M1 and notifies the operator of the work machine of information indicating the identified type of sound.

[0265] This configuration allows the operator of the work machine 100 to easily recognize the direction from which the sound is coming and the type of sound, eliminating the need to always carefully listen to the collected sounds, thereby reducing the workload.

[0266] Furthermore, the controller 30 causes the display device D1 to continuously display information indicating the type of sound for a certain period of time after the sound is detected. With this configuration, for example, even if the operator misses a speech from a nearby worker WK while operating the work machine 100, the contents of the speech from the worker WK can be confirmed.

[0267] Furthermore, the controller 30 identifies the direction from which the sound originated based on the collected sound, and displays the identified direction in association with information indicating the type of sound on the display device D1. With this configuration, the operator of the work machine 100 can visually recognize the occurrence of an event, the type of event, and the direction from which the event occurred.

[0268] Furthermore, during the period when sound is being collected by the external sound collection device M1, the controller 30 causes the display device D1 to display the image data acquired by the imaging device S6 together with information indicating the type of sound. This configuration allows the operator of the work machine 100 to easily visually identify the type and direction of sound generated around the work machine 100.

[0269] Furthermore, in this embodiment, the sound collected by the external sound collection device M1 is the voice of a worker around the work machine 100, and the type of sound may be the speech of a worker around the work machine 100. This configuration allows the operator of the work machine 100 to easily recognize the speech around the work machine 100.

[0270] The controller 30 also stores voice data for each worker in advance, and identifies the worker who made the utterance based on the sound collected by the external sound collection device M1 and the voice data for each contractor. This configuration allows the operator of the work machine 100 to easily recognize the person who made the utterance.

[0271] Furthermore, the controller 30 identifies the direction in which the worker who made the utterance is located based on the sound collected by the external sound collection device M1, and displays information indicating the worker who made the utterance in association with the direction in which the worker who made the utterance is located on the display device D1. This configuration allows the operator of the work machine 100 to easily visually identify the worker who has appeared in the vicinity of the work machine 100 and the direction in which the utterance was made.

[0272] Furthermore, the work machine 100 of this embodiment has an internal sound collection device M2 placed inside the driver's cab, and the controller 30 associates text data converted from the voices of workers around the work machine 100 with text data converted from the voice of the operator of the work machine 100 collected by the internal sound collection device M2, and displays the text data on the display device D1.

[0273] With this configuration, even if the operator of the work machine 100 misses what the surrounding worker has said, the operator of the work machine 100 can confirm what the worker has said.

[0274] Furthermore, in this embodiment, the information indicating the type of sound is an icon image corresponding to the type of sound, and is displayed superimposed on the image data acquired by the imaging device S6. This configuration allows the operator of the work machine 100 to easily grasp the type and direction of the sound that has occurred.

[0275] Furthermore, the controller 30 stores in the storage device log information including the sound, image data captured by the imaging device S6 during the period when the sound is being collected, and operation information of the work machine 100 acquired during the period when the sound is being collected. With this configuration, the function of a drive recorder can be realized.

[0276] The work machine 100 is also provided with multiple external sound collection devices M2. This configuration enables the function of a drive recorder to be realized. Therefore, the controller 30 can detect the direction of the sound source based on differences in the collected sounds (for example, phase shifts or volume differences).

[0277] The work machine 100 also has an internal sound collection device M2 located inside the cab 10, and an external sound output device SP1 located outside the cab 10 and outputting sound collected by the internal sound collection device M2. With this configuration, the operator of the work machine 100 can use the external sound output device SP1 to talk to the worker.

[0278] The work machine 100 is also equipped with a talk button, and when the talk button is pressed, the internal sound collection device M2 collects sound, and the external sound output device SP1 outputs the sound collected by the internal sound collection device M2 toward the surroundings of the work machine. With this configuration, the operator of the work machine 100 can talk to workers around the work machine 100 by pressing the talk button when they want to speak.

[0279] Furthermore, the work machine 100 of this embodiment identifies the type of sound detected based on the sound collected by the external sound collection device M1, and notifies the operator in the remote control room of information indicating the identified type of sound. This configuration allows the remote operator to easily understand the sounds and types of sounds occurring around the work machine 100.

[0280] Furthermore, the controller 30 of the work machine 100 in this embodiment transmits log information acquired during the period when sound is being collected by the external sound collection device M1 to the management device 200. Furthermore, the management device 200 stores the log information received from the work machine 100 in a storage device. This configuration allows the manager of the management device 200 to understand events that have occurred around the work machine 100 when sound is detected by the work machine 100.

[0281] The preferred embodiments of the present invention have been described above in detail. However, the present invention is not limited to the above-described embodiments. Various modifications, substitutions, etc. may be applied to the above-described embodiments without departing from the scope of the present invention. Furthermore, features described separately may be combined unless technical contradictions arise. [Explanation of symbols]

[0282] 1 Undercarriage 2. Swivel mechanism 3 Upper rotating body 4. Boom 5 Arm 6 buckets 30 Controllers D1 display device 100 Work Machinery 301 Sound collection control unit 302 Sound Analysis Department 303 Event detection unit 304 Event Specific Department 305 Mapping Section 306 Display control unit 307 Information Storage Unit

Claims

1. An upper rotating body; a lower running body; an operator's cab provided on the upper rotating body; an external sound collecting device disposed outside the driver's cab; an internal sound output device that is arranged in the driver's cab and outputs the sound collected by the external sound collection device; a control device that identifies the type of sound collected by the external sound collecting device and notifies an operator of the work machine of information indicating the identified type of sound.

2. The control device 2. The work machine according to claim 1, wherein the information indicating the type of sound is continuously displayed on a display device for a predetermined time period after the sound is detected.

3. The control device 3. The work machine according to claim 1, wherein the direction from which the sound originated is identified based on the collected sound, and the identified direction is associated with information indicating the type of sound and displayed on a display device.

4. An imaging device is provided on the upper rotating body, The control device The work machine according to claim 1 , wherein image data acquired by the imaging device is displayed on a display device together with information indicating the type of the sound during a period in which the sound is collected by the external sound collecting device.

5. 2. The work machine according to claim 1, wherein the sound collected by the external sound collection device is the voice of a worker around the work machine, and the type of sound is speech of the worker around the work machine.

6. The control device Voice data for each worker is stored in advance, The work machine according to claim 5 , wherein the worker who made the utterance is identified based on the sound collected by the external sound collection device and the voice data for each worker.

7. The control device Identifying the direction in which the worker who made the utterance is located based on the sound collected by the external sound collection device; The work machine according to claim 6, wherein the information indicating the worker who made the utterance and the direction in which the worker who made the utterance is located are displayed on a display device in association with each other.

8. an internal sound collecting device disposed in the cab; The control device 8. The work machine according to claim 7, wherein text data converted from the voices of workers around the work machine and text data converted from the voice of the operator of the work machine collected by the internal sound collection device are displayed in association with each other on the display device.

9. The control device the information indicating the type of sound is an icon image corresponding to the type of sound, The work machine according to claim 4 , wherein the image data is displayed superimposed on the image data acquired by the imaging device.

10. The control device 2. The work machine according to claim 1, wherein log information including the sound, image data captured by an imaging device during the period in which the sound is collected, and operation information of the work machine acquired during the period in which the sound is collected is stored in a storage device.

11. The work machine according to claim 1 , wherein a plurality of the external sound collecting devices are provided.

12. an internal sound collecting device disposed in the cab; The work machine according to claim 1 , further comprising: an external sound output device disposed outside the operator's cab and configured to output the sound collected by the internal sound collection device.

13. Equipped with a talk button, When the talk button is pressed, the internal sound collection device collects sound, and the external sound output device outputs the sound collected by the internal sound collection device toward the periphery of the work machine.

13. The work machine according to claim 12.

14. an external sound collection device that collects sounds around the work machine; an external device that remotely controls the work machine; a sound output device that outputs a sound to an operator of the external device; a control device that identifies the type of sound detected based on the sound collected by the external sound collecting device and notifies the operator of the external device of information indicating the identified type of sound.

15. A work machine management system having a work machine and a management device that manages the work machine, The work machine includes: An upper rotating body; a lower running body; an operator's cab provided on the upper rotating body; an external sound collecting device disposed outside the driver's cab; an internal sound output device that is arranged in the driver's cab and outputs the sound collected by the external sound collection device; a control device that transmits log information acquired during a period in which sound is collected by the external sound collection device to the management device; The management device A work machine management system that stores the log information received from the work machine in a storage device.

Images