Work machine and operation system for work machine

Abstract

This work machine comprises: a lower traveling body; an upper turning body that is mounted on the lower traveling body so as to be able to turn; a driver's cab that is provided to the upper turning body; an external sound collection device that is positioned outside the driver's cab; and an internal sound output device that is positioned inside the driver's cab. The work machine has an external sound playback function that makes it possible for the internal sound output device to output sound that is collected by the external sound collection device, and is configured so that the external sound playback function is switched to an effective state according to conditions that are based on surrounding persons.

Description

Construction machine and operating system of construction machine

[0001] The present disclosure relates to a construction machine and an operating system of a construction machine.

[0002] Conventionally, a construction machine equipped with a microphone for collecting ambient sound is known. For example, Patent Document 1 discloses an excavator having a sound collection device disposed outside the cab for collecting sounds around the cab, and a control device for controlling the operation of the excavator based on voice data generated by removing noise from the sounds collected by the sound collection device.

[0003] Japanese Patent Application Laid-Open No. 2019-167680

[0004] There is a demand to let the operator present in the driver's cab hear the sounds generated around the construction machine. However, since it is burdensome for the operator to continuously listen to the sounds around the construction machine, the operator may disable the function of reproducing the sounds around the construction machine.

[0005] One aspect of the present disclosure aims to provide a construction machine capable of reducing the overlooking of sounds around the construction machine.

[0006] A construction machine according to one aspect of the present disclosure includes a lower traveling body, an upper revolving body rotatably mounted on the lower traveling body, a cab provided on the upper revolving body, an external sound collection device disposed outside the cab, and an internal sound output device disposed inside the cab, and has an external sound reproduction function that enables the sound collected by the external sound collection device to be output by the internal sound output device, and is configured such that the external sound reproduction function switches to an active state according to conditions based on surrounding people.

[0007] According to one aspect of the present disclosure, the overlooking of sounds around the construction machine can be reduced.

[0008] Figure 1 is a side view of a work machine according to an embodiment of the present disclosure. Figure 2 is a top view of the work machine shown in Figure 1. Figure 3 is a diagram showing an example of the configuration of an external sound collection device and an information transmission device attached to the work machine shown in Figure 1. Figure 4 is a schematic diagram showing an example of the configuration of the work machine shown in Figure 1. Figure 5 is a top view of the operator's cabin of the work machine shown in Figure 1. Figure 6 is a perspective view of the work machine with an operator on board and the workers around it. Figure 7 is a block diagram showing an example of a controller according to an embodiment of the present disclosure. Figure 8 is a diagram showing an example of conditions based on people in the surroundings. Figure 9 is a diagram showing an example of pre-actions performed by people in the surroundings. Figure 10 is a diagram showing an example of a display screen displayed on a display device. Figure 11 is a flowchart showing an example of the flow of state switching processing for the external sound playback function. Figure 12 is a top view of a work machine showing another example of the configuration of a work machine according to an embodiment of the present disclosure. Figure 13 is a schematic diagram showing an example of the configuration of an operating system according to an embodiment of the present disclosure.

[0009] Embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are illustrative and do not limit the invention. Not all features and combinations thereof in the embodiments of this disclosure are necessarily essential to the invention. In each drawing, the same or corresponding components are denoted by the same or corresponding reference numerals, and redundant descriptions may be omitted.

[0010] The working machine 100 according to the embodiment of this disclosure is a shovel. The working machine 100 may be a machine other than a shovel, such as a crane, an asphalt finisher, or a forklift. In the illustrated example, the shovel as the working machine 100 is an excavator equipped with a bucket 6 as an end attachment, but it may be an applied machine such as a forestry machine equipped with an end attachment other than a bucket 6.

[0011] [Overview of the Machinery] First, an overview of the machinery 100 will be explained with reference to Figures 1 and 2. Figure 1 is a side view of the machinery 100, and Figure 2 is a top view of the machinery 100.

[0012] In Figure 1, +X represents one direction of the X-axis in the three-dimensional Cartesian coordinate system, and -X represents the other direction of the X-axis. In Figure 2, +Y represents one direction of the Y-axis in the three-dimensional Cartesian coordinate system, and -Y represents the other direction of the Y-axis. In Figure 1, +Z represents one direction of the Z-axis in the three-dimensional Cartesian coordinate system, and -Z represents the other direction of the Z-axis. In Figure 1, the +X side of the work machine 100 corresponds to the front side of the work machine 100, and the -X side of the work machine 100 corresponds to the rear side of the work machine 100. Also, the +Y side of the work machine 100 corresponds to the left side of the work machine 100, and the -Y side of the work machine 100 corresponds to the right side of the work machine 100. Furthermore, the +Z side of the work machine 100 corresponds to the top side of the work machine 100, and the -Z side of the work machine 100 corresponds to the bottom side of the work machine 100. The same applies to the other figures.

[0013] The work machine 100 comprises a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a slewing mechanism 2, an attachment AT for performing various tasks, and a driver's cab 10.

[0014] The driver's cab 10 is also called a cabin or cab. A windshield is provided at the front of the driver's cab 10 so that the operator can see the work site. The front of the work machine 100 (upper slewing body 3) corresponds to the side on which the attachment AT is attached to the upper slewing body 3 when the work machine 100 is viewed from directly above along the slewing axis of the upper slewing body 3. The left, right, and rear sides of the work machine 100 (upper slewing body 3) correspond to the left, right, and rear sides, respectively, as seen from the perspective of the operator seated in the driver's seat inside the driver's cab 10.

[0015] The operator's cab 10 is a compartment where the operator sits and is located on the front left side of the upper rotating body 3. However, the operator's cab 10 may be omitted if the work machine 100 is remotely controlled or if the work machine 100 operates by fully automatic operation.

[0016] The lower travel body 1 includes, for example, a pair of left and right crawlers 1C. Specifically, the crawlers 1C include a left crawler 1CL and a right crawler 1CR. The left crawler 1CL is driven by a left travel hydraulic motor 2ML, and the right crawler 1CR is driven by a right travel hydraulic motor 2MR. The left travel hydraulic motor 2ML is a travel drive unit that drives the left crawler 1CL, which is the driven part, and can rotate the left crawler 1CL. The right travel hydraulic motor 2MR is a travel drive unit that drives the right crawler 1CR, which is the driven part, and can rotate the right crawler 1CR. Note that the travel drive units may also be electric motors.

[0017] Figure 1 illustrates a lower traveling body 1 having a pair of left and right crawlers, but the work machine 100 is not limited to a crawler-type excavator. The work machine 100 may also be a wheeled excavator including a lower traveling body 1 with multiple tires.

[0018] The upper rotating body 3 rotates relative to the lower traveling body 1 when the rotation mechanism 2 is driven by the rotation hydraulic motor 2A. The rotation hydraulic motor 2A is a rotation drive unit that drives the upper rotating body 3 as the driven part, and can change the orientation of the upper rotating body 3. The rotation drive unit may also be an electric motor.

[0019] A boom 4 is rotatably mounted to the front center of the upper slewing body 3, an arm 5 is rotatably mounted to the tip of the boom 4, and a bucket 6 is rotatably mounted to the tip of the arm 5. In the illustrated example, the boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of attachment AT. The boom 4, arm 5, and bucket 6 are driven by a boom cylinder 7, arm cylinder 8, and bucket cylinder 9, respectively.

[0020] Bucket 6 is an example of a work tool (end attachment). Bucket 6 is used, for example, for excavation work. Depending on the work content, other work tools may be attached to the tip of arm 5 instead of bucket 6. Other work tools may be other types of buckets, such as large buckets, slope buckets, or dredging buckets. Other work tools may also be types of work tools other than buckets, such as agitators, breakers, grapples, or lifting magnets.

[0021] The slewing hydraulic motor 2A, the left travel hydraulic motor 2ML, the right travel hydraulic motor 2MR, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9 are hydraulic actuators driven by hydraulic fluid discharged from a hydraulic pump.

[0022] Furthermore, the work machine 100 may have all or part of its driven parts, such as the lower traveling body 1, upper slewing body 3, boom 4, arm 5, and bucket 6, electrically driven. In other words, the work machine 100 may be a hybrid work machine or an electric work machine in which all or part of its driven parts are driven by electric actuators.

[0023] Furthermore, the work machine 100 is equipped with an information transmission device G1, an external sound collection device M1, an imaging device S6, and an external sound output device SP1.

[0024] The imaging device S6 is installed in the upper rotating body 3 or the operator's cab 10 and captures images of the area around the work machine 100, acquiring image information representing the area around the work machine 100. In the illustrated example, the imaging device S6 includes a front camera S6F, a left camera S6L, a right camera S6R, and a rear camera S6B.

[0025] The front camera S6F is a camera that captures images in front of the work machine 100 and is mounted on the outside of the operator's cab 10, such as on the roof of the operator's cab 10 or on the side of the boom 4. Alternatively, the front camera S6F may be mounted inside the operator's cab 10, such as on the ceiling of the operator's cab 10. The left camera S6L is a camera that captures images to the left of the work machine 100, the right camera S6R is a camera that captures images to the right of the work machine 100, and the rear camera S6B is a camera that captures images behind the work machine 100. Specifically, the front camera S6F, left camera S6L, right camera S6R, and rear camera S6B are all monocular wide-angle cameras equipped with image sensors such as CCD (Charge Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor), and they output the captured images to the display device D1. The image information captured by the imaging device S6 is also received by the controller 30.

[0026] In the illustrated example, the front camera S6F is mounted on the roof of the driver's cab 10, the left camera S6L is mounted on the upper left end of the upper surface of the upper rotating body 3, the right camera S6R is mounted on the upper right end of the upper surface of the upper rotating body 3, and the rear camera S6B is mounted on the upper rear end of the upper surface of the upper rotating body 3.

[0027] The imaging device S6 may be configured as an object detection device for detecting objects around the work machine 100. These objects may be, for example, people, animals, vehicles, construction machinery, buildings, or holes. The object detection device may be configured to distinguish between people and other objects. That is, the object detection device may be configured to function as a person detection device. The object detection device may consist of devices other than a camera. For example, the object detection device may be a LiDAR (Light Detection And Ranging). A LiDAR is, for example, a device capable of measuring the distance between a point cloud of 1 million or more points within a monitoring range and a LiDAR (laser source). Alternatively, the object detection device may be another device capable of measuring the distance to an object, such as a stereo camera, a depth image camera, or a millimeter-wave radar. When a millimeter-wave radar or the like is used as the object detection device, the object detection device may derive the distance and direction of an object by transmitting a large number of signals (such as laser light) towards the object and receiving the reflected signals. Alternatively, the object detection device may be a combination of two or more devices. For example, the object detection system may be a combination of an imaging device and a LiDAR, an imaging device and a millimeter-wave radar, or an imaging device and a stereo camera.

[0028] The imaging device S6 may consist of at least one of the following: a monocular camera, a stereo camera, a depth image camera, a LiDAR, or a millimeter-wave radar. For example, the imaging device S6 may consist of any combination of the following: a monocular camera only, a stereo camera only, a depth image camera only, a combination of LiDAR and a monocular camera, a combination of LiDAR and a stereo camera, a combination of LiDAR and a depth image camera, a combination of millimeter-wave radar and a monocular camera, a combination of millimeter-wave radar and a stereo camera, a combination of millimeter-wave radar and a depth image camera, etc.

[0029] The external sound collection device M1 is a device that collects external sounds and is also called a microphone. In the illustrated example, the external sound collection device M1 functions as an information acquisition device (IAD) that acquires information to determine the magnitude of the background noise generated around the work machine 100. Background noise refers to all sounds that are simultaneously generated elsewhere, for example, when focusing on a sound from a specific source. In the illustrated example, background noise refers to sounds other than those output by the external sound output device SP1, and includes the operating sounds of the work machine 100, traffic noise, living noise, or urban noise. Specifically, the external sound collection device M1 is installed in the upper rotating body 3 or the operator's cab 10 and converts sounds (air vibrations) generated around the work machine 100 into mechanical vibrations, and then converts those mechanical vibrations into electrical signals. Specifically, the external sound collection device M1 is configured to pick up the voices of workers around the work machine 100 and includes a front microphone M1F, a left microphone M1L, a right microphone M1R, and a rear microphone M1B.

[0030] The front microphone M1F is a microphone that collects sound generated in front of the work machine 100 and is mounted on the outside of the operator's cab 10, such as on the roof of the operator's cab 10 or on the side of the boom 4. Alternatively, the front microphone M1F may be mounted inside the operator's cab 10, such as on the ceiling of the operator's cab 10. The left microphone M1L is a microphone that collects sound generated to the left of the work machine 100, the right microphone M1R is a microphone that collects sound generated to the right of the work machine 100, and the rear microphone M1B is a microphone that collects sound generated behind the work machine 100. The electrical signals generated by the front microphone M1F, left microphone M1L, right microphone M1R, and rear microphone M1B are taken up by the controller 30.

[0031] In the illustrated example, the front microphone M1F is mounted on the roof of the driver's cab 10, the left microphone M1L is mounted on the upper left end of the upper surface of the upper rotating body 3, the right microphone M1R is mounted on the upper right end of the upper surface of the upper rotating body 3, and the rear microphone M1B is mounted on the upper rear end of the upper surface of the upper rotating body 3. In this way, the four external sound collection devices M1 (front microphone M1F, left microphone M1L, right microphone M1R, and rear microphone M1B) are installed at different positions on the upper rotating body 3. Therefore, the controller 30 can detect the direction of the sound source based on the differences in sound collected by each of the four external sound collection devices M1 (for example, differences in volume). Furthermore, if array microphones are used as the external sound collection devices M1, the direction of the sound source can be detected based on, for example, a phase shift or a difference in volume.

[0032] In the illustrated example, each of the four external sound collection devices M1 and each of the four imaging devices S6 are arranged to correspond to one another. Specifically, the front microphone M1F is positioned adjacent to the front camera S6F, the left microphone M1L is positioned adjacent to the left camera S6L, the right microphone M1R is positioned adjacent to the right camera S6R, and the rear microphone M1B is positioned adjacent to the rear camera S6B.

[0033] The external sound output device SP1 is a device that outputs sound towards the surroundings of the work machine 100. In the illustrated example, the external sound output device SP1 is an omnidirectional speaker and is configured to output sound uniformly in all directions. However, the external sound output device SP1 may also be a directional speaker that outputs sound in a specific direction, such as forward.

[0034] The information transmission device G1 is a device for informing the outside of the work machine 100 of the status of the work machine 100. In the illustrated example, the information transmission device G1 is installed in the upper rotating body 3 or the operator's cab 10 and is configured to inform workers around the work machine 100 of the status of the work machine 100. Specifically, the information transmission device G1 is a light-emitting device and includes a front light bar G1F, a left light bar G1L, a right light bar G1R, and a rear light bar G1B.

[0035] The front light bar G1F is a light-emitting device that can visually convey information to workers, etc., who are in front of the work machine 100, and is mounted on the outside of the operator's cab 10, such as on the roof of the operator's cab 10 or on the side of the boom 4. The front light bar G1F may also be mounted inside the operator's cab 10, such as on the ceiling of the operator's cab 10. The left light bar G1L is a light-emitting device that can visually convey information to workers, etc., who are to the left of the work machine 100, the right light bar G1R is a light-emitting device that can visually convey information to workers, etc., who are to the right of the work machine 100, and the rear light bar G1B is a light-emitting device that can visually convey information to workers, etc., who are behind the work machine 100. The front light bar G1F, left light bar G1L, right light bar G1R, and rear light bar G1B each emit light in response to an electrical signal from the controller 30. In the illustrated example, the light-emitting devices are LED lights, but other light-emitting devices such as halogen lamps may also be used. Furthermore, the light-emitting device is a multi-color type, but it may also be a mono-color type.

[0036] In the illustrated example, the front light bar G1F is attached to the roof of the cab 10, the left light bar G1L is attached to the upper left end of the upper surface of the upper rotating body 3, the right light bar G1R is attached to the upper right end of the upper surface of the upper rotating body 3, and the rear light bar G1B is attached to the upper rear end of the upper surface of the upper rotating body 3. In this way, the four information transmission devices G1 (front light bar G1F, left light bar G1L, right light bar G1R, and rear light bar G1B) are installed at different positions on the upper rotating body 3. Therefore, by operating each of the four information transmission devices G1 separately, the controller 30 can communicate the status of the work machine 100 to workers, etc., located in front of, to the left, to the right, and behind the work machine 100.

[0037] In the illustrated example, each of the four information transmission devices G1 and each of the four external sound collection devices M1 are arranged to correspond to one another. Specifically, the front light bar G1F is positioned adjacent to the front microphone M1F, the left light bar G1L is positioned adjacent to the left microphone M1L, the right light bar G1R is positioned adjacent to the right microphone M1R, and the rear light bar G1B is positioned adjacent to the rear microphone M1B.

[0038] Figure 3 shows an example configuration of an external sound collection device M1 and an information transmission device G1 attached to a work machine 100. Specifically, Figure 3 is a perspective view of a left microphone M1L and a left light bar G1L attached to a roughly rectangular housing. The following explanation, referring to Figure 3, pertains to the combination of the left microphone M1L and the left light bar G1L, but also applies similarly to the combination of the front microphone M1F and the front light bar G1F, the right microphone M1R and the right light bar G1R, and the rear microphone M1B and the rear light bar G1B.

[0039] As shown in Figure 3, the left microphone M1L and the left light bar G1L are positioned on the left side of a roughly rectangular prism-shaped housing so as to face left of the work machine 100. This arrangement allows the left microphone M1L to efficiently collect sounds generated to the left of the work machine 100, and the left light bar G1L to efficiently communicate the status of the work machine 100 to an operator located to the left of the work machine 100. For example, the left microphone M1L can capture the voice of an operator located to the left of the work machine 100, and the left light bar G1L can inform the operator that the left microphone M1L has captured the operator's voice by emitting light in a predetermined color. In this case, an operator located to the left of the work machine 100 who speaks into the left microphone M1L can confirm that their voice reached the left microphone M1L (i.e., the operator of the work machine 100) by looking at the left light bar G1L emitting light in the predetermined color.

[0040] The information transmission device G1 may be installed on the upper part of each of the four sides of the driver's cab 10 (see Figure 6). For example, the information transmission device G1 may be configured such that the front light bar G1F is mounted on the upper front of the driver's cab 10, the left light bar G1L is mounted on the upper left side of the driver's cab 10, the right light bar G1R is mounted on the upper right side of the driver's cab 10, and the rear light bar G1B is mounted on the upper rear of the driver's cab 10. Alternatively, the information transmission device G1 may be a single rotating light such as a Nico Torch mounted on the top surface of the driver's cab 10, or it may be a display device such as a liquid crystal display or an organic EL display.

[0041] The controller 30 is an example of a control device and is composed of a computer including, for example, a CPU, a volatile memory device, a non-volatile memory device, and various input / output interfaces. The controller 30 implements various functions, for example, by reading a program from the non-volatile memory device, loading it into the volatile memory device, and having the CPU execute it. In the illustrated example, the controller 30 is configured to implement various functions and control the work machine 100. These functions include, for example, a machine guidance function that guides the operator in manually operating the work machine 100. The functions may also include a contact avoidance function that automatically or autonomously operates or stops the work machine 100 to avoid contact between the work machine 100 and objects within the monitoring range around the work machine 100.

[0042] The boom angle sensor S1 detects the boom angle, which is the rotation angle of the boom 4 relative to the upper slewing body 3. The arm angle sensor S2 detects the arm angle, which is the rotation angle of the arm 5 relative to the boom 4. The bucket angle sensor S3 detects the bucket angle, which is the rotation angle of the bucket 6 relative to the arm 5.

[0043] Each of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 may be, for example, a rotary encoder, an acceleration sensor, a 6-axis sensor, an IMU (Inertial Measurement Unit), or the like, or may be a potentiometer using a variable resistor, a cylinder stroke sensor for detecting the stroke amount of a hydraulic cylinder, or the like.

[0044] The detection signal corresponding to the boom angle by the boom angle sensor S1, the detection signal corresponding to the arm angle by the arm angle sensor S2, and the detection signal corresponding to the bucket angle by the bucket angle sensor S3 are taken into the controller 30.

[0045] The machine body tilt sensor S4 detects the tilt state of the machine body (the lower traveling body 1 or the upper slewing body 3) with respect to the horizontal plane. The machine body tilt sensor S4 is attached to, for example, the upper slewing body 3 and detects the tilt angles around two axes in the front-rear direction and the left-right direction of the work machine 100 (that is, the upper slewing body 3). The machine body tilt sensor S4 may be, for example, an acceleration sensor, a 6-axis sensor, or an IMU or the like. The detection signal corresponding to the tilt angle by the machine body tilt sensor S4 is taken into the controller 30.

[0046] The slewing sensor S5 outputs information regarding the slewing of the upper slewing body 3. The slewing sensor S5 detects, for example, the slewing angular velocity of the upper slewing body 3 with respect to the lower traveling body 1. The slewing sensor S5 may detect the slewing angle. The slewing sensor S5 may be, for example, a gyro sensor, a resolver, or a rotary encoder or the like. The detection signal corresponding to the slewing angle or the slewing angular velocity of the upper slewing body 3 by the slewing sensor S5 is taken into the controller 30.

[0047] The positioning device PS is configured to acquire information regarding the position of the working machine 100. In the present embodiment, the positioning device PS is configured to measure the position and orientation of the working machine 100 in a reference coordinate system. Specifically, the positioning device PS is a GNSS (Global Navigation Satellite System) receiver incorporating an electronic compass, which measures the latitude, longitude, and altitude of the current position of the working machine 100, and also measures the orientation of the working machine 100. The reference coordinate system according to the present embodiment is, for example, the World Geodetic System. The World Geodetic System is a three-dimensional orthogonal XYZ coordinate system with the origin at the center of gravity of the Earth, the X-axis in the direction of the intersection of the Greenwich meridian and the equator, the Y-axis in the direction of 90 degrees east longitude, and the Z-axis in the direction of the North Pole. The detection signal corresponding to the position and orientation of the upper swing body 3 is taken into the controller 30. The function of detecting the orientation of the upper swing body 3 may be realized by an azimuth sensor attached to the upper swing body 3.

[0048] The communication device T1 is connected to an external communication line and is configured to control communication with devices outside the working machine 100. The communication device T1 may communicate with a device provided separately from the working machine 100. The device provided separately from the working machine 100 may include, in addition to the devices outside the working machine 100, portable terminal devices (portable terminals) brought into the cab 10 by the operator of the working machine 100. The communication device T1 also functions as an information acquisition device IAD that acquires information for determining the magnitude of the ambient noise generated around the working machine 100.

[0049] In this embodiment, the communication device T1 is configured to control communication between the communication device T1 and equipment located outside the work machine 100 via a wireless communication network. The communication device T1 may include, for example, a mobile communication module that supports mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation). The communication device T1 may also include, for example, a satellite communication module for connecting to a satellite communication network. Furthermore, the communication device T1 may include, for example, a Wi-Fi communication module or a Bluetooth® communication module. In addition, if there are multiple connectable communication lines, the communication device T1 may include multiple communication devices T1 according to the type of communication line.

[0050] For example, communication device T1 communicates with external devices such as a remote control room within the work site via a local communication line established at the work site. The local communication line is, for example, a local 5G (so-called local 5G) mobile communication line or a local Wi-Fi network established at the work site. Furthermore, communication device T1 is configured to send and receive information with communication devices installed in the remote control room via a wide-area communication line that includes the work site, i.e., a wide-area network.

[0051] The work machine 100 operates actuators in response to the operation of the operator seated in the cab 10, driving the driven parts such as the lower traveling body 1, the upper slewing body 3, the boom 4, the arm 5, and the bucket 6.

[0052] Alternatively, the work machine 100 may be configured to be remotely controlled from outside the work machine 100. When the work machine 100 is remotely controlled, the inside of the operator's cab 10 may be unoccupied.

[0053] Furthermore, the work machine 100 may automatically operate the actuators regardless of the operator's actions. This enables the work machine 100 to automatically operate at least a portion of the driven parts, such as the lower traveling body 1, the upper slewing body 3, the boom 4, the arm 5, and the bucket 6, that is, to realize a so-called "machine control function".

[0054] Figure 4 is a schematic diagram showing an example of the configuration of the work machine 100. In Figure 4, the mechanical power transmission system, hydraulic fluid line, pilot line, and electrical control system are indicated by double lines, thick solid lines, thick dashed lines, and dotted lines, respectively.

[0055] The drive system of the work machine 100 includes an engine 11, a regulator 13, a main pump 14, and a control valve unit 17. The hydraulic drive system of the work machine 100 also includes hydraulic actuators such as a slewing hydraulic motor 2A, a left travel hydraulic motor 2ML, a right travel hydraulic motor 2MR, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9.

[0056] The engine 11 is an example of a power source for the work machine 100, and is mounted, for example, at the rear of the upper rotating body 3. The power source for the work machine 100 may be a combination of a battery or fuel cell and an electric motor. Specifically, the engine 11 rotates at a constant speed at a preset target rotational speed under direct or indirect control by the controller 30, driving the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 may also be a gasoline engine or a hydrogen engine, etc.

[0057] The regulator 13 controls the discharge rate of the main pump 14. For example, the regulator 13 controls the discharge rate of the main pump 14 by adjusting the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the controller 30.

[0058] The main pump 14, for example, is mounted at the rear of the upper rotating body 3, similar to the engine 11, and supplies hydraulic fluid to the control valve unit 17 through the hydraulic fluid line. In the illustrated example, the main pump 14 is a variable displacement hydraulic pump.

[0059] The control valve unit 17 is one of the hydraulic control devices that control the hydraulic system in the work machine 100. In the illustrated example, the control valve unit 17 includes control valves 171 to 176. The control valve unit 17 is configured to selectively supply hydraulic fluid discharged by the main pump 14 to one or more hydraulic actuators through the control valves 171 to 176. The control valves 171 to 176 control the flow rate of hydraulic fluid flowing from the main pump 14 to the hydraulic actuators, and the flow rate of hydraulic fluid flowing from the hydraulic actuators to the hydraulic fluid tank. The hydraulic actuators include a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, a left-travel hydraulic motor 2ML, a right-travel hydraulic motor 2MR, and a slewing hydraulic motor 2A. Specifically, control valve 171 corresponds to the slewing hydraulic motor 2A, control valve 172 corresponds to the right-travel hydraulic motor 2MR, and control valve 173 corresponds to the left-travel hydraulic motor 2ML. Furthermore, control valve 174 corresponds to bucket cylinder 9, control valve 175 corresponds to boom cylinder 7, and control valve 176 corresponds to arm cylinder 8.

[0060] The pilot pump 15 is an example of a pilot pressure generating device and is configured to supply hydraulic fluid to a hydraulic control device via a pilot line. In the illustrated example, the pilot pump 15 is a fixed-displacement hydraulic pump. However, the pilot pressure generating device may be implemented by the main pump 14. That is, the main pump 14 may have the function of supplying hydraulic fluid to the control valve unit 17 via a hydraulic fluid line, as well as the function of supplying hydraulic fluid to various hydraulic control devices via a pilot line. In this case, the pilot pump 15 may be omitted.

[0061] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In the illustrated example, the discharge pressure sensor 28 outputs the detected value to the controller 30.

[0062] The operating device 26 is a device used by the operator to operate the actuator. The operating device 26 includes, for example, an operating lever and an operating pedal. The actuator may be a hydraulic actuator or an electric actuator.

[0063] The operation sensor 29 is configured to detect the operator's actions using the operation device 26. In this embodiment, the operation sensor 29 detects the operating direction and amount of operation of the operation device 26 corresponding to each actuator and outputs the detected values ​​to the controller 30. In the illustrated example, the controller 30 can control the opening area of ​​the valve 31 according to the output of the operation sensor 29. The controller 30 then supplies the hydraulic fluid discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17. The pressure of the hydraulic fluid supplied to each pilot port (pilot pressure) is, in principle, the pressure corresponding to the operating direction and amount of operation of the operation device 26 corresponding to each hydraulic actuator. Thus, the operation device 26 is configured to supply the hydraulic fluid discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17.

[0064] Furthermore, the directional control valves that drive each hydraulic actuator, which are built into the control valve unit 17, may be of the electromagnetic solenoid type. In this case, the operating signal output from the operating device 26 may be directly input to the control valve unit 17 (i.e., to the electromagnetic solenoid type directional control valve).

[0065] The operating device 26 may also be a hydraulic pilot type. Specifically, the operating device 26 uses hydraulic fluid supplied from the pilot pump 15 through the pilot line to output a pilot pressure corresponding to the operation to the secondary pilot line. The secondary pilot line is then connected to the control valve unit 17. As a result, the control valve unit 17 can receive pilot pressure corresponding to the operation of various driven elements (hydraulic actuators) in the operating device 26. Therefore, the control valve unit 17 can drive each hydraulic actuator according to the operation performed on the operating device 26 by the operator or the like. In this case, an operation sensor 29 capable of acquiring information about the operating state of the operating device 26 is provided, and the output of the operation sensor 29 is taken up by the controller 30. As a result, the controller 30 can grasp the operating state of the operating device 26. The operation sensor 29 is, for example, a pressure sensor that acquires information about the pilot pressure (operating pressure) of the secondary pilot line of the operating device 26.

[0066] Furthermore, some or all of the hydraulic actuators may be replaced with electric actuators. In this case, for example, the controller 30 may output operation commands to the electric actuator or a driver that drives the electric actuator, etc., according to the operation content of the operating device 26 or the content of the remote operation defined by the remote operation signal. Alternatively, the electric actuator may be configured to be operable by the operating device 26 when an operation signal is input from the operating device 26 to the electric actuator or driver, etc.

[0067] Furthermore, if the work machine 100 is operated exclusively by remote control or exclusively by a fully automatic operation function, the operating device 26 may be omitted.

[0068] Valve 31 functions as a control valve for machine control. Valve 31 is located in the pipeline connecting the pilot pump 15 and the pilot port of the control valve in the control valve unit 17, and is configured to change the flow area of ​​the pipeline. In the illustrated example, valve 31 is a solenoid valve that operates in response to a control command output by the controller 30. Therefore, the controller 30 can adjust the pilot pressure acting on the pilot port of the control valve by valve 31, independently of the operation of the operating device 26 by the operator.

[0069] With this configuration, the controller 30 can operate the hydraulic actuator corresponding to a specific operating device 26 even when no operation is being performed on that specific operating device 26.

[0070] Furthermore, the controller 30 may control the valve 31 to realize an automatic operation function for the work machine 100. Specifically, the controller 30 outputs an operation command corresponding to the automatic operation function to the valve 31. In this way, the controller 30 can realize the operation of the work machine 100 using the automatic operation function.

[0071] Furthermore, the controller 30 may control the valve 31 to enable remote operation of the work machine 100. Specifically, the controller 30 outputs an operation command to the valve 31 corresponding to the content of the remote operation specified by the remote operation signal received from the remote operation room via the communication device T1. As a result, the controller 30 causes the valve 31 to supply a pilot pressure corresponding to the content of the remote operation to the control valve unit 17, thereby enabling the operation of the work machine 100 based on the operator's remote operation.

[0072] In this embodiment, we will describe a case in which the engine 11 is used as the drive source, and the hydraulic pump is operated by the driving force generated by the engine 11 to perform the operation of the attachment AT, the rotation of the upper rotating body 3, and the driving motion. However, this embodiment is not limited to the engine 11 as the drive source, and a motor may also be used as the drive source. In other words, the control described in this embodiment may be applied to a so-called electric excavator in which the motor, which is the drive source, is driven by power supplied from a battery, or it may be applied to a hybrid excavator equipped with multiple drive sources including the engine 11 and a motor.

[0073] The control system of the work machine 100 includes a controller 30, a display device D1, an input device D2, a horn button HS, a speech button KS, an external sound collection device M1, an internal sound collection device M2, an external sound output device SP1, an internal sound output device SP2, an external volume dial DL1, an internal volume dial DL2, a switch SW, and a communication device T1, etc.

[0074] The controller 30 is configured to output control commands to the regulator 13 as needed, thereby changing the discharge amount of the main pump 14. The controller 30 may also be configured to perform control related to a machine guidance function that guides the manual operation of the work machine 100 by the operator through the operating device 26. The controller 30 may also be configured to perform control related to a machine control function that automatically assists the manual operation of the work machine 100 by the operator through the operating device 26. Some of the functions of the controller 30 may be implemented by other controllers (control devices). That is, the functions of the controller 30 may be implemented in a manner distributed among multiple controllers. For example, the machine guidance function and the machine control function may be implemented by dedicated controllers (control devices).

[0075] Now, with reference to Figure 5, the interior of the driver's cab 10 will be described. Figure 5 is a top view of the interior of the driver's cab 10. The work machine 100 is equipped with a driver's seat 50, an operating device 26, and a display device D1, etc., which are located inside the driver's cab 10. An access door is provided on the left side of the driver's seat 50. The operator can open the access door and enter the interior of the driver's cab 10.

[0076] The driver's seat 50 is located in the center of the driver's cab 10 when viewed from above. The driver's seat 50 includes a seat 51 on which the operator sits and a backrest 52. The driver's seat 50 is a reclining seat, and the tilt angle of the backrest 52 is adjustable. A left armrest 53L is located on the left side of the driver's seat 50, and a right armrest 53R is located on the right side. The left armrest 53L and the right armrest 53R are rotatably supported by the backrest 52.

[0077] A left console 54L is located to the left of the driver's seat 50, and a right console 54R is located to the right. The left console 54L and the right console 54R extend along the front-to-back direction. The driver's seat 50 is slidable in the front-to-back direction. The driver's seat 50 may also be configured to slide in the front-to-back direction together with the left console 54L and the right console 54R.

[0078] The left armrest 53L is positioned on top of the left console 54L. The right armrest 53R is positioned on top of the right console 54R. In a top view, the left armrest 53L is positioned to cover a portion of the left console 54L. In a top view, the right armrest 53R is positioned to cover a portion of the right console 54R.

[0079] The operating device 26 includes a left operating lever 26L, a right operating lever 26R, a left travel pedal 26PL, a right travel pedal 26PR, a left travel lever 26DL, and a right travel lever 26DR. The left operating lever 26L is located at the front of the left console 54L. Similarly, the right operating lever 26R is located at the front of the right console 54R. An operator seated in the driver's seat 50 can operate the left operating lever 26L while grasping it with their left hand, and can also operate the right operating lever 26R while grasping it with their right hand. An operator seated in the driver's seat 50 can drive the arm cylinder 8 and the slewing hydraulic motor 2A by operating the left operating lever 26L with their left hand. An operator seated in the driver's seat 50 can also drive the boom cylinder 7 and the bucket cylinder 9 by operating the right operating lever 26R with their right hand. The bases of the left operating lever 26L and the right operating lever 26R are covered by lever boots 27.

[0080] The left drive pedal 26PL and the right drive pedal 26PR are located on the floor in front of the driver's seat 50. An operator seated in the driver's seat 50 can operate the left drive hydraulic motor 2ML by operating the left drive pedal 26PL with their left foot. An operator seated in the driver's seat 50 can also operate the right drive hydraulic motor 2MR by operating the right drive pedal 26PR with their right foot.

[0081] The left travel lever 26DL and the right travel lever 26DR are positioned between the left travel pedal 26PL and the right travel pedal 26PR in a top view. The left travel lever 26DL and the right travel lever 26DR extend upward from the floor surface in front of the driver's seat 50. An operator seated in the driver's seat 50 can drive the left travel hydraulic motor 2ML by grasping the left travel lever 26DL with their left hand, similar to operation via the left travel pedal 26PL. Similarly, an operator seated in the driver's seat 50 can drive the right travel hydraulic motor 2MR by grasping the right travel lever 26DR with their right hand, similar to operation via the right travel pedal 26PR. Furthermore, the left travel lever 26DL and the right travel lever 26DR are positioned so that an operator can simultaneously operate both the left travel lever 26DL and the right travel lever 26DR with one hand.

[0082] The display device D1 is located in a place easily visible to a seated operator in the driver's cab 10 and displays various image information under the control of the controller 30. In the illustrated example, the display device D1 is located to the right front of the driver's seat 50 and is connected to the controller 30 via a dedicated line. The display device D1 displays various image information. The display device D1 includes a display screen that displays information such as the working conditions or operating status of the work machine 100. The operator seated in the driver's seat 50 can perform work with the work machine 100 while checking the various information displayed on the display device D1. The display device D1 may also be provided with an input device D2.

[0083] The input device D2 is located within reach of the operator seated in the driver's seat 50 and receives various operation inputs from the operator, outputting signals corresponding to the operation inputs to the controller 30. The input device D2 includes a touch panel mounted on the display of the display device D1 which displays various image information, a knob switch provided at the tip of one or more of the operation levers included in the operation device 26, or a button switch, lever, toggle switch, or rotary dial installed around the display device D1. Signals corresponding to the content of operations on the input device D2 are received by the controller 30.

[0084] A gate bar 55 is attached to the front of the front end of the left console 54L. The gate bar 55 operates in conjunction with the operation of the gate lock lever GL provided on the left console 54L. The gate bar 55 is mounted on the internal frame of the left console 54L so as to be able to move up and down around an axis that extends in the left-right direction at its upper end.

[0085] The gate lock lever GL is a mechanical input operating part for switching between a state in which the work machine 100 can be operated by the operating device 26 (operable state) and a state in which the work machine 100 cannot be operated by the operating device 26 (unoperable state). In the illustrated example, the gate lock lever GL is configured so that the operator can switch between a first operating position that realizes the unoperable state and a second operating position that realizes the operable state. The controller 30 switches between the operable state and the unoperable state according to the operating state of the gate lock lever GL. In the illustrated example, the controller 30 switches between the operable state and the unoperable state of the work machine 100 by electrically switching between a connected state and a disconnected state of the pilot line according to the operating state of the gate lock lever GL.

[0086] Furthermore, when the gate lock lever GL is in the second operating position, the gate bar 55 is raised forward (passage prohibited) as shown in Figure 5, preventing the operator from passing through the passenger door. On the other hand, when the gate lock lever GL is in the first operating position, the gate bar 55 is retracted inside the left console 54L so as not to obstruct the operator from passing through the passenger door (passage permitted).

[0087] With this configuration, the operator cannot operate the work machine 100 unless the gate lock lever GL is set to the second operating position, preventing passage of the gate bar 55. Therefore, this configuration prevents the work machine 100 from moving unintentionally even if the operator accidentally touches the operating device 26 when getting on or off the machine. Thus, this configuration can improve the safety of the work machine 100.

[0088] Furthermore, the work machine 100 may be configured to accept a predetermined operation to start the engine 11 only when the gate lock lever GL is in the second operating position and the gate bar 55 is in a no-passing state. In other words, the work machine 100 may be configured so that the engine 11 cannot be started when the gate lock lever GL is in the first operating position and the gate bar 55 is in a pass-permitting state.

[0089] A switch SW is installed on the right console 54R. To the right of the right console 54R is the window-side console 56. The window-side console 56 extends along the entire length of the driver's cab 10 in the front-to-back direction and is installed parallel to the right console 54R. The display device D1 is installed at the front of the window-side console 56. The window-side console 56 is equipped with an external volume dial DL1, an internal volume dial DL2, an internal sound collection device M2, a radio tuner, etc. The radio tuner, etc. may also be installed on the left console 54L or the right console 54R.

[0090] The internal sound collection device M2 is a device that collects sounds generated inside the driver's cab 10. In the illustrated example, the internal sound collection device M2 is an indoor microphone and is configured to pick up the voice of the operator inside the driver's cab 10.

[0091] The horn button HS is a button operated by the operator of the work machine 100 when sounding the horn. In the illustrated example, the horn button HS is a knob switch located at the tip of the left operating lever 26L.

[0092] The speech button KS is a button operated by the operator of the work machine 100 when speaking to workers around the work machine 100. In the illustrated example, the speech button KS is a knob switch located at the tip of the right operating lever 26R.

[0093] The internal sound output device SP2 is a device that outputs sound to the operator inside the driver's cab 10 and is installed inside the driver's cab 10. The internal sound output device SP2 converts the electrical signal input from the controller 30 into physical sound (air vibration) and outputs it. The internal sound output device SP2 may be installed in any location, for example, near the display device D1, near the input device D2, or near the passenger door of the driver's cab 10. In the illustrated example, the internal sound output device SP2 includes a left interior speaker SP2L mounted on the upper left corner of the rear wall of the driver's cab 10 and a right interior speaker SP2R mounted on the upper right corner of the rear wall of the driver's cab 10. The internal sound output device SP2 may also be headphones or earphones worn by the operator. In this case, the headphones or earphones are connected to communicate with the controller 30 via, for example, Bluetooth®.

[0094] The external volume dial DL1 is configured to adjust the volume of the sound output by the external sound output device SP1. In addition, the volume of the sound output by each of the external sound output devices SP1 may be further adjusted using a device other than the external volume dial DL1, such as a touch panel attached to the display device D1.

[0095] The external volume dial DL1 may be configured to rotate infinitely in both clockwise and counterclockwise directions. This is to accommodate cases where volume adjustment using the external volume dial DL1 and volume adjustment using a device other than the external volume dial DL1 are used in combination.

[0096] The internal volume dial DL2 is configured to adjust the volume of the sound output by the internal sound output device SP2. In addition, the volume of the sound output by each of the internal sound output devices SP2 may be further adjusted using a device other than the internal volume dial DL2, such as a touch panel attached to the display device D1.

[0097] The internal volume dial DL2 may be configured to rotate infinitely in both clockwise and counterclockwise directions. This is to accommodate cases where volume adjustment using the internal volume dial DL2 and volume adjustment using a device other than the internal volume dial DL2 are used in combination.

[0098] A switch SW is an example of an operating device for switching the operating state of a conversation function, which is an example of a voice transmission function. In the illustrated example, the switch SW is located on the upper surface of the right console 54R. However, the switch SW may also be one of the input devices D2, implemented by a touch panel provided on the display device D1, or a knob switch.

[0099] The voice transmission function is a function for realizing one-way or two-way voice transmission between the operator OP of the work machine 100 and people (workers WK) around the work machine 100, as shown in Figure 6. For example, the voice transmission function includes an internal sound playback function, which collects the operator OP's voice using an internal sound collection device M2 and outputs it from an external sound output device SP1, and an external sound playback function, which collects the worker WK's voice using an external sound collection device M1 and outputs it from an internal sound output device SP2. The conversation function is realized by a combination of the internal sound playback function and the external sound playback function.

[0100] Specifically, the conversation function is a function that enables conversation between the operator OP of the work machine 100 and the workers WK surrounding the work machine 100, as shown in Figure 6. Figure 6 is a perspective view of the work machine 100 on which the operator OP is riding and the workers WK located to the left front of the work machine 100. Figure 6 shows how the operator OP's voice is collected by the internal sound collection device M2 and output from the external sound output device SP1, and how the workers WK's voice is collected by the external sound collection device M1 and output from the internal sound output device SP2. In addition, in the work machine 100 shown in Figure 6, the information transmission device G1 is provided on the upper part of each of the four sides of the driver's cab 10. The front light bar G1F, provided on the upper part of the front of the driver's cab 10, emits green light, and the left light bar G1L, provided on the upper part of the left side of the driver's cab 10, emits white light. In Figure 6, the front light bar G1F, which emits green light, has a dot pattern. Upon seeing the front light bar G1F emitting green light, worker WK can recognize that their voice is being detected by the front microphone M1F. Note that, for clarity, other devices such as the imaging device S6 are omitted from the illustration in Figure 6.

[0101] The operating states of the conversation function include an ON state (shown in Figure 6) in which conversation is possible between the operator OP and the worker WK, and an OFF state in which conversation is not possible between the operator OP and the worker WK. However, the operating states of the conversation function may additionally include at least one of the following: an Audible State (relating to the operator OP) in which the operator OP can hear the worker WK's voice but the worker WK cannot hear the operator OP's voice, and an Audible State (relating to the operator OP) in which the worker WK can hear the operator OP's voice but the operator OP cannot hear the worker WK's voice.

[0102] Specifically, when the switch SW is operated and the conversation function is switched to the ON state, the external sound collector M1, external sound output device SP1, internal sound collector M2, and internal sound output device SP2 become available for use. Conversely, when the switch SW is operated and the conversation function is switched to the OFF state, the external sound collector M1, external sound output device SP1, internal sound collector M2, and internal sound output device SP2 become unavailable. Also, when the switch SW is operated and the conversation function is switched to the audible state, the external sound collector M1 and internal sound output device SP2 become available for use. Furthermore, when the switch SW is operated and the conversation function is switched to the speech-enabled state, the external sound output device SP1 and internal sound collector M2 become available for use. In the illustrated example, operator OP can speak to worker WK using the external sound output device SP1 by pressing the speech button KS while speaking, when the internal sound collection device M2 is available.

[0103] [Controller Functional Configuration] Referring to Figure 7, the configuration of the controller 30 for controlling bidirectional conversation will be described. Figure 7 is a block diagram showing an example of the controller 30. The controller 30 includes an acquisition unit 301, a detection unit 302, a noise reduction unit 303, a state switching unit 304, an operation reception unit 305, an output control unit 306, and a display control unit 307.

[0104] The acquisition unit 301 acquires detection results from various sensors installed on the work machine 100. For example, the acquisition unit 301 acquires sound signals from the external sound collection device M1 that represent sounds generated around the work machine 100. The acquisition unit 301 also acquires sound signals from the internal sound collection device M2 that represent sounds including the voice of the operator sitting in the operator's seat. Furthermore, the acquisition unit 301 acquires image information showing the imaging results from the imaging device S6.

[0105] The detection unit 302 performs a person detection process on the image information acquired by the acquisition unit 301 from the imaging device S6 to detect people present around the work machine 100. Any method can be used for person detection, not limited to well-known methods. For example, a determination may be made as to whether the feature quantities extracted from the image information approximate a person to a predetermined value or more. Alternatively, for example, a person detection sensor may be provided that monitors the imaging spaces of cameras S6F, S6L, S6R, and S6B, and the person detection sensor may detect people present around the work machine 100. The person detection sensor is a sensor that distinguishes and detects people from objects other than people, and for example, it is a sensor that detects energy changes in the corresponding monitoring space, and includes motion detection sensors that utilize output signals from pyroelectric infrared sensors, bolometer infrared sensors, infrared cameras, etc.

[0106] In this embodiment, the detection unit 302 detects the operating state of the external sound playback function based on the outputs of various sensors. The operating state of the external sound playback function may include an enabled state in which the external sound playback function is active and an disabled state in which the external sound playback function is disabled. The enabled state is a state in which the sound collected by the external sound collection device M1 is output from the internal sound output device SP2. The disabled state is a state in which the output of the sound collected by the external sound collection device M1 from the internal sound output device SP2 is stopped (muted).

[0107] The detection unit 302 may detect the operating state of the conversation function based on the output of various sensors and determine the operating state of the external sound playback function based on the detected operating state of the conversation function. The detection unit 302 may also determine the operating state of the external sound playback function based on the operating state of the conversation function. For example, if the operating state of the conversation function is on or audible, the detection unit 302 may determine that the operating state of the external sound playback function is enabled. Also, if the operating state of the conversation function is off or ready to speak, the detection unit 302 may determine that the operating state of the external sound playback function is disabled.

[0108] The noise reduction unit 303 processes the sound signal acquired by the acquisition unit 301 from the external sound collection device M1, emphasizing the frequency band of human voices and reducing frequencies outside the human voice band. These processes may utilize well-known methods; for example, noise cancellation processing may be performed, or frequencies outside the human voice band may be suppressed using a bandpass filter or the like. Specifically, as a noise cancellation process, the noise reduction unit 303 suppresses sounds generated from the work machinery 100, such as the engine 11, based on sound signals from multiple external sound collection devices M1. In this way, the noise reduction unit 303 removes noise from the sound signal collected by the external sound collection device M1.

[0109] The state switching unit 304 switches the operating state of the external sound playback function. The state switching unit 304 may determine whether or not to switch the operating state of the external sound playback function, and if it determines that the state of the external sound playback function should be switched, it may switch the operating state of the external sound playback function. If the external sound playback function is in an disabled state, the state switching unit 304 may determine whether or not to switch the external sound playback function to an enabled state. If the external sound playback function is in an enabled state, the state switching unit 304 may determine whether or not to switch the external sound playback function to an disabled state.

[0110] The state switching unit 304 may switch the external sound playback function from disabled to enabled depending on a first condition based on a person in the vicinity of the work machine 100. The first condition may include the detection of a pre-action for speaking towards the work machine 100 from a signal observing a person in the vicinity of the work machine 100. In other words, the first condition may include the fact that a person in the vicinity of the work machine 100 has performed a pre-action. Furthermore, the first condition may include the fact that the pre-action detected from the signal observing a person in the vicinity of the work machine 100 does not fall under a predetermined exclusion action. The signal observing a person in the vicinity of the work machine 100 may include sound signals acquired from the external sound collection device M1 and image information acquired from the imaging device S6.

[0111] Figure 8 shows an example of conditions based on surrounding people. As shown in Figure 8, preparatory actions can be classified into vocal actions, bodily actions, signal transmission, and preparation actions. Vocal actions can be detected, for example, based on sound signals acquired from an external sound collection device M1. Bodily actions can be detected, for example, based on image information acquired from an imaging device S6. Signal transmission can be detected, for example, based on signals received by a communication device T1. Preparation actions can be detected, for example, based on sound signals acquired from an external sound collection device M1 and image information acquired from an imaging device S6, at least one of the two.

[0112] The vocalization action may include vocalizing a predetermined phrase or a predetermined pitch. The predetermined phrase may also be a predetermined keyword used as a trigger to activate the voice input function. The predetermined keyword may be, for example, a wake-up word used to activate a voice assistant or the like. The predetermined phrase may be set during the manufacturing of the work machine 100, or it may be set arbitrarily by the operator or the like.

[0113] The physical movement may include a predetermined gesture or action. The predetermined gesture may be, for example, the action of pointing the palm towards the imaging device S6 of the work machine 100. The predetermined action may be, for example, the action of an operator of the surrounding work machine 100 opening the window of the driver's cab.

[0114] Signal transmission may include signal transmission by terminal operation. Terminal operation may be performed by worker WK. Worker WK may hold a transmitter that emits a dedicated signal and transmit a signal to the work machine 100 by operating the transmitter. Alternatively, worker WK may hold a mobile information terminal such as a smartphone with a dedicated application installed and transmit a signal to the work machine 100 by operating the application.

[0115] Preparatory actions may include preparatory actions for vocalization. Preparatory actions may also include anticipating vocalization or physical movements. Preparatory actions for vocalization are generally actions performed by a person immediately before vocalization. Examples of preparatory actions for vocalization include inhaling and lip movement.

[0116] An excluded action is an action that falls under the category of a pre-action but can be considered not to be an action for speaking to the work machine 100. Excluded actions can be classified into vocal actions and bodily actions. Examples of excluded actions classified as vocal actions include talking on a mobile phone, speaking in a direction other than the work machine 100, or talking to another worker. Examples of excluded actions classified as bodily actions include worker WK having their palm facing the work machine 100 but being presumed to be trying to touch another object, or worker WK performing a predetermined gesture but being making the gesture towards another worker (or another work machine 100).

[0117] Figure 9 shows an example of preparatory actions performed by people in the vicinity. Figure 9(A) shows an example of a preparatory action in which an operator WK around the work machine 100 utters a predetermined phrase. In the illustrated example, operator WK utters the predetermined phrase, the wake-up word 401, towards the work machine 100 from behind the work machine 100. Even if the operator of the work machine 100 has disabled the external sound playback function, they can hear the voice uttered by operator WK after the wake-up word 401 inside the operator's cab 10.

[0118] Figure 9(B) shows an example of a preliminary action performed by a worker WK around the work machine 100, in which a predetermined gesture is made. In the illustrated example, worker WK performs action 402, which is an example of a predetermined gesture, by pointing their palm towards the work machine 100 from behind the work machine 100. Even if the external sound playback function is disabled, the operator of the work machine 100 can hear the voice spoken by worker WK after gesture 402 inside the operator's cab 10.

[0119] The state switching unit 304 may switch the external sound playback function from enabled to disabled depending on a second condition based on people around the work machine 100. The second condition may include not detecting any prior action for a certain period of time from signals observing people around the work machine 100. In other words, the second condition may include not performing any prior action for a certain period of time from people around the work machine 100.

[0120] The operation reception unit 305 receives operations from the operator via one or more of the input device D2, the speech button KS, and the switch SW. For example, the operation reception unit 305 receives whether or not the speech button KS has been pressed.

[0121] The output control unit 306 controls the output of sound based on an audio signal from either the internal sound output device SP2 or the external sound output device SP1. For example, the output control unit 306 controls the output of sound based on an audio signal acquired from the external sound collection device M1 from the internal sound output device SP2. Alternatively, for example, the output control unit 306 controls the output of sound based on an audio signal acquired from the internal sound collection device M2 from the external sound output device SP1.

[0122] When the speech button KS is pressed, the output control unit 306 controls the output of sound based on the sound signal acquired from the internal sound collection device M2 through the external sound output device SP1. As long as the speech button KS remains pressed, the output control unit 306 controls the output of sound based on the sound signal acquired from the internal sound collection device M2 through the external sound output device SP1. When the speech button KS is released, the output control unit 306 stops the control of outputting sound based on the sound signal acquired from the internal sound collection device M2 through the external sound output device SP1 and controls the output of sound based on the sound signal acquired from the external sound collection device M1 through the internal sound output device SP2.

[0123] This control allows the operator of the work machine 100 to speak to people in the vicinity of the work machine 100 at a desired time. Furthermore, this control allows the operator of the work machine 100 to hear sounds in the vicinity of the work machine 100 when there is no need to speak to people in the vicinity.

[0124] Furthermore, the predetermined operation for switching the output control described above is not limited to whether or not the voice button KS is being pressed continuously. For example, the output control may be switched each time the predetermined button is pressed, or the output control may be switched based on whether or not the operator is looking at the display device D1 based on an image captured by an imaging device installed in the driver's cab 10, or the output control may be switched in response to an operation on the touch panel included in the input device D2. In this way, any switching of the output control in response to a predetermined operation is acceptable.

[0125] The output control unit 306 controls the output of the sound collected by the external sound collection device M1 from the external sound output device SP1 unless the speech button KS is pressed or the switch SW is turned off. As a result of this control, unless the operator intentionally operates the device, the sound collected by the external sound collection device M1 is output from the external sound output device SP1, allowing the operator to recognize the surrounding conditions of the work machine 100.

[0126] The output control unit 306 performs a noise cancellation function on the sound when outputting sound from the internal sound output device SP2 or the external sound output device SP1. Noise cancellation can be performed using well-known methods; for example, noise components can be removed by superimposing a sound with the opposite phase to the noise component. Furthermore, the output control unit 306 may perform emphasis processing on the frequency range of the human voice for the sound to be output. By performing the noise cancellation function on the output control unit 306, it becomes easier to understand what is being said.

[0127] The output control unit 306 controls the output of one of the sounds collected by the internal sound collection device M2, an alarm, or a horn from the external sound output device SP1. If there are multiple types of sounds to be output from the external sound output device SP1, the output control unit 306 controls the output of one of the sounds collected by the internal sound collection device M2, an alarm, or a horn from the external sound output device SP1 based on priority.

[0128] The display control unit 307 controls the display on the display device D1. The display control unit 307 may output an image of the surroundings of the work machine 100, detection results from various sensors, and the operating status of the external sound playback function to the display device D1. The display device D1 displays the image of the surroundings of the work machine 100 output from the display control unit 307. The display device D1 also displays information indicating the operating status of the external sound playback function output from the display control unit 307.

[0129] [Display Example of Display Device] Next, with reference to Figure 10, an example of a display screen 85 displayed on the display device D1 will be described. Figure 10 is a diagram showing an example of a display screen 85. The display screen 85 is displayed on the image display unit D1a while the work machine 100 is in operation.

[0130] The controller 30 generates a display screen 85 based on various information acquired from various devices and image information input from the imaging device S6. The various information includes the rotation angle, information indicating the image area in which a person is visible, location information of people present around the work machine 100 (information indicating the direction and distance where the detected person is located), and detection results from various sensors.

[0131] The image display unit D1a displays a display screen 85, which includes a date and time display area 42a, a driving mode display area 42b, an attachment display area 42c, a fuel consumption display area 42d, an engine control status display area 42e, a conversation status display area 42f, a coolant temperature display area 42g, a fuel level display area 42h, a rotation speed level display area 42i, a urea solution level display area 42j, a hydraulic oil temperature display area 42k, an operating status display area 42m, a work machine status display area 421, a first image display area 422, a second image display area 423, and a notification display area 424, according to control from the controller 30. The display screen 85 may also include other display areas, such as an engine operating time display area which displays the cumulative operating time of the engine 11.

[0132] The driving mode display area 42b, attachment display area 42c, engine control status display area 42e, and rotation speed level display area 42i are areas that display setting status information, which is information related to the setting status of the work machine 100. The fuel consumption display area 42d, conversation status display area 42f, coolant temperature display area 42g, fuel level display area 42h, urea solution level display area 42j, and hydraulic oil temperature display area 42k are areas that display operating status information, which is information representing the operating status of the work machine 100 based on the detection results of various sensors.

[0133] The date and time display area 42a is an area that displays the current date and time. The driving mode display area 42b is an area that displays the current driving mode. The attachment display area 42c is an area that displays an image representing the attachment currently installed. The fuel consumption display area 42d is an area that displays fuel consumption information calculated by the controller 30. The fuel consumption display area 42d includes an average fuel consumption display area 42d1 that displays lifetime average fuel consumption or section average fuel consumption, and an instantaneous fuel consumption display area 42d2 that displays instantaneous fuel consumption.

[0134] The engine control status display area 42e is an area that displays the control status of the engine 11. The coolant temperature display area 42g is an area that displays the current temperature of the engine coolant. The fuel level display area 42h is an area that displays the remaining amount of fuel stored in the fuel tank.

[0135] The conversation state display area 42f is an area for displaying an image representing the state of conversation between the operator OP and a person (worker WK, etc.) in the vicinity of the work machine 100. In this disclosure, "conversation state" refers to a state relating to the function using a sound output device (e.g., external sound output device SP1 or internal sound output device SP2) and a sound collection device (e.g., external sound collection device M1 or internal sound collection device M2) provided on the work machine 100, and will hereinafter also be simply referred to as "conversation state". In the illustrated example, the conversation state includes a first conversation state in which the operator OP is not speaking and a person in the vicinity of the work machine 100 is speaking, a second conversation state in which the operator OP is speaking and a person in the vicinity of the work machine 100 is not speaking, and a third conversation state in which neither the operator OP nor a person in the vicinity of the work machine 100 is speaking. In other words, the first conversation state is when the voice of the operator OP, collected by the internal sound collection device M2, is output from the external sound output device SP1 to people around the work machine 100, and the second conversation state is when the voices of people around the work machine 100, collected by the external sound collection device M1, are output from the internal sound output device SP2 to the operator OP. Icons representing the first conversation state, icons representing the second conversation state, and icons representing the third conversation state are selectively displayed in the conversation state display area 42f. In Figure 10, the icon representing the third conversation state is displayed.

[0136] The operator OP can recognize the current conversation status between the operator OP and the worker WK by looking at the icon displayed in the conversation status display area 42f. Furthermore, even while operating the attachment AT, the operator OP can recognize the current conversation status by, for example, looking at the tip of the attachment AT with their central field of view while looking at the icon displayed in the conversation status display area 42f with their peripheral field of view. In the illustrated example, the conversation status display area 42f is reserved as a fixed area, and the icon representing the conversation status is always displayed, but it may also be displayed only while the conversation status is in a predetermined state (when the operator OP is speaking or when the worker WK is speaking). In this case, the icon representing the conversation status may be superimposed on the current date and time displayed in the date and time display area 42a, or superimposed on the edge of the camera image displayed in the work machine status display area 421, the first image display area 422, or the second image display area 423, etc.

[0137] The display device D1 may be configured to display an icon representing the conversation status in the conversation status display area 42f when the conversation function is ON, and not display an icon representing the conversation status in the conversation status display area 42f when the conversation function is OFF. Alternatively, the icon representing the conversation status may include an icon representing that the conversation function is OFF. In this case, when the conversation function is OFF, the display device D1 can display an icon representing that the conversation function is OFF in the conversation status display area 42f. The display device D1 may also be configured to display an icon representing the conversation status in the conversation status display area 42f when the internal sound playback function is ON, and not display an icon representing the conversation status in the conversation status display area 42f when the internal sound playback function is OFF. Alternatively, the display device D1 may be configured to display an icon representing the conversation status in the conversation status display area 42f when the external sound playback function is ON, and not display an icon representing the conversation status in the conversation status display area 42f when the external sound playback function is OFF.

[0138] The rotation speed level display area 42i is an area that displays the current level set by the rotation speed adjustment dial (not shown) as an image. The rotation speed level display area 42i displays a number indicating the selected level. The number "1" displayed in the rotation speed level display area 42i indicates that the selected rotation speed level is "Level 1". The number "n" displayed in the rotation speed level display area 42i indicates that the selected rotation speed level is "Level n". "n" is a natural number. When the operator OP rotates the rotation speed adjustment dial, the number displayed in the rotation speed level display area 42i changes.

[0139] The urea solution remaining amount display area 42j is an area that displays the remaining amount of urea solution stored in the urea solution tank as an image. The hydraulic oil temperature display area 42k is an area that displays the temperature of the hydraulic oil in the hydraulic oil tank.

[0140] The operating status display area 42m is an area that displays the operating status of the conversation function as an image. The operating status display area 42m may also display the operating status of the external sound playback function as an image. The operating status display area 42m may also display an icon corresponding to the operating status of the conversation function. Alternatively, the operating status display area 42m may display an icon corresponding to the operating status of the external sound playback function. For example, when the external sound playback function is enabled, the operating status display area 42m may display an icon corresponding to the operating status of the external sound playback function, and when the external sound playback function is disabled, the icon corresponding to the operating status of the external sound playback function may be hidden. Also, for example, when the external sound playback function is enabled, the operating status display area 42m may display an icon corresponding to the operating status of the conversation function, and when the external sound playback function is disabled, the icon corresponding to the operating status of the conversation function may be hidden.

[0141] The icons corresponding to the operating status may be image icons corresponding to the operating status, or icons that display the same image in a color corresponding to the operating status. For example, the operating status display area 42m may display an icon representing the speaker outputting sound when the external sound playback function is enabled, and an icon representing the speaker not outputting sound when the external sound playback function is disabled. Alternatively, for example, the operating status display area 42m may display an icon in an easily visible color (e.g., white on a black background) when the external sound playback function is enabled, and an icon in a less visible color (e.g., gray on a black background) when the external sound playback function is disabled.

[0142] The conversation status display area 42f displays the status of the conversation. In contrast, the operation status display area 42m displays the operation status of the conversation function. The operation status of the conversation function indicates whether the internal sound playback function and external sound playback function that realize the conversation function are enabled or disabled. Regardless of whether the operator OP and a person near the work machine 100 are having a conversation, the operation status of the conversation function can be one of the following: on, off, audible, or speaking. On the other hand, the conversation status indicates who is speaking in the conversation between the operator OP and a person near the work machine 100. The conversation status, assuming that the operation status of the conversation function is not off, can be one of the first conversation status, second conversation status, or third conversation status. Therefore, for example, if the conversation function is off, even if the operator OP is speaking, the conversation status display area 42f will not show an icon indicating the second conversation status, which is the state in which the operator OP is speaking. At this time, the operating status display area 42m displays an icon indicating that the conversation function is off, so the operator OP can immediately understand that the conversation function is off. Then, the operator OP can operate the switch SW to turn on the conversation function and communicate with people around the work machine 100.

[0143] The work machine status display area 421 is an area that displays information representing the positional relationship between the work machine 100 and a person detected in the vicinity of the work machine 100. Specifically, the work machine status display area 421 is an area that displays an overhead image representing the real space centered on the work machine 100 at a predetermined scale. A work machine icon 421b indicating the presence of the work machine 100 is placed at the center of the work machine status display area 421. The overhead image may be a viewpoint transformation image generated based on an image captured by the imaging device S6, or it may be a computer graphics (CG) image. In the illustrated example, the overhead image is a CG image that schematically represents the state of the real space centered on the work machine 100 when viewed from directly above.

[0144] In addition to the work machine icon 421b, the work machine status display area 421 simultaneously displays an icon indicating the direction in which the work machine 100 can move (direction indicator icon 421a in the example of Figure 10) and icons representing the location of people detected around the work machine 100 (person detection icon 421e, person detection icon 421f, person detection icon 421g in the example of Figure 10). In the work machine status display area 421, areas other than the direction indicator icon 421a, work machine icon 421b, person detection icon 421e, person detection icon 421f, and person detection icon 421g (in other words, the background) may be represented by a single color (for example, black).

[0145] In the illustrated example, the work machine icon 421b is an icon that combines an image showing the upper slewing body 3 and an image showing the lower traveling body 1, according to the positional relationship between the upper slewing body 3 and the lower traveling body 1 based on the slewing angle. In the illustrated example, the image showing the lower traveling body 1 is tilted 45 degrees counterclockwise relative to the image showing the upper slewing body 3, which is facing upwards on the screen.

[0146] In the illustrated example, the direction indicator icon 421a is triangular in shape, indicating the direction in which the work machine 100 travels when the travel lever is pushed forward. However, the direction indicator icon 421a can be any shape as long as it represents the direction in which the work machine 100 can move.

[0147] The person detection icons 421e, 421f, and 421g are icons that represent the location of a person detected based on the image captured by the imaging device S6. The positional relationship between the work machine icon 421b and each of the person detection icons 421e, 421f, and 421g corresponds to the positional relationship between the work machine 100 in real space and the people present around the work machine 100.

[0148] In the illustrated example, a computer graphics (CG) image is displayed as an overhead view in the machine status display area 421. However, a viewpoint-converted image generated by combining images captured by the imaging device S6 installed on the machine 100 may also be displayed as the overhead view. However, the overhead view will also display objects other than people around the machine 100, as well as the condition of the ground, such as unevenness. Therefore, when the operator (OP) views the viewpoint-converted image as an overhead view, they will perceive information about various objects, making it difficult to identify people around the machine 100.

[0149] Therefore, in the illustrated example, the work machine status display area 421 displays an icon representing the work machine 100, an icon indicating the direction in which the work machine 100 can move (for example, a direction indicator icon 421a), and icons indicating people detected around the work machine 100 (for example, a person detection icon 421e, a person detection icon 421f, and a person detection icon 421g) on ​​a monochrome background image. In other words, the work machine status display area 421 suppresses (omits) the display of images of objects other than people present around the work machine so that the operator OP's attention is not drawn to images of objects other than people present around the work machine.

[0150] In other words, the operator OP can recognize the current status of the work machine 100, the direction in which it can move, and the positional relationship between the work machine 100 and people present around it by looking at the work machine status display area 421.

[0151] Furthermore, by looking at the work machine status display area 421, the operator OP can infer how the positional relationship between the work machine 100 and people surrounding it will change when the work machine 100 is moved. Also, since the display of objects other than people is suppressed in the work machine status display area 421, the operator OP is less likely to be distracted by other objects. As a result, the work machine 100 can prevent the operator OP, who is looking at the work machine status display area 421, from overlooking the presence of people. Therefore, the work machine 100 can achieve improved safety.

[0152] Furthermore, the work machine status display area 421 may display one or more distance display images representing the distance from the work machine 100. The distance display image is a graphic image used to allow the operator OP to recognize the relative distance from the work machine 100, and is, for example, an image of a circle centered on the center point of the work machine icon 421b (for example, the position of the pivot axis). The circle image may include, for example, at least one of an image of a first circle with a radius equal to the distance between the tip position of the attachment AT and the pivot axis, calculated based on the current posture of the attachment AT of the work machine 100, and an image of a second circle with a radius equal to the distance between the tip position of the attachment AT and the pivot axis when the attachment AT is fully extended horizontally. The current distance between the tip position of the attachment AT and the pivot axis is calculated, for example, based on the detection results of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3.

[0153] The controller 30 may change the display mode of at least one of the circle image used as a distance display image and the person detection icon depending on whether the person detection icon is included within the range of the circle image used as a distance display image. For example, a person detection icon within the range of the first circle may be displayed in a first color (e.g., red), a person detection icon outside the range of the first circle but within the range of the second circle may be displayed in a second color (e.g., yellow), and a person detection icon outside the range of the second circle may be displayed in a third color (e.g., blue). The same applies to the color of the circle image. Note that the display mode may be changed by changing the flashing period, contrast, or brightness, etc. In this way, the display device D1 can alert the operator OP according to the distance between the work machine 100 and the person by displaying a person detection icon whose color changes according to the distance from the center point of the work machine icon 421b. Therefore, the work machine 100 can achieve improved safety.

[0154] The operator OP can view the work machine status display area 421 to recognize the positional relationship between the distance display image and the person detection icons 421e, 421f, and 421g, and can easily operate the attachment AT in a way that prevents people from coming into contact with it. Therefore, the work machine 100 can achieve improved safety.

[0155] The image display unit D1a displays the working machine status display area 421, as well as images captured by the imaging device S6. Therefore, the operator OP can recognize the specific situation around the working machine 100 not only by looking at the working machine status display area 421, but also by looking at the images captured by the imaging device S6. Thus, the working machine 100 can achieve improved safety.

[0156] Specifically, the first image display area 422 and the second image display area 423 are areas for displaying images captured by the imaging device S6. Specifically, the rightward image is displayed in the first image display area 422, and the rearward image is displayed in the second image display area 423. The rightward image is an image showing the space to the right of the work machine 100, and includes an end image 422c, which is an image of the upper right end of the upper surface of the upper rotating body 3. The rightward image is a real viewpoint image (through image) generated by the controller 30, and is generated based on an image acquired by the right camera S6R. The rearward image is an image showing the space behind the work machine 100, and includes a counterweight image 423c, which is an image of the counterweight provided at the rear of the upper rotating body 3. The rearward image is a real viewpoint image (through image) generated by the controller 30, and is generated based on an image acquired by the rear camera S6B.

[0157] The first image display area 422 is displayed to the right of the work machine status display area 421. The second image display area 423 is displayed below the work machine status display area 421. In the illustrated example, the area above the image display unit D1a corresponds to the front of the image showing the upper rotating body 3 among the work machine icon 421b in the work machine status display area 421. Therefore, the first image display area 422 is displayed in a position corresponding to the right of the image showing the upper rotating body 3 in the work machine status display area 421, and the second image display area 423 is displayed in a position corresponding to the bottom of the image showing the upper rotating body 3 in the work machine status display area 421.

[0158] Therefore, the operator OP can intuitively recognize that the image captured by the right camera S6R is displayed to the right of the work machine status display area 421, and the image captured by the rear camera S6B is displayed behind the work machine status display area 421. Thus, the work machine 100 can achieve improved safety.

[0159] Furthermore, the example shown in Figure 10 is merely one example of the arrangement of image information, and the arrangement of image information is not limited to the arrangement shown in Figure 10. For example, the first image display area 422 and the second image display area 423 may be arranged independently of the imaging direction of the imaging device S6.

[0160] Furthermore, if the controller 30 detects a person image (person image) based on at least one of the right image and the rear image, it may superimpose a frame surrounding the person image. In the illustrated example, a frame 422b surrounding the person image 422a is displayed in the right image of the first image display area 422, and a frame 423b surrounding the person image 423a is displayed in the rear image of the second image display area 423. In this case, the controller 30 may match the color of the frame 422b with the color of the person detection icon 421e, and also match the color of the frame 423b with the color of the person detection icon 421f. In other words, the image display unit D1a displays frames indicating detected people for the right image and the rear image, and displays the correspondence between the person represented by the person image surrounded by the frame and the person represented by the person detection icon so that the operator OP can recognize it. As a result, the operator OP can recognize the situation of the person displayed in the work machine status display area 421 by looking at the right image and the rear image. Therefore, the operator OP can operate the work machine 100 while taking into account the situation of people present around the work machine 100. Thus, the work machine 100 can achieve improved safety. In the illustrated example, as an example of a display that allows the operator OP to recognize the correspondence between people represented by framed human images and people represented by human detection icons, the controller 30 is configured to match the color of the frame with the color of the human detection icon. However, the controller 30 may also be configured to allow the operator OP to recognize the correspondence between people represented by framed human images and people represented by human detection icons by matching the flashing cycle of the frame with the flashing cycle of the human detection icon.

[0161] As described above, the person detection icon is configured to change color depending on whether the detected person is within range of the sound output by the external sound output device SP1. Furthermore, the color of the person detection icon is configured to match the color of the frame displayed in the right or rear image. Therefore, the operator OP can determine, by looking at the color of the frame, whether the person represented by the person image enclosed in the frame is within range of the sound output by the external sound output device SP1.

[0162] The notification display area 424 displays notifications 424a related to the external sound playback function. The notification display area 424 may also display a string indicating that the external sound playback function has been switched from an enabled state to a disabled state, or a string indicating that the external sound playback function has been switched from a disabled state to an enabled state. The notification display area 424 may also display notifications 424a in a form other than a string. For example, notification 424a may be an icon, image, or video showing the external sound playback function being switched from an enabled state to a disabled state, or from a disabled state to an enabled state.

[0163] The image display unit D1a displays the work machine status display area 421, the first image display area 422, and the second image display area 423, as well as the operation status display area 42m and the notification display area 424. When the operator looks at the display device D1 to check the operating status of the external sound playback function, they are unable to visually check the surroundings of the work machine 100. At this time, the display device D1 displays the work machine status display area 421, the first image display area 422, and the second image display area 423 on the same display screen 85 as the operation status display area 42m and the notification display area 424. The operator can check the operation status display area 42m and the notification display area 424 while checking the surroundings of the work machine 100, thus being able to check the operating status of the external sound playback function without compromising safety.

[0164] [Flowchart of External Sound Playback Function State Switching Process] Next, the state switching process for the external sound playback function executed by the controller 30 will be described. Figure 11 is a flowchart showing an example of the external sound playback function state switching process. Note that the external sound playback function state switching process is repeatedly executed at predetermined time intervals while the work machine 100 is in operation.

[0165] In step S101, the acquisition unit 301 of the controller 30 acquires the detection results of various sensors. The detection results of the various sensors include sound signals collected by the external sound collection device M1 and image information acquired by the imaging device S6. The acquisition unit 301 sends the detection results of the various sensors to the detection unit 302.

[0166] In step S102, the detection unit 302 of the controller 30 receives detection results from various sensors from the acquisition unit 301. Based on the detection results from the various sensors, the detection unit 302 detects the operating state of the external sound playback function. The detection unit 302 notifies the state switching unit 304 of the operating state of the external sound playback function.

[0167] The state switching unit 304 determines whether the operating state of the external sound playback function notified by the detection unit 302 is disabled or not. If the external sound playback function is disabled (YES), the state switching unit 304 proceeds to step S103. On the other hand, if the external sound playback function is enabled (NO), the state switching unit 304 proceeds to step S106.

[0168] In step S103, the state switching unit 304 of the controller 30 detects any prior actions by a person around the work machine 100 based on the detection results of various sensors acquired in step S101. If prior actions by a person around the work machine 100 are detected (YES), the state switching unit 304 proceeds to step S104. On the other hand, if no prior actions by a person around the work machine 100 are detected (NO), the state switching unit 304 terminates the state switching process for the external sound playback function.

[0169] In step S104, the state switching unit 304 of the controller 30 determines whether the pre-operation detected in step S103 corresponds to an excluded operation. If it is determined that the pre-operation does not correspond to an excluded operation (NO), the state switching unit 304 proceeds to step S105. On the other hand, if it is determined that the pre-operation corresponds to an excluded operation (YES), the state switching unit 304 terminates the state switching process for the external sound playback function.

[0170] In step S105, the state switching unit 304 of the controller 30 switches the operating state of the external sound playback function from disabled to enabled. Thereafter, the output control unit 306 of the controller 30 controls the output of sound based on the sound signal acquired from the external sound collection device M1 from the internal sound output device SP2. Alternatively, the output control unit 306 may also control the output of sound based on the sound signal acquired from the internal sound collection device M2 from the external sound output device SP1.

[0171] Furthermore, the display control unit 307 of the controller 30 outputs the operating status of the external sound playback function to the display device D1. The display device D1 displays the operating status of the external sound playback function on the display screen 85. For example, the display device D1 displays an icon in the operating status display area 42m indicating that the external sound playback function is enabled. The display device D1 also displays a notification in the notification display area 424 indicating that the external sound playback function has been switched to the enabled state.

[0172] In step S106, the state switching unit 304 of the controller 30 detects any prior actions by a person around the work machine 100 based on the detection results of various sensors acquired in step S101. If no prior actions by a person around the work machine 100 are detected (NO), the state switching unit 304 proceeds to step S107. On the other hand, if prior actions by a person around the work machine 100 are detected (YES), the state switching unit 304 terminates the state switching process for the external sound playback function.

[0173] In step S107, the state switching unit 304 of the controller 30 determines whether a certain amount of time has elapsed since the last detection of a pre-action by a person around the work machine 100. If a certain amount of time has elapsed since the last detection of a pre-action (YES), the state switching unit 304 proceeds to step S108. On the other hand, if a certain amount of time has not elapsed since the last detection of a pre-action (NO), the state switching unit 304 terminates the state switching process for the external sound playback function.

[0174] In step S108, the state switching unit 304 of the controller 30 switches the operating state of the external sound playback function from the enabled state to the disabled state. The output control unit 306 of the controller 30 stops the control of outputting sound based on the sound signal acquired from the external sound collection device M1 from the internal sound output device SP2. Alternatively, the output control unit 306 may also stop the control of outputting sound based on the sound signal acquired from the internal sound collection device M2 from the external sound output device SP1.

[0175] Furthermore, the display control unit 307 of the controller 30 outputs the operating status of the external sound playback function to the display device D1. The display device D1 displays the operating status of the external sound playback function on the display screen 85. For example, the display device D1 displays an icon in the operating status display area 42m indicating that the external sound playback function is disabled (or hides the operating status display area 42m). The display device D1 also displays a notification in the notification display area 424 indicating that the external sound playback function has been switched to the disabled state.

[0176] In this way, the work machine 100 according to this embodiment comprises a lower traveling body 1, an upper rotating body 3 rotatably mounted on the lower traveling body 1, a driver's cab 10 provided on the upper rotating body 3, an external sound collection device M1 located outside the driver's cab 10, and an internal sound output device SP2 located inside the driver's cab 10. The machine has an external sound playback function that allows the sound collected by the external sound collection device M1 to be output by the internal sound output device SP2, and the external sound playback function is configured to switch to an enabled state according to conditions based on people in the surroundings. According to this embodiment, since the external sound playback function is switched to an enabled state according to conditions based on people in the surroundings of the work machine 100, it is possible to reduce the chances of missing speech from people in the surroundings of the work machine 100.

[0177] The conditions based on people in the surroundings may include the fact that the people in the surroundings have performed a preliminary action to speak to the work machine 100. According to this embodiment, the external sound playback function is switched to an enabled state when a person in the surroundings of the work machine 100 performs a preliminary action, thereby reducing the chance of missing the beginning of what the person in the surroundings of the work machine 100 is saying.

[0178] The pre-action may include at least one of the following: uttering a predetermined phrase, uttering a predetermined pitch, preparing for utterance, a predetermined gesture, a predetermined action, or a predetermined terminal operation. According to this embodiment, the external sound playback function is switched to an enabled state based on various pre-actions, so the pre-actions can be flexibly set according to the work site.

[0179] The conditions based on surrounding people may include ensuring that the prior action does not fall under a predetermined exclusion action. According to this embodiment, if the action falls under a predetermined exclusion action, the external sound playback function is not switched to an enabled state, thus suppressing unnecessary listening to sounds around the work machine 100.

[0180] The active state may be a state in which the sound collected by the external sound collection device M1 is output from the internal sound output device SP2. According to this embodiment, the sounds around the work machine 100 are reproduced inside the operator's cab 10 according to conditions based on the people around the work machine 100, so that the chances of missing speech from people around the work machine 100 can be reduced.

[0181] The work machine 100 may further include an internal sound collection device M2 located inside the operator's cab 10 and an external sound output device SP1 located outside the operator's cab 10. The work machine 100 may further have an internal sound playback function that enables the sound collected by the internal sound collection device M2 to be output by the external sound output device SP1. According to this embodiment, in a two-way conversation between the operator of the work machine 100 and people around the work machine 100, it is possible to reduce the chances of missing speech from people around the work machine 100.

[0182] The work machine 100 may be configured to switch off the external sound playback function in accordance with a second condition based on the people in its vicinity. According to this embodiment, since the external sound playback function is switched off in accordance with the condition based on the people in the vicinity of the work machine 100, it is possible to suppress the unnecessary continuous listening of sounds in the vicinity of the work machine 100.

[0183] A second condition based on the surrounding people may include the fact that the surrounding people do not perform any preliminary actions for speaking to the work machine 100 for a certain period of time. According to this embodiment, the external sound playback function is switched to an inactive state when the people surrounding the work machine 100 do not perform any preliminary actions for a certain period of time, thereby preventing unnecessary listening to sounds around the work machine 100.

[0184] The work machine 100 may further include a display device D1 that displays an image of the surroundings of the work machine 100 and information indicating the operating status of the external sound playback function. According to this embodiment, the operating status of the external sound playback function can be checked while checking the surroundings of the work machine 100, thereby improving safety.

[0185] [Another Configuration Example of the Work Machine] Next, with reference to Figure 12, another configuration example of the work machine 100 will be described. Figure 12 is a top view of another configuration example of the work machine 100. The work machine 100 shown in Figure 12 differs from the work machine 100 shown in Figure 1 in that the external sound output device SP1 is composed of four speakers (front speaker SP1F, left speaker SP1L, right speaker SP1R, and rear speaker SP1B). In the work machine 100 shown in Figure 1, the external sound output device SP1 is composed of one omnidirectional speaker installed above the operator's cab 10.

[0186] With this configuration, the work machine 100 shown in Figure 12 can, for example, output sound towards an operator WK in front of the work machine 100 without outputting sound towards an operator WK to the left, right, or rear of the work machine 100, by turning on the front speaker SP1F (to enable sound output) and turning off the left speaker SP1L, right speaker SP1R, and rear speaker SP1B (to disable sound output).

[0187] Furthermore, in the example shown in Figure 12, a front camera S6F and a front microphone M1F are provided adjacent to the front speaker SP1F, and a front light bar G1F is provided on the housing of the front microphone M1F. Also, a left camera S6L and a left microphone M1L are provided adjacent to the left speaker SP1L, and a left light bar G1L is provided on the housing of the left microphone M1L. Also, a right camera S6R and a right microphone M1R are provided adjacent to the right speaker SP1R, and a right light bar G1R is provided on the housing of the right microphone M1R. Also, a rear camera S6B and a rear microphone M1B are provided adjacent to the rear speaker SP1B, and a rear light bar G1B is provided on the housing of the rear microphone M1B.

[0188] With this configuration, the work machine 100 can turn on (make the light bar that corresponds to the speaker that is turned on) and turn off (make the light bar that corresponds to the speaker that is turned off)

[0189] The external sound output device SP1 may consist of one or more parametric speakers. A parametric speaker is an ultrasonic speaker that can selectively transmit sound to people within a specific narrow range. A parametric speaker can transmit sound to any desired location.

[0190] In the work machine 100 shown in Figure 12, the controller 30 may detect workers WK around the work machine 100 based on the image captured by the imaging device S6 and identify the positions of the workers WK. If there are multiple workers WK around the work machine 100, the controller 30 may distinguish between those being spoken to (workers WK who are speaking) and those not being spoken to (workers WK who are not speaking) based on the output of the four external sound collection devices M1. The controller 30 may also distinguish between those being spoken to (workers WK facing the work machine 100) and those not being spoken to (workers WK not facing the work machine 100) based on the image captured by the imaging device S6. The controller 30 may also turn on the speaker and light bar that are facing the workers WK. For example, if there is a worker WK (who is speaking) behind the work machine 100, the controller 30 may turn on the rear speaker SP1B while keeping the front speaker SP1F, left speaker SP1L, and right speaker SP1R off. In this case, the controller 30 may also turn on the rear light bar G1B while keeping the front light bar G1F, left light bar G1L, and right light bar G1R off. Such a function may be implemented in the work machine 100 shown in Figures 1 to 5.

[0191] This configuration allows the controller 30 to output sound in the direction where worker WK is located, without outputting sound in the direction where worker WK is not located. Therefore, worker WK can easily recognize whether they are being spoken to or not.

[0192] [Operating System for Work Machines] Next, with reference to Figure 13, an example of the configuration of the operating system SYS according to the embodiment of this disclosure will be described. Figure 13 is a schematic diagram showing an example of the configuration of the operating system SYS. As shown in Figure 13, the operating system SYS includes a work machine 100, a remote control room RC, and a management center MC. Note that the detailed configuration of the work machine 100 is omitted from Figure 13. This is because the work machine 100 shown in Figure 13 has the same configuration as the work machine 100 shown in Figure 1 or Figure 12.

[0193] The work machine 100, the remote control room RC, and the management center MC are connected to each other so that data can be sent and received via a communication network NW. Alternatively, the work machine 100, the remote control room RC, and the management center MC may be connected to each other so that data can be sent and received directly without using the communication network NW. In the illustrated example, the work machine 100 transmits information about the work site to the remote control room RC. This allows the remote operator RO in the remote control room RC to understand the situation at the work site based on the information from the work machine 100.

[0194] The work machine 100 is equipped with sensors capable of recognizing the position and shape of objects present at the work site in three dimensions. For example, the work machine 100 is equipped with a spatial recognition device. Therefore, the work machine 100 can transmit the results of three-dimensional measurements of the work site to the remote control room RC.

[0195] The spatial recognition device is a device for recognizing the space surrounding the work machine 100. In the illustrated example, the spatial recognition device is a LiDAR. The LiDAR measures the distance between each of more than one million points within the monitoring range and the LiDAR itself. Note that the spatial recognition device can be any device capable of measuring the distance to an object. For example, the spatial recognition device may be a stereo camera, or a combination of an imaging device S6 and a ranging device such as a millimeter-wave radar.

[0196] The operating system SYS may include one work machine 100 or multiple work machines. If the system includes multiple work machines 100, the remote operator RO of a particular work machine 100 can obtain information about the work site obtained by that particular work machine 100, as well as information about the work site obtained by one or more other work machines 100.

[0197] The remote control room RC is equipped with a communication device T2, a remote controller 40, an operating device 42, an operating sensor 43, a display device D1E, an internal sound collection device M2E, and an internal sound output device SP2E. The remote control room RC also has an operating seat DS where the remote operator RO sits to remotely control the work machine 100.

[0198] The communication device T2 is configured to communicate with the communication device T1 attached to the work machine 100.

[0199] The remote controller 40 is an arithmetic unit that performs various calculations. In this embodiment, the remote controller 40 is composed of a microcomputer including a CPU and memory. The various functions of the remote controller 40 are realized by the CPU executing a program stored in memory.

[0200] The remote controller 40 may be configured in the same way as the controller 30 shown in Figure 7. That is, the remote controller 40 may include an acquisition unit 301, a detection unit 302, a noise reduction unit 303, a state switching unit 304, an operation reception unit 305, an output control unit 306, and a display control unit 307.

[0201] The display device D1E is a device capable of displaying various types of information. The display device D1E displays images based on information transmitted from the work machine 100 so that the remote operator RO in the remote control room RC can visually inspect the area around the work machine 100. In the illustrated example, the display device D1E is a liquid crystal display that displays images captured by the imaging device S6 mounted on the work machine 100. The display device D1E may also be a display or projector that enables naked-eye stereoscopic viewing, or it may be VR goggles or the like.

[0202] The internal sound output device SP2E is a device capable of outputting various types of sound information. The internal sound output device SP2E outputs sound based on information transmitted from the work machine 100 so that the remote operator RO in the remote control room RC can hear the sounds emitted at the work site. The internal sound output device SP2E may be configured to output sound captured by an external sound collection device M1 installed outside the driver's cab 10, or it may be configured to output sound captured by an internal sound collection device M2 installed inside the driver's cab 10. In this case, the internal sound collection device M2 may be installed at a position corresponding to the ear position of the operator seated in the driver's seat 50 inside the driver's cab 10. The internal sound output device SP2E may be a stationary device such as a speaker, or a wearable device such as earphones or headphones. The speaker may be a monaural speaker, a stereo speaker, or a surround speaker. The speaker may also be an omnidirectional speaker or a directional speaker. The wearable device may have noise-canceling capabilities, spatial audio capabilities (3D sound capabilities), or bone conduction capabilities.

[0203] The operating device 42 is equipped with an operation sensor 43 for detecting the operation of the operating device 42. The operation sensor 43 is, for example, a tilt sensor that detects the tilt angle of the operating lever, or an angle sensor that detects the oscillation angle of the operating lever around its pivot axis. The operation sensor 43 may also consist of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor 43 outputs information regarding the detected operation of the operating device 42 to the remote controller 40. The remote controller 40 generates an operation signal based on the received information and transmits the generated operation signal to the work machine 100. The operation sensor 43 may be configured to generate the operation signal. In this case, the operation sensor 43 may output the operation signal to the communication device T2 without going through the remote controller 40. With this configuration, the remote operator RO can remotely operate the work machine 100 from the remote control room RC.

[0204] The control center MC is a facility equipped with various devices for managing the remote operation of the work machine 100 located at the work site, or by a remote operator RO located in the remote control room RC. In the illustrated example, the control center MC is installed at a distance from both the work site of the work machine 100 and the remote control room RC. The control center MC is equipped with a management device 200, an internal sound collection device M2C, and an internal sound output device SP2C.

[0205] The management device 200 is an example of a control device, and is, for example, a server computer (a so-called cloud server) or an edge server. The management device 200 is typically a fixed terminal device, but may also be a portable terminal device (for example, a laptop computer, tablet, or smartphone).

[0206] With this configuration, a manager in the control center MC can, for example, use a sound collection device (external sound collection device M1 or internal sound collection device M2) and an internal sound output device SP2C attached to the work machine 100 to hear sounds emitted at the work site. Also, a manager in the control center MC can, for example, use an internal sound collection device M2E and an internal sound output device SP2C installed in the remote control room RC to hear sounds emitted in the remote control room RC. Furthermore, a manager in the control center MC can, for example, use an internal sound collection device M2C and an external sound output device SP1 attached to the work machine 100 to transmit their own voice to workers WK around the work machine 100. Also, a manager in the control center MC can, for example, use an internal sound collection device M2C and an internal sound output device SP2 attached to the work machine 100 to transmit their own voice to the operator OP of the work machine 100. Furthermore, the administrator at the control center (MC) can, for example, use the internal sound collection device (M2C) and the internal sound output device (SP2E) installed in the remote control room (RC) to transmit their own voice to the remote operator (RO) in the remote control room (RC).

[0207] In the aforementioned operating system SYS, the sounds surrounding the work machine 100 collected by the external sound collection device M1 are output from the internal sound output device SP2E. At that time, the remote controller 40 switches the sound output function to an enabled state according to conditions based on the people around the work machine 100. This reduces the chance that the remote operator RO will miss hearing speech from people around the work machine 100, even when the work machine 100 is being operated remotely.

[0208] Preferred embodiments of the present disclosure have been described above. However, the inventions of the present disclosure are not limited to the embodiments described above. Various modifications, substitutions, etc., can be applied to the embodiments described above without departing from the scope of the inventions of the present disclosure. Furthermore, each of the features described with reference to the embodiments described above may be combined as appropriate, as long as they do not contradict each other technically.

[0209] This application claims priority to Japanese Patent Application No. 2024-228371, filed with the Japan Patent Office on December 25, 2024, which is incorporated herein by reference to its entire contents.

[0210] 1...Lower running body 1C...Crawler 1CL...Left crawler 1CR...Right crawler 2...Swing mechanism 2A...Swing hydraulic motor 2ML...Left running hydraulic motor 2MR...Right running hydraulic motor 3...Upper swing body 4...Boom 5...Arm 6...Bucket 7...Boom cylinder 8...Arm cylinder 9...Bucket cylinder 10...Operator's cab 11...Engine 13...Regulator 14...Main pump 15...Pilot pump 17...Control valve unit 26...Operating device 26DL...Left running lever 26DR...Right running lever 26L...Left operating lever 26PL...Left running pedal 26PR...Right running pedal 26R...Right operating lever 27...Lever boot 28...Discharge pressure sensor 29...Operating sensor 30...Controller 31...Valve 40...Remote controller 42...Operating device 43...Operating sensor 50...Driver's seat 51...Seat 52...Backrest 53L...Left armrest 53R...Right armrest 54L...Left console 54R...Right console 55...Gate bar 56...Window side console 85...Display screen 100...Working machine 200...Management device 171-176...Control valve 421...Working machine status display area 421a...Direction indicator icon 421b...Working machine icon 421e, 421f,421g... Human detection icon 422... First image display area 422a... Human image 422b... Frame 422c... Edge image 423... Second image display area 423a... Human image 423b... Frame 423c... Counterweight image 424... Notification display area AT... Attachment D1... Display device D1a... Image display unit D1E... Display device D2... Input device DL... Volume dial DS... Operator's seat G1... Information transmission device G1B... Rear light bar G1F... Front light bar G1L... Left light bar G1R... Right light bar GL... Gate lock lever HS... Horn button IAD... Information acquisition device KS... Speech button M1... External sound collection device M1B... Rear microphone M1F... Front microphone M1L... Left microphone M1R... Right microphone M2, M2C, M2E... Internal sound collection device OP... Operator PS... Positioning device RC... Remote control room RO... Remote operator S1... Boom angle sensor S2... Arm angle sensor S3... Bucket angle sensor S4... Machine tilt sensor S5... Swivel sensor S6... Imaging device S6B... Rear camera S6F... Front camera S6L... Left camera S6R... Right camera SP1... External sound output device SP1B... Rear speaker SP1F... Front speaker SP1L... Left speaker SP1R... Right speaker SP2, SP2C, SP2E... Internal sound output device SP2L... Left room speaker SP2R... Right room speaker SW... Switch SYS... Operating system T1, T2... Communication device WD... Window WK... Operator

Claims

1. A work machine comprising: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; a driver's cab provided on the upper rotating body; an external sound collection device located outside the driver's cab; and an internal sound output device located inside the driver's cab, wherein the machine has an external sound playback function that enables the output of sound collected by the external sound collection device through the internal sound output device, and the external sound playback function is configured to switch to an enabled state according to conditions based on people in the surrounding area.

2. The working machine according to claim 1, wherein the condition based on the surrounding persons includes the fact that the surrounding persons have performed a preliminary action to speak toward the working machine.

3. The work machine according to claim 2, wherein the pre-action includes at least one of uttering a predetermined phrase, uttering a predetermined pitch, preparing for utterance, a predetermined gesture, a predetermined action, or a predetermined terminal operation.

4. The working machine according to claim 2, wherein the condition based on the surrounding persons includes the prior action not falling under a predetermined exclusion action.

5. The working machine according to claim 1, wherein the effective state is a state in which the sound collected by the external sound collection device is output from the internal sound output device.

6. The work machine according to any one of claims 1 to 5, further comprising: an internal sound collection device disposed inside the driver's cab; and an external sound output device disposed outside the driver's cab, and further having an internal sound playback function that enables the sound collected by the internal sound collection device to be output by the external sound output device.

7. The work machine according to any one of claims 1 to 5, wherein the external sound playback function is configured to switch to an disabled state in accordance with a second condition based on the surrounding people.

8. The work machine according to claim 7, wherein the second condition includes that the surrounding persons refrain from taking any action for a certain period of time to speak to the work machine.

9. The work machine according to any one of claims 1 to 5, further comprising a display device that displays an image of the surroundings of the work machine and information indicating the operating status of the external sound playback function.

10. An operating system for a work machine comprising: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; and an external sound collection device provided on the upper rotating body; and a sound output device provided around an operator's seat for operating the work machine, wherein the system has an external sound playback function that enables the sound collected by the external sound collection device to be output by the sound output device, and the external sound playback function is configured to switch to an enabled state according to conditions based on people around the work machine.

Images