Remote control system for work machine

Abstract

To more reliably convey to an operator a sound that the operator wants to hear.SOLUTION: A remote control system SYS for a work machine 100 having a lower traveling body 1, an upper rotating body 3 rotatably mounted on the lower traveling body 1, and a sound collection device M1 is also provided with a control device CTR and a sound output device V1. The control device CTR generates playback sound data based on data of original sounds collected by the sound collection device M1 and registration information of sounds generated by a predetermined sound generation source, and outputs the generated playback sound from the sound output device V1.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a remote control system for a work machine. [Background technology]

[0002] Conventionally, there is known a wireless remote control system for a work machine that uses wireless communication to operate an unmanned work machine deployed at a work site from a control room located remotely away from the work site (see Patent Document 1).In an environment where a plurality of control room devices corresponding to a plurality of work machines are installed in a single control room, this system uses narrow directional speakers to prevent mixing of sounds provided to each operator with sounds provided to other operators, so that each operator does not mistake the sound of the work machine they are operating. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-354479 Summary of the Invention [Problem to be solved by the invention]

[0004] However, this system outputs the sounds collected by the sound collection device installed on each work machine from narrow-directivity speakers installed on each control room device, so if the sounds collected by the sound collection device contain a lot of noise (sounds other than the sounds the operator wants to hear), there is a risk that the sounds the operator wants to hear will not be conveyed to them.

[0005] Therefore, it is desirable to be able to more reliably convey to the operator the sounds that the operator wants to hear. [Means for solving the problem]

[0006] A remote control system for a work machine according to an embodiment of the present disclosure is a remote control system for a work machine that includes a lower running body, an upper rotating body that is rotatably mounted on the lower running body, and a sound collection device, and is also provided with a control device and a sound output device, wherein the control device generates playback sound data based on data of original sound collected by the sound collection device and registration information of sound generated by a specified sound generation source, and outputs the playback sound from the sound output device. [Effects of the Invention]

[0007] In the above-described remote control system for a work machine, the operator can more reliably hear the sounds he or she wants to hear. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic diagram illustrating a configuration example of a remote control system according to an embodiment of the present disclosure. [Figure 2] 2 is a diagram showing an example of the configuration of a drive control system of a work machine that constitutes the remote control system of FIG. 1. FIG. [Figure 3] 2 is a functional block diagram showing an example of the configuration of the remote control system of FIG. 1. FIG. [Figure 4] FIG. 1 is a frequency spectrum diagram showing an example of the relationship between the volume and frequency of sounds generated at a work site. [Figure 5] 10 is a flowchart showing an example of the flow of a playback sound generation process. [Figure 6] 10 is a flowchart showing another example of the flow of the playback sound generation process. [Figure 7] FIG. 10 is a schematic diagram illustrating another configuration example of a remote control system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The embodiments described below are merely examples and do not limit the invention, and all features and combinations thereof related to the embodiments are not necessarily essential to the invention. In addition, identical or corresponding components in each drawing are denoted by identical or corresponding reference numerals, and descriptions thereof may be omitted.

[0010] First, an overview of a remote control system SYS according to an embodiment of the present disclosure will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing an example configuration of a remote control system SYS according to an embodiment of the present disclosure.

[0011] <Devices that make up the remote control system> As shown in FIG. 1, a remote control system SYS according to an embodiment of the present disclosure includes a work machine 100 and a remote control room RC.

[0012] The work machine 100 and the remote control room RC are connected via a communication network NW so as to be able to send and receive data.

[0013] The work machine 100 transmits information about the work site to the remote control room RC. This allows the remote control room RC to check the work site from multiple angles in accordance with the detection results from the work machine 100.

[0014] The work machine 100 is equipped with a sensor that can three-dimensionally recognize the position and shape of objects present at the work site. For example, the work machine 100 is equipped with a spatial recognition device S7. Therefore, the work machine 100 can transmit the results of three-dimensionally measuring the work site to the remote control room RC.

[0015] The spatial recognition device S7 may be a LIDAR for monitoring a work site. The LIDAR measures the distance between the LIDAR and, for example, one million or more points within a monitoring range. The spatial recognition device S7 may be any device capable of measuring the distance to an object. For example, the spatial recognition device S7 may be a stereo camera or a combination of an imaging device and a distance measuring device such as a millimeter-wave radar.

[0016] The remote control system SYS may include one or more work machines 100. This allows the remote control system SYS to transmit information about the work site to the remote control room RC via the multiple work machines 100.

[0017] <Example of remote control room configuration> The remote control room RC is equipped with a communication device T2, a remote controller 40, an operation device 42, an operation sensor 43, a display device D1, and a sound output device V1. The remote control room RC also is equipped with an operation seat DS where a remote operator OP who remotely operates the work machine 100 sits.

[0018] The communication device T2 is configured to be able to communicate with the communication device T1 attached to the work machine 100.

[0019] The remote controller 40 is a computing device that executes various calculations. In this embodiment, the remote controller 40 is configured as a microcomputer including a CPU and a memory. The various functions of the remote controller 40 are realized by the CPU executing programs stored in the memory.

[0020] The display device D1 is a device capable of displaying various types of information. The display device D1 displays images based on information transmitted from the work machine 100 so that the remote operator OP in the remote control room RC can visually recognize the surroundings of the work machine 100. In this embodiment, the display device D1 is a liquid crystal display. The display device D1 may be a display or projector that realizes naked-eye stereoscopic vision, or may be VR goggles or the like.

[0021] The sound output device V1 is a device capable of outputting various types of sound information. The sound output device V1 outputs sound based on information transmitted from the work machine 100 so that the remote operator OP in the remote control room RC can hear sounds being emitted around the work machine 100. In this embodiment, the sound output device V1 may be an installed device such as a speaker, or may be a wearable device such as earphones or headphones. The speaker may be a monaural speaker, a stereo speaker, or a surround speaker. The speaker may also be an omnidirectional speaker or a superdirectional speaker. The wearable device may be one that has a noise canceling function, a spatial audio function (stereophonic function), or a bone conduction function.

[0022] The operation device 42 is provided with an operation sensor 43 for detecting the operation content of the operation device 42. The operation sensor 43 is, for example, an inclination sensor that detects the inclination angle of the operation lever, or an angle sensor that detects the swing angle of the operation lever around the swing axis. The operation sensor 43 may be configured with other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor 43 outputs information related to the detected operation content of the operation device 42 to the remote controller 40. The remote controller 40 generates an operation signal based on the received information and transmits the generated operation signal to the work machine 100. The operation sensor 43 may be configured to generate an operation signal. In this case, the operation sensor 43 may output the operation signal to the communication device T2 without passing through the remote controller 40. This allows the remote operator OP to remotely operate the work machine 100 from the remote control room RC. <Work machine configuration> The work machine 100 is, for example, a manually operated work machine that operates in response to operation by an operator or a remote operator OP. In the illustrated example, the work machine 100 is an excavator (shovel), but it may also be another work machine equipped with a lower traveling body and an upper rotating body, such as a jib crane or a harbor crane (gantry crane).

[0023] Specifically, an upper rotating body 3 is rotatably mounted on a lower traveling body 1 of the work machine 100 via a rotating mechanism 2. A boom 4 is attached to the upper rotating body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6 is attached to the tip of the arm 5 as an end attachment. In the illustrated example, the lower traveling body 1 is configured to include crawlers, but it may also be configured to include wheels.

[0024] The boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of an attachment, and are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively. A boom angle sensor S1 is attached to the boom 4, an arm angle sensor S2 is attached to the arm 5, and a bucket angle sensor S3 is attached to the bucket 6.

[0025] The boom angle sensor S1 detects the rotation angle of the boom 4. In this embodiment, the boom angle sensor S1 is an acceleration sensor, and can detect the boom angle, which is the rotation angle of the boom 4 relative to the upper rotating body 3. For example, the boom angle is at its minimum when the boom 4 is lowered to the lowest position, and increases as the boom 4 is raised.

[0026] The arm angle sensor S2 detects the rotation angle of the arm 5. In this embodiment, the arm angle sensor S2 is an acceleration sensor, and can detect the arm angle, which is the rotation angle of the arm 5 relative to the boom 4. For example, the arm angle is at its smallest when the arm 5 is fully closed, and increases as the arm 5 opens.

[0027] The bucket angle sensor S3 detects the rotation angle of the bucket 6. In this embodiment, the bucket angle sensor S3 is an acceleration sensor, and can detect the bucket angle, which is the rotation angle of the bucket 6 with respect to the arm 5. For example, the bucket angle is at its smallest when the bucket 6 is fully closed, and increases as the bucket 6 opens.

[0028] The boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 may be a potentiometer using a variable resistor, a stroke sensor that detects the stroke amount of the corresponding hydraulic cylinder, or a rotary encoder that detects the rotation angle around the connecting pin. The boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 form a posture sensor that detects the posture of the excavation attachment.

[0029] The upper rotating body 3 is equipped with a cabin 10 as a driver's cab, an engine 11, an aircraft tilt sensor S4, a turning angular velocity sensor S5, an imaging device S6, a spatial recognition device S7, a positioning device S8, a communication device T1, and a sound collection device M1 (outdoor sound collection device M1E), etc.

[0030] An excavator controller 30 and a sound collection device M1 (indoor sound collection device M1N) are installed inside the cabin 10. A driver's seat, operating devices, etc. are also installed inside the cabin 10. The cabin 10 may be omitted. In this case, the work machine 100 is operated via an operating device 42 installed in the remote control room RC, or operates autonomously without being operated by an operator.

[0031] The shovel controller 30 is an example of a control device CTR. In the illustrated example, the shovel controller 30 is a calculation device (electronic circuit) that executes various calculations, is provided inside the cabin 10, and controls the drive of the work machine 100. The functions of the shovel controller 30 are realized by any hardware, software, or a combination thereof. For example, the shovel controller 30 is configured mainly with a microcomputer that includes a CPU, a volatile storage device such as RAM, a non-volatile storage device such as ROM, and various interface devices for input and output.

[0032] The engine 11 is a drive source for the work machine 100. In this embodiment, the engine 11 is a diesel engine. The output shaft of the engine 11 is connected to the input shafts of the main pump 14 and the pilot pump 15. The drive source for the work machine 100 may be another internal combustion machine such as a hydrogen engine, or may be a combination of an electric motor with a power generation device such as a fuel cell or an electricity storage device such as a lithium-ion battery.

[0033] The machine body tilt sensor S4 is configured to detect the tilt of the upper rotating body 3 with respect to a predetermined plane. In this embodiment, the machine body tilt sensor S4 is an acceleration sensor that detects the tilt angle about the longitudinal axis and the lateral axis of the upper rotating body 3 with respect to a horizontal plane. The longitudinal axis and the lateral axis of the upper rotating body 3 are, for example, perpendicular to each other and pass through the center point of the work machine 100, which is a point on the rotation axis of the work machine 100.

[0034] The rotation angular velocity sensor S5 is configured to detect the rotation angular velocity of the upper rotating body 3. In this embodiment, the rotation angular velocity sensor S5 is a gyro sensor. The rotation angular velocity sensor S5 may be a resolver, a rotary encoder, or the like. The rotation angular velocity sensor S5 may detect a rotation speed. The rotation speed may be calculated from the rotation angular velocity.

[0035] The imaging device S6 is configured to acquire images of the periphery of the work machine 100. In this embodiment, the imaging device S6 includes a front camera S6F that images the space in front of the work machine 100, a left camera S6L that images the space to the left of the work machine 100, a right camera S6R that images the space to the right of the work machine 100, and a rear camera S6B that images the space behind the work machine 100. The imaging device S6 is, for example, a monocular camera having an imaging element such as a CCD or CMOS, and transmits the captured images to the remote control room RC. In the illustrated example, the front camera S6F is attached to the roof of the cabin 10, the left camera S6L is attached to the left end of the top surface of the upper rotating body 3, the right camera S6R is attached to the right end of the top surface of the upper rotating body 3, and the rear camera S6B is attached to the rear end of the top surface of the upper rotating body 3.

[0036] The spatial recognition device S7 is configured to recognize the state of the space around the work machine 100. The spatial recognition device S7 includes a rear spatial recognition device S7B that recognizes the state of the space behind the work machine 100, a left spatial recognition device S7L that recognizes the state of the space to the left of the work machine 100, a right spatial recognition device S7R that recognizes the state of the space to the right of the work machine 100, and a front spatial recognition device S7F that recognizes the state of the space in front of the work machine 100. The spatial recognition device S7 may be a LIDAR. Note that the spatial recognition device S7 may be any device that is capable of measuring the distance to an object. For example, the spatial recognition device S7 may be a stereo camera, a range imaging camera, or a ranging device such as millimeter-wave radar. When millimeter-wave radar or the like is used as the spatial recognition device S7, the spatial recognition device S7 may emit a number of signals (laser light, etc.) toward an object and receive the reflected signals, thereby deriving the distance and direction of the object from the reflected signals. In the illustrated example, the rear space recognition device S7B is attached to the rear end of the upper surface of the upper rotating body 3, the left space recognition device S7L is attached to the left end of the upper surface of the upper rotating body 3, the right space recognition device S7R is attached to the right end of the upper surface of the upper rotating body 3, and the forward space recognition device S7F is attached to the front end of the upper surface of the cabin 10.

[0037] The spatial recognition device S7 may be configured to detect a predetermined object within a predetermined area set around the work machine 100. For example, the spatial recognition device S7 may have a human detection function and be configured to be able to detect a person while distinguishing between people and objects other than people.

[0038] The positioning device S8 is configured to acquire information relating to the position of the work machine 100. In this embodiment, the positioning device S8 is configured to measure the position and orientation of the work machine 100 in a reference coordinate system. Specifically, the positioning device S8 is a GNSS receiver with an integrated electronic compass, and measures the latitude, longitude, and altitude of the current position of the work machine 100, and measures the orientation of the work machine 100. The reference coordinate system is, for example, the World Geodetic System.

[0039] The communication device T1 is configured to control communications with devices external to the work machine 100. In this embodiment, the communication device T1 is configured to control communications between the communication device T1 and devices external to the work machine 100 via a wireless communication network. The communication device T1 includes, for example, a mobile communication module compatible with mobile communication standards such as LTE, 4G, or 5G, or a satellite communication module for connecting to a satellite communication network. The communication device T1 may also be configured to control wireless communications between the work machine 100 and an external GNSS surveying system, for example.

[0040] The sound collection device M1 is configured to acquire sounds emitted around the work machine 100. In this embodiment, the sound collection device M1 is an omnidirectional microphone. However, the sound collection device M1 may also be a directional microphone. In the illustrated example, the sound collection device M1 includes an outdoor sound collection device M1E and an indoor sound collection device M1N. The outdoor sound collection device M1E, located outside the cabin 10, is provided on the roof of the cabin 10. This is to enable the sound generated around the work machine 100 (front, back, left, and right) to be collected as evenly as possible. The indoor sound collection device M1N, located inside the cabin 10, is provided at a position corresponding to the position of the ears of the operator sitting in the driver's seat. This is to enable the remote operator OP to experience a sense of realism as if he or she were sitting in the driver's seat inside the cabin 10 when he or she hears the sound collected by the indoor sound collection device M1N through the sound output device V1. Note that either the outdoor sound collector M1E or the indoor sound collector M1N may be omitted. Also, in the illustrated example, the outdoor sound collector M1E and the indoor sound collector M1N are each configured as a single sound collector, but at least one of the outdoor sound collector M1E and the indoor sound collector M1N may be configured as a combination of multiple sound collectors, and may be configured to enable stereophonic recording or binaural recording, etc., in order to realize a stereophonic sound system or a 3D sound system.

[0041] Fig. 2 is a diagram showing an example of the configuration of a drive control system of the work machine 100. In Fig. 2, the mechanical power transmission system is indicated by double lines, the hydraulic oil lines are indicated by thick solid lines, the pilot lines are indicated by dashed lines, and the electric drive and control system is indicated by dotted lines.

[0042] The drive system of the work machine 100 according to this embodiment includes the engine 11, regulator 13, main pump 14, and control valve unit 17. Furthermore, the hydraulic drive system of the work machine 100 according to this embodiment includes actuators AC (hydraulic actuators) such as the left traveling hydraulic motor 1L, right traveling hydraulic motor 1R, swing hydraulic motor 2A, boom cylinder 7, arm cylinder 8, and bucket cylinder 9, which hydraulically drive the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6, respectively, as described above.

[0043] The engine 11 is a power source for the work machine 100, and is mounted, for example, on the rear of the upper rotating body 3. Specifically, the engine 11 rotates at a preset target rotation speed under direct or indirect control by the shovel controller 30, and drives the main pump 14 and the pilot pump 15. In the illustrated example, the engine 11 is a diesel engine that uses light oil as fuel.

[0044] The regulator 13 controls the discharge amount of the main pump 14. For example, the regulator 13 adjusts the angle (tilting angle) of the swash plate of the main pump 14 in response to a control command from the shovel controller 30.

[0045] The main pump 14 is mounted, for example, on the rear of the upper rotating body 3, similar to the engine 11, and supplies hydraulic oil to the control valve unit 17 through a hydraulic oil line. As described above, the main pump 14 is driven by the engine 11. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the excavator controller 30, the tilt angle of the swash plate is adjusted by the regulator 13, thereby adjusting the stroke length of the piston and controlling the discharge flow rate.

[0046] The control valve unit 17 is a hydraulic control device that controls the hydraulic system in the work machine 100. In this embodiment, the control valve unit 17 includes control valves 171 to 176. The control valve unit 17 is configured to selectively supply hydraulic oil discharged by the main pump 14 to one or more hydraulic actuators via the control valves 171 to 176. The control valves 171 to 176 control, for example, the flow rate of hydraulic oil flowing from the main pump 14 to the hydraulic actuators and the flow rate of hydraulic oil flowing from the hydraulic actuators to a hydraulic oil tank. The hydraulic actuators include a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, a left traveling hydraulic motor 1L, a right traveling hydraulic motor 1R, and a swing hydraulic motor 2A. More specifically, the control valve 171 corresponds to the left traveling hydraulic motor 1L, the control valve 172 corresponds to the right traveling hydraulic motor 1R, and the control valve 173 corresponds to the swing hydraulic motor 2A. Furthermore, the control valve 174 corresponds to the bucket cylinder 9 , the control valve 175 corresponds to the boom cylinder 7 , and the control valve 176 corresponds to the arm cylinder 8 .

[0047] The pilot pump 15 is an example of a pilot pressure generating device, and is configured to supply hydraulic oil to hydraulic control devices via a pilot line. In this embodiment, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pressure generating device may be realized by the main pump 14. That is, the main pump 14 may have a function of supplying hydraulic oil to various hydraulic control devices via a pilot line, in addition to a function of supplying hydraulic oil to the control valve unit 17 via a hydraulic oil line. In this case, the pilot pump 15 may be omitted.

[0048] The operating device 26 is a device used by an operator to operate the actuator AC. The actuator AC includes at least one of a hydraulic actuator and an electric actuator. In the illustrated example, the operating device 26 is a hydraulic operating device, but it may also be an electric operating device.

[0049] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In this embodiment, the discharge pressure sensor 28 outputs the detected value to the shovel controller 30.

[0050] The operation sensor 29 is configured to detect the operation content of the operator using the operating device 26. The operation sensor 29 is, for example, an inclination sensor that detects the inclination angle of the operating lever, or an angle sensor that detects the swing angle of the operating lever around the swing axis. The operation sensor 29 may be configured with other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. In this embodiment, the operation sensor 29 detects the operation direction and operation amount of the operating device 26 corresponding to each actuator AC and outputs the detected values ​​to the shovel controller 30. In this embodiment, the shovel controller 30 controls the proportional valve 31 in accordance with the output of the operation sensor 29. Then, the shovel controller 30 supplies the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17. The pressure of the hydraulic oil supplied to each pilot port (pilot pressure) is, in principle, a pressure corresponding to the operation direction and operation amount of the operating device 26 corresponding to each hydraulic actuator. In this way, the operating device 26 is configured to supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17 .

[0051] The proportional valve 31, which functions as a control valve for machine control, is disposed in a pipe connecting the pilot pump 15 and the pilot port of the control valve in the control valve unit 17, and is configured to be able to change the pressure of the hydraulic oil in that pipe. In this embodiment, the proportional valve 31 operates in response to a control command output by the shovel controller 30. Therefore, the shovel controller 30 can supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the control valve in the control valve unit 17 via the proportional valve 31, regardless of the operation of the operating device 26 by the operator.

[0052] With this configuration, the shovel controller 30 can operate the hydraulic actuator corresponding to a specific operating device 26 even when no operation is being performed on that specific operating device 26.

[0053] For example, the shovel controller 30 sets a target rotation speed based on a work mode or the like that is set in advance by a predetermined operation by an operator or the like, and performs drive control to rotate the engine 11 at that target rotation speed. Also, for example, the shovel controller 30 outputs a control command to the regulator 13 as necessary to change the discharge rate of the main pump 14. Also, the shovel controller 30 can execute control related to a machine guidance function that guides (guidances) the operator's manual operation of the work machine 100 via the operation device 26. Also, the shovel controller 30 can execute control related to a machine control function that automatically assists the operator's manual operation of the work machine 100 via the operation device 26.

[0054] Note that some of the functions of the shovel controller 30 may be realized by another controller (control device). That is, the functions of the shovel controller 30 may be realized in a distributed manner by a plurality of controllers. For example, the machine guidance function and the machine control function may be realized by a dedicated controller (control device).

[0055] <Block diagram of the remote control system> Fig. 3 is a functional block diagram showing an example configuration of the remote control system SYS according to this embodiment. Specifically, Fig. 3 shows the block configurations of the remote control room RC and the work machine 100 included in the remote control system SYS. The hardware configuration of the work machine 100 has been described above, so a description thereof will be omitted. <Configuration of the remote control room RC> The remote control room RC includes a remote controller 40, a communication device T2, an operation device 42, an operation sensor 43, a display device D1, and a sound output device V1.

[0056] The remote controller 40 is an example of a control device CTR. In the illustrated example, the remote controller 40 is a computing device (electronic circuit) that enables the remote operator OP to remotely control the work machine 100.

[0057] Next, the respective functional blocks of the shovel controller 30 of the work machine 100 and the remote controller 40 of the remote control room RC will be described. <<Excavator function blocks>> Each functional block in the shovel controller 30 of the work machine 100 will be described. Each functional block in the shovel controller 30 is conceptual and does not necessarily have to be physically configured as shown in the drawing. Therefore, all or part of each functional block can be configured by functionally or physically distributing or integrating it in any unit. All or any part of the processing functions performed by each functional block are realized by a program executed by the shovel controller 30. Each functional block may also be realized as hardware using wired logic. The same applies to the remote controller 40 described below. In this embodiment, the shovel controller 30 executes a program to cause an shovel state identification unit 301, a transmission control unit 302, a reception control unit 303, and an actuator driving unit 304 to function.

[0058] The shovel state identifying unit 301 is configured to identify the state of the work machine 100. In this embodiment, the state of the work machine 100 includes the position and orientation of the work machine 100, and the states of the attachments of the work machine 100 (for example, the positions of the boom 4, arm 5, and bucket 6). The position of the work machine 100 is, for example, the position of the work machine 100 in a reference coordinate system (the latitude, longitude, and altitude of the reference point of the work machine 100). The shovel state identifying unit 301 identifies the position and orientation of the work machine 100 based on the output of the positioning device S8.

[0059] The state of the attachment (e.g., the positions of the boom 4, arm 5, and bucket 6) can be determined from the detection results of the angle sensors (boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3) and the respective sizes of the boom 4, arm 5, and bucket 6.

[0060] The transmission control unit 302 performs control to transmit various information to the remote control room RC via the communication device T1. For example, the transmission control unit 302 performs control to transmit to the remote control room RC image information captured by the imaging device S6, measurement information detected by the spatial recognition device S7, position information indicating the position and orientation of the work machine 100, and status information indicating the status of the attachment. Transmission by the transmission control unit 302 is performed at predetermined time intervals. The predetermined time may be any time, but in this embodiment, it is a time interval that allows recognition of changes in the situation due to work by the work machine 100. For example, the transmission control unit 302 may transmit the above information at one-second intervals.

[0061] The reception control unit 303 performs control to receive various information from the remote control room RC via the communication device T1. For example, the reception control unit 303 receives a control command for controlling the operation of the work machine 100 from the remote control room RC.

[0062] The actuator driving unit 304 is configured to drive the actuators AC mounted on the work machine 100. In this embodiment, the actuator driving unit 304 generates and outputs control commands for each of a plurality of solenoid valves including the proportional valve 31, based on control commands transmitted from the remote control room RC.

[0063] Each solenoid valve that receives the control command increases or decreases the pilot pressure acting on the pilot port of the corresponding control valve in the control valve unit 17. As a result, the hydraulic actuator corresponding to each control valve operates at a speed according to the stroke amount of the control valve. <<Remote Control Room Function Blocks>> The following describes each functional block in the remote controller 40 in the remote control room RC. In this embodiment, the remote controller 40 executes a program to cause a reception control unit 401, a signal generation unit 402, a transmission control unit 403, and a playback sound generation unit 404 to function.

[0064] The reception control unit 401 performs control for receiving various information from the work machine 100 via the communication device T2. For example, the reception control unit 401 receives image information, measurement information, position information, and status information from the work machine 100. The image information is, for example, information captured by the imaging device S6. The measurement information is, for example, information measured by the spatial recognition device S7. The position information is, for example, information indicating the position and orientation of the work machine 100 in a reference coordinate system measured by the positioning device S8. The status information is, for example, information indicating the status of the attachment (for example, the positions of the boom 4, arm 5, and bucket 6).

[0065] The signal generating unit 402 generates a control command for operating each component of the work machine 100 (for example, one or more of the boom 4, arm 5, bucket 6, upper rotating body 3, and lower traveling body 1) based on the operation content of the operating device 42 detected by the operation sensor 43.

[0066] The transmission control unit 403 performs control for transmitting the control command generated by the signal generation unit 402 to the work machine 100. This allows the communication device T2 to transmit a control command to the work machine 100 to operate the work machine 100 in response to operation of the operation device 42 by the remote operator OP. Therefore, the remote operator OP can remotely operate the work machine 100 from the remote control room RC.

[0067] The playback sound generation unit 404 is configured to generate playback sound data (playback sound data) corresponding to the original sound collected by the sound collection device M1. The playback sound generation unit 404 is also configured to generate the playback sound data so that the playback sound can be reproduced by the sound output device V1.

[0068] In the illustrated example, the playback sound generation unit 404 acquires data of the original sound (original sound data) collected by a sound collection device M1 provided on the work machine 100 via a communication device T1 provided on the work machine 100 and a communication device T2 provided in the remote control room RC. In principle, the playback sound generation unit 404 is configured to generate playback sound data so that the acquired original sound can be faithfully reproduced. Specifically, the playback sound generation unit 404 generates playback sound data so that the properties (volume, frequency, etc.) of the acquired original sound and the properties (volume, frequency, etc.) of the playback sound are the same.

[0069] The playback sound generation unit 404 may also be configured to adjust a sound (predetermined sound) generated by a predetermined sound source from among the acquired original sounds, and generate playback sound data so as to faithfully reproduce sounds other than the predetermined sound. Specifically, the playback sound generation unit 404 may identify the predetermined sound using registration information for the predetermined sound and adjust the predetermined sound by changing the properties (volume, frequency, etc.) of the predetermined sound. The predetermined sound source may be, for example, the engine 11, the main pump 14, the actuator AC, or a worker or another work machine working around the work machine 100. Note that the sound source may also be rain or wind. In other words, the predetermined sound may be engine sound, hydraulic operating sound, actuator sound, the voice of a worker, rain sound, wind sound, etc. The registration information for the predetermined sound is information for identifying the sound source, such as a frequency range or a waveform pattern. In the illustrated example, the registration information for the predetermined sound is a frequency range and is pre-stored in a non-volatile storage device or the like. However, the registration information of the predetermined sound may be a value that is calculated and registered while the sound is being collected by the sound collection device M1.

[0070] The frequency range is, for example, a range represented by a lower limit frequency and an upper limit frequency, and is registered for each sound source. FIG. 4 is a frequency spectrum diagram showing an example of the relationship between the volume and frequency of the original sound collected by the sound collection device M1. Note that the frequency spectrum diagram in FIG. 4 is a simplified fictitious diagram for ease of understanding. FIG. 4 shows that the frequency of engine sound, which is an example of a predetermined sound, falls within a first frequency range Z1, the frequency of hydraulic operation sound, which is another example of a predetermined sound, falls within a second frequency range Z2, and the frequency of actuator sound, which is yet another example of a predetermined sound, falls within a third frequency range Z3. A non-volatile storage device or the like stores, as a reference table, for example, the first frequency range Z1 for engine sound being a range from frequency f1 [Hz] to frequency f2 [Hz], the second frequency range Z2 for hydraulic operation sound being a range from frequency f3 [Hz] to frequency f4 [Hz], and the third frequency range Z3 for actuator sound being a range from frequency f5 [Hz] to frequency f6 [Hz]. The reference table may also store a frequency range relating to vehicle body vibration noise caused by vibration of the engine 11, which is yet another example of the predetermined noise.

[0071] The reproduced sound generation unit 404 may be configured to determine what kind of sound (predetermined sound) is included in the original sound by using the frequency components of the acquired original sound and a reference table. For example, the reproduced sound generation unit 404 may determine that a predetermined sound corresponding to a frequency range is occurring if the volume of the sound belonging to that frequency range is equal to or greater than a threshold value Vt. In this case, in the example shown in Fig. 4, the reproduced sound generation unit 404 can determine that the acquired original sound includes engine sound, hydraulic operating sound, and actuator sound.

[0072] The playback sound generation unit 404 may be configured to generate playback sound data according to the determination result as to what kind of sound (predetermined sound) is included in the acquired original sound. That is, the playback sound generation unit 404 may change the content of the generated playback sound data according to the determination result. For example, when it is determined that the original sound includes an engine sound and an actuator sound, the playback sound generation unit 404 may generate playback sound data such that the volume of the engine sound is reduced.

[0073] Alternatively, the playback sound generation unit 404 may be configured to generate playback sound data using registered information on a predetermined sound without determining what kind of sound (predetermined sound) is included in the acquired original sound. For example, the playback sound generation unit 404 may acquire a frequency range associated with a hydraulic operating sound using registered information on the hydraulic operating sound, regardless of whether the acquired original sound includes a hydraulic operating sound, and generate playback sound data such that the volume of sounds having frequencies included in that frequency range is reduced.

[0074] Next, an example of a process in which the control device CTR generates playback sound data (hereinafter referred to as "playback sound generation process") will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of the playback sound generation process. In the example shown in Fig. 5, the control device CTR is configured to execute the playback sound generation process at a predetermined timing.

[0075] First, the control device CTR determines whether or not a predetermined adjustment start condition is satisfied (step ST1). The predetermined adjustment start condition is, for example, that the engine 11 has started, that the gate lock lever has been operated and the work machine 100 is in an operable state, or that a person has been detected by the space recognition device S7.

[0076] If it is determined that the predetermined adjustment start condition is not satisfied (NO in step ST1), the control device CTR terminates the playback sound generation process without adjusting the predetermined sound. In this case, the control device CTR may generate playback sound data so as to faithfully reproduce the original sound collected by the sound collection device M1, and output the generated playback sound data to the sound output device V1. Alternatively, the control device CTR may be configured not to generate playback sound data, or may be configured not to output playback sound data to the sound output device V1.

[0077] When it is determined that the predetermined adjustment start condition is satisfied (YES in step ST1), the control device CTR starts generating playback sound data in which the predetermined sound is adjusted (step ST2). In the illustrated example, the control device CTR starts generating playback sound data that reduces the volume of the engine sound. This is to enable the operator to easily distinguish sounds other than the engine sound. In other words, this is to prevent the engine sound from drowning out sounds that the operator should hear. Specifically, the playback sound generation unit 404 of the remote controller 40 serving as the control device CTR references the registration information of the engine sound stored in the non-volatile storage device and starts generating playback sound data that reduces the volume of sounds having frequencies belonging to the first frequency range Z1. The portion of the frequency spectrum shown in FIG. 4 represented by the dotted line indicates the frequency spectrum of the engine sound (engine sound included in the original sound) before the volume is reduced, and the portion represented by the thick solid line in FIG. 4 indicates the frequency spectrum of the engine sound (engine sound included in the playback sound) after the volume is reduced. In this way, the playback sound generation unit 404 starts generating playback sound data that reduces the volume of sounds having frequencies that belong to the first frequency range Z1 to less than the threshold value Vt. In this case, the playback sound generation unit 404 may uniformly reduce the volume of all sounds having frequencies that belong to the first frequency range Z1 by a predetermined rate, or may reduce the volume of some sounds having frequencies that belong to the first frequency range Z1 by a different rate than the volume of another part of sounds having frequencies that belong to the first frequency range Z1.

[0078] Note that instead of suppressing the volume of sounds having frequencies within the first frequency range Z1, the reproduction sound generation unit 404 may shift the frequencies of the sounds having frequencies within the first frequency range Z1 to outside the first frequency range Z1. The portion represented by the dashed-dotted line in FIG. 4 indicates the frequency spectrum of the engine sound (engine sound included in the reproduction sound) shifted outside the first frequency range Z1. In this case, the reproduction sound generation unit 404 can make sounds (hydraulic operation sounds) having frequencies within a frequency range close to the first frequency range Z1 (e.g., a second frequency range) more audible. This is because by shifting the frequencies (pitch) of the sounds within the first frequency range Z1 away from the frequencies (pitch) of the sounds within the second frequency range Z2, the sounds within the second frequency range Z2 can be made more noticeable.

[0079] The playback sound generation unit 404 may be configured to generate playback sound data that reduces the volume of the engine sound only when the engine sound contained in the original sound can be identified. In this case, the playback sound generation unit 404 may determine that the original sound contains an engine sound when the volume of the sound belonging to a first frequency range Z1 related to the engine sound exceeds a threshold Vt, for example, and may start generating playback sound data such that the volume of the sound having a frequency belonging to the first frequency range Z1 is less than the threshold Vt. In other words, when the engine sound contained in the original sound cannot be identified, the playback sound generation unit 404 may generate playback sound data without suppressing the volume of the sound belonging to the first frequency range Z1.

[0080] Thereafter, the control device CTR starts outputting the generated playback sound data to the sound output device V1 (step ST3). In the illustrated example, the playback sound generation unit 404 outputs the generated playback sound data to the sound output device V1. Then, the sound output device V1 plays back the playback sound based on the playback sound data.

[0081] In this way, the playback sound generation unit 404 can generate playback sound data that suppresses the volume of predetermined sounds, such as engine sounds, that are included in the original sound. In other words, the sound output device V1 can output playback sound in which predetermined sounds, such as engine sounds, are suppressed. Therefore, the playback sound generation unit 404 can suppress sounds other than predetermined sounds, such as the voices of workers working around the work machine 100, from being drowned out by the predetermined sounds. Therefore, an operator listening to the playback sound reproduced by the sound output device V1 can easily hear sounds other than predetermined sounds, such as the voices of workers working around the work machine 100.

[0082] In the above example, the playback sound generation unit 404 reduces the engine sound included in the playback sound to make it easier for the operator to hear the worker's voice, but it may also be possible to make it easier for the operator to hear the worker's voice by reducing the sound generated by a sound generation source other than the engine 11 that is associated with a frequency range close to the frequency range to which human voices belong.

[0083] Alternatively, instead of reducing the engine sound contained in the playback sound, the playback sound generation unit 404 may increase the volume of the worker's voice contained in the playback sound, thereby making it easier for the operator to hear the worker's voice.

[0084] Alternatively, the playback sound generation unit 404 may increase the volume of the hydraulic operating sound so that the operator can hear the hydraulic operating sound more easily, thereby making it easier for the operator to understand the state of hydraulic equipment such as the main pump 14 based on the hydraulic operating sound.

[0085] Next, another example of the playback sound generation process will be described with reference to Fig. 6. Fig. 6 is a flowchart showing another example of the flow of the playback sound generation process. In the example shown in Fig. 6, the control device CTR repeatedly executes this playback sound generation process at every predetermined control period.

[0086] First, the control device CTR determines whether or not a predetermined operation is being performed (step ST11). The predetermined operation is, for example, a manual operation by a remote operator OP performed through an operation device 42 provided in the remote control room RC, and specifically, a swing operation, a boom operation, an arm operation, a bucket operation, a traveling operation, or the like. In the example shown in Fig. 6, the remote controller 40 serving as the control device CTR determines whether or not a predetermined operation is being performed based on the output of the operation sensor 43.

[0087] If the control device CTR determines that the specified operation has not been performed (NO in step ST11), it generates playback sound data in which the specified sound has not been adjusted (step ST12N), and if it determines that the specified operation has been performed (YES in step ST11), it generates playback sound data in which the specified sound has been adjusted (step ST12Y).

[0088] 6, the predetermined sound differs for each predetermined operation. For example, when the predetermined operation is a swing operation, the predetermined sound is a sound related to the swing operation, such as the contact sound between the swing gear and the swing ring gear, the sound generated by the swing hydraulic motor 2A, or the sound of hydraulic oil being released from the swing relief valve (swing relief valve release sound). When the predetermined operation is a boom operation, the predetermined sound is a sound related to the boom operation, such as the sound generated by the boom cylinder 7, the creaking sound of the boom 4, or the sliding sound of the boom foot pin.

[0089] For example, when it is determined that a turning operation is being performed, the control device CTR generates playback sound data that increases the volume of the sound related to the turning operation and keeps the volume of sounds other than the sound related to the turning operation constant. Alternatively, when it is determined that a turning operation is being performed, the control device CTR may generate playback sound data that increases the volume of the sound related to the turning operation and decreases the volume of the sound related to the boom operation and keeps the volume of the other sounds constant.

[0090] Thereafter, the control device CTR outputs the generated playback sound data to the sound output device V1 (step ST13).

[0091] By repeatedly executing this playback sound generation process, the control device CTR can, for example, differentiate between a sound (part of the playback sound) that is easy for the operator to hear when a swing operation is being performed and a sound (part of the playback sound) that is easy for the operator to hear when a boom operation is being performed (when a swing operation is not being performed).As a result, the operator can easily hear, for example, the release sound of the swing relief valve when a swing operation is being performed, and can easily hear the creaking sound of the boom 4 when a boom operation is being performed.

[0092] Next, another configuration example of the remote control system SYS will be described with reference to Fig. 7. Fig. 7 is a schematic diagram showing another configuration example of the remote control system SYS. The remote control system SYS shown in Fig. 7 differs from the remote control system SYS shown in Fig. 1 in that it includes a management device 200, a display device D1A, and a sound output device V1A, all of which are provided in a management center MC.

[0093] The management center MC is located away from both the work site of the work machine 100 and the remote control room RC. A management device 200, a display device D1A, and a sound output device V1A are installed in the management center MC.

[0094] The management device 200 is an example of a control device CTR, and is, for example, a server computer (a so-called cloud server) or an edge server. The management device 200 is typically a fixed terminal device, but may also be a portable terminal device (for example, a laptop computer, a tablet, or a smartphone).

[0095] Furthermore, the playback sound generation unit 404 of the control device CTR may be implemented in any of the shovel controller 30, the remote controller 40, and the management device 200. The playback sound generation unit 404 may be configured to generate playback sound data based on data of the original sound collected by the sound collection device M1 attached to the work machine 100 and registration information of a predetermined sound stored in a storage device in any of the shovel controller 30, the remote controller 40, and the management device 200, and to output the generated playback sound data to each of the sound output device V1 provided in the remote control room RC and the sound output device V1A provided in the management center MC.

[0096] With this configuration, the control device CTR can play the reproduced sound not only to the remote operator OP in the remote control room RC but also to the manager at the management center MC. In this case, the control device CTR may differentiate the reproduced sound data output to the sound output device V1 in the remote control room RC from the reproduced sound data output to the sound output device V1A in the management center MC. This is to accommodate cases where the sound that the remote operator OP wants to hear is different from the sound that the manager wants to hear. For example, the control device CTR may be configured to output reproduced sound data with emphasized engine sound to the sound output device V1A in the management center MC, and to output reproduced sound data with suppressed engine sound to the sound output device V1 in the remote control room RC.

[0097] As described above, the remote control system SYS according to an embodiment of the present disclosure includes a control device CTR and a sound output device V1, as shown in FIG. 1 . The control device CTR is configured to generate playback sound data based on data of original sound collected by a sound collection device M1 provided on the work machine 100 and registration information of sound generated by a predetermined sound source, and to output the generated playback sound from the sound output device V1. The control device CTR may be the shovel controller 30 provided in the cabin 10 of the work machine 100, the remote controller 40 provided in the remote control room RC, or the management device 200 provided in the management center MC. The various functions of the shovel controller 30, the remote controller 40, and the management device 200 may be realized by hardware, software, or a combination thereof distributed in the shovel controller 30, the remote controller 40, and the management device 200, respectively. The sound output device V1 may be provided in the remote control room RC or the management center MC. The sound output device V1 may also be a wearable device such as earphones or headphones worn by the operator. The control device CTR may also generate data of a reproduced sound corresponding to the original sound, in which only the volume of the sound generated by a predetermined sound generating source identified by the registration information is increased. This is to make the sound generated by the predetermined sound generating source easier to hear. The control device CTR may also generate data of a reproduced sound corresponding to the original sound, in which only the volume of the sound generated by the predetermined sound generating source identified by the registration information is decreased. This is to make sounds other than the sound generated by the predetermined sound generating source easier to hear. The control device CTR may also generate data of a reproduced sound corresponding to the original sound, in which only the frequency of the sound generated by the predetermined sound generating source identified by the registration information is shifted. This is to make the sound to be heard easier to hear by shifting the frequency of the sound generated by the predetermined sound generating source away from the frequency of the sound to be heard, which has a frequency close to the frequency of the sound generated by the predetermined sound generating source.

[0098] This configuration has the effect of more reliably communicating sounds that the operator wants to hear to the operator. Sound information is particularly important when remotely operating the work machine 100, but if the sound collected by the sound collection device M1 is simply reproduced faithfully by the sound output device V1, the operator of the work machine 100 may miss the sounds that he or she wants to hear. This configuration adjusts the properties of the sounds generated by a specified sound generation source, making it easier for the operator to hear the sounds that he or she wants to hear. Note that the properties of the sound include, for example, the amplitude (volume) or frequency (pitch) of the sound.

[0099] Furthermore, the control device CTR may be configured to generate data of the reproduced sound so that the sound generated by a predetermined sound generation source included in the reproduced sound differs from the sound generated by a predetermined sound generation source included in the original sound. For example, the control device CTR may be configured to generate data of the reproduced sound so that the amplitude (volume) or frequency (pitch) of the sound generated by the engine 11 included in the reproduced sound differs from the amplitude (volume) or frequency (pitch) of the sound generated by the engine 11 included in the original sound.

[0100] This configuration can convert sounds that are difficult for the operator to hear into sounds that are easier to hear, thereby bringing about the effect of improving the audibility of a predetermined sound in the reproduced sound, for example.

[0101] The control device CTR may be configured to generate playback sound data based on the original sound data and registration information of the sound generated by the predetermined sound source when a predetermined condition is satisfied, and to generate playback sound data based on the original sound data without using the registration information of the sound generated by the predetermined sound source when the predetermined condition is not satisfied. The predetermined condition may be, for example, that the engine 11 has started, that the gate lock lever has been operated and the work machine 100 is in an operable state, or that a person has been detected by the spatial recognition device S7.

[0102] This configuration makes it possible to change how a predetermined sound in the reproduced sound sounds when a predetermined condition is satisfied and when the predetermined condition is not satisfied, thereby bringing about the effect of making the predetermined sound that is heard when the predetermined condition is satisfied stand out.

[0103] 1, the remote control system SYS may also be equipped with a display device D1 that is visually recognized by a user (remote operator OP) of the sound output device V1. In this case, an imaging device S6 may be attached to the work machine 100. The control device CTR (remote controller 40) may be configured to output a reproduced sound from the sound output device V1 while displaying an image captured by the imaging device S6 on the display device D1.

[0104] This configuration allows the playback sound to be played in synchronization with the image displayed on the display device D1, thereby having the effect of more reliably transmitting the sounds that the remote operator OP remotely operating the work machine 100 in the remote control room RC wants to hear to the remote operator OP.

[0105] The control device CTR may be configured to be able to switch between a first mode in which data of a reproduced sound is generated based on data of an original sound without using registration information of a sound generated by a predetermined sound generation source, and a second mode in which data of a reproduced sound is generated based on data of the original sound and registration information of a sound generated by a predetermined sound generation source. For example, the control device CTR may be configured so that the operation mode of the sound output device V1 becomes the first mode when an operating tool such as a switch provided in the remote control room RC is turned on, and so that the operation mode of the sound output device V1 becomes the second mode when the operating tool is turned off.

[0106] This configuration has the effect of enabling the operator to switch the operation mode of the control device CTR between the first mode and the second mode at any timing. In other words, this configuration has the effect of more reliably transmitting to the operator the sound that the operator wants to hear at the timing that the operator wants.

[0107] The control device CTR may also be configured to identify the position of the sound source based on the data of the original sound, and to generate playback sound data so that the sound generated by a sound source in a predetermined direction when viewed from the center of the work machine 100 becomes louder or quieter. The center of the work machine 100 is, for example, a point on the rotation axis of the work machine 100. For example, the control device CTR may identify the position of the sound source based on the data of the original sound acquired by the sound collection device M1.

[0108] In this case, the sound collection device M1 may be configured with a combination of multiple microphones installed at different positions. The control device CTR may then generate playback sound data, for example, so that the volume of the sound (voice) generated by a sound generation source (worker) located in front of the swing shaft of the work machine 100 does not decrease, and so that the volume of the sound generated by a sound generation source (engine 11) located behind the swing shaft of the work machine 100 is reduced. Alternatively, the control device CTR may generate playback sound data so that the volume of the sound generated by a sound generation source (another work machine) located to the right of the work machine 100 is reduced, and so that the volume of the sound generated by a sound generation source (worker) located behind the work machine 100 is not reduced. This is because reducing sounds other than the worker's voice, such as engine noise, makes it easier for the operator of the work machine 100 to hear the worker's voice. Conversely, the control device CTR may generate sound playback data so that the volume of sound generated by a sound generating source (worker) in front of the work machine 100 is increased, and so that the volume of sound generated by a sound generating source (engine 11) behind the swing shaft of the work machine 100 is not increased. This is to make it easier for the operator of the work machine 100 to hear the voice of the worker by increasing only the voice of the worker without increasing sounds other than the worker's voice, such as engine noise. Alternatively, the control device CTR may generate sound playback data so that the volume of sound generated by a sound generating source (worker) in front of the work machine 100 is increased, and so that the volume of sound generated by a sound generating source (another worker) behind the swing shaft of the work machine 100 is not increased. This is to make it easier for the operator of the work machine 100 to hear the voice of a worker in a specific direction as seen from the work machine 100, as distinguished from the voice of a worker in a different direction as seen from the work machine 100.

[0109] The control device CTR may also be configured to generate playback sound data by replacing sounds included in the original sound and generated by a predetermined sound source with pre-created sounds. The pre-created sounds may be electronic sounds such as beeps or computer-generated sounds. For example, the control device CTR may generate playback sound data such that only the engine sound included in the original sound becomes a pre-created electronic sound, while the other sounds included in the original sound are faithfully reproduced.

[0110] This configuration has the effect of more reliably conveying to the operator the sounds that the operator wants to hear, because sounds that are easy for the operator to hear can be created in advance and then included in the reproduced sounds.

[0111] Furthermore, the control device CTR may be configured to generate playback sound data based on the original sound data and registration information on sounds generated by a predetermined sound generation source when a predetermined operation is being performed on the work machine 100, and to generate playback sound data based on the original sound data without using the registration information on sounds generated by the predetermined sound generation source when a predetermined operation is not being performed on the work machine 100. For example, when a swing operation is being performed, the control device CTR may identify sounds of the original sounds other than those generated by the swing mechanism 2 based on the original sound data and registration information (information on the frequency range) on sounds generated by the swing mechanism 2, and generate playback sounds so as to reduce the volume of sounds other than those generated by the swing mechanism 2. Alternatively, when a swing operation is being performed, the control device CTR may identify sounds generated by the swing mechanism 2 based on the original sound data and registration information (information on the frequency range) on sounds generated by the swing mechanism 2, and generate playback sounds so as to increase the volume of sounds generated by the swing mechanism 2. The sound generated by the swing mechanism 2 is, for example, the sound generated by the swing hydraulic motor 2A, or the contact sound between the swing gear and the swing ring gear, etc. Alternatively, when a boom-raising operation is being performed, the control device CTR may identify the sound generated by the boom 4 and the boom cylinder 7 among the original sounds based on the data of the original sounds and the registration information (information on the frequency range) of the sounds generated by the boom 4 and the boom cylinder 7, and may generate a reproduced sound so that the volume of the sound generated by the boom 4 and the boom cylinder 7 is increased, or may generate a reproduced sound so that the volume of sounds other than the sounds generated by the boom 4 and the boom cylinder 7 is decreased.

[0112] This configuration has the effect that, when an operation is being performed on a specific actuator AC, the operator (particularly the remote operator OP) can easily distinguish the sound related to that specific actuator AC. Therefore, the remote operator OP can infer the subsequent state of the actuator AC (e.g., the operating speed of the actuator AC or whether or not the actuator AC has started to move, etc.) and ultimately the subsequent state of the work machine 100 from the sound generated when the actuator AC starts to move (e.g., the creaking sound of the actuator AC or the sound of parts coming into contact with each other), the sound generated when the actuator AC is moving (e.g., the sound of hydraulic oil being released from the relief valve), or the sound generated when the actuator AC stops. For example, by listening to the sound related to the bucket cylinder 9, the remote operator OP can determine whether the bucket 6 has actually started to move (especially when trying to move the bucket 6 slightly at a slow speed), or can predict the state of the bucket cylinder 9 immediately before it extends to the stroke end. Alternatively, the remote operator OP can estimate the direction of the upper rotating body 3 after the rotation has stopped by listening to the sound related to the rotation mechanism 2 that is generated when the rotation operation lever is returned to the neutral position. That is, the remote operator OP can estimate how much the upper rotating body 3 will rotate due to inertial force from the time the rotation operation lever is returned to the neutral position before stopping. Furthermore, this configuration allows the remote operator OP to hear the sound that is generated when an operation is being performed on a specific actuator AC, thereby enabling the remote operator OP to understand the load on the actuator AC and preventing the remote operator OP from operating the lever in an unreasonable manner.

[0113] 7, the remote control system SYS may also include a sound output device V1A separate from the sound output device V1 used by the operator (remote operator OP) of the work machine 100. The separate sound output device V1A may be, for example, a sound output device installed in the management center MC.

[0114] With this configuration, the manager of the work machine 100 at the management center MC can listen in the desired manner to the sound collected by the sound collection device M1 provided on the work machine 100. Therefore, the manager of the work machine 100 can confirm, for example, whether the engine 11 of the work machine 100 is operating normally or whether the main pump 14 of the work machine 100 is operating normally.

[0115] In addition, the control device CTR may be configured to identify a first portion of the original sound data that corresponds to the sound generated by a specified sound source based on the data of the original sound collected by the sound collection device M1 and the registration information of the sound generated by the specified sound source, and to generate playback sound data based on data of another sound that corresponds to the first portion and the remaining portion of the original sound data.

[0116] This configuration has the effect of more reliably conveying to the operator the sounds that the operator wants to hear, because it is possible to selectively adjust only the sounds that are generated by a predetermined sound source that is included in the original sound.

[0117] Furthermore, the registration information of the sound generated by the predetermined sound source may be the frequency range of the sound generated by the predetermined sound source.

[0118] This configuration has the effect of more reliably conveying to the operator the sounds that the operator wants to hear, because by simply setting the frequency range for each predetermined sound, it is possible to selectively adjust only the sounds generated by predetermined sound sources contained in the original sound.

[0119] The registration information may also be dynamically set information. In this case, the registration information may be stored so that it differs depending on whether the work site is in an urban area or a mountainous region, the weather (humidity, temperature, etc.) of the work site, the altitude of the work site, etc. Alternatively, the registration information may be calculated based on the properties of the original sound collected by the sound collection device M1. For example, the control device CTR may be configured to calculate the lower limit frequency and the upper limit frequency of the first frequency range Z1 related to the engine sound based on data of the original sound collected for a predetermined time after the work machine 100 is started.

[0120] This configuration brings about the effect that the sounds that the operator wants to hear can be more reliably conveyed to the operator, because the registration information for each predetermined sound can be flexibly set.

[0121] The preferred embodiments of the present disclosure have been described above. However, the invention according to the present disclosure is not limited to the above-described embodiments. Various modifications, substitutions, etc. may be applied to the above-described embodiments without departing from the scope of the invention according to the present disclosure. Furthermore, each of the features described with reference to the above-described embodiments may be combined as appropriate unless technically inconsistent. [Explanation of symbols]

[0122] 1···Undercarriage 1L··Left travel hydraulic motor 1R···Right travel hydraulic motor 2···Slewing mechanism 2A···Slewing hydraulic motor 3···Upper rotating body 4···Boom 5···Arm 6···Bucket 7···Boom cylinder 8···Arm cylinder 9···Bucket cylinder 10···Cabin 11···Engine 13···Regulator 14···Main pump 15···Pilot pump 17···Control valve unit 26···Operation device 28···Discharge pressure sensor 29···Operation sensor 30···Excavator controller 31···Proportional valve 40··Remote controller 42···Operation device 43···Operation sensor 100···Work machine 171-175···Control valve 200···Management device 301... Excavator status identification unit 302... Transmission control unit 303... Reception control unit 304... Actuator drive unit 401... Reception control unit 402... Signal generation unit 403... Transmission control unit 404... Playback sound generation unit AC... Actuator CTR... Control unit D1... Display unit D1A... Display unit DS... Operator's seat M1... Sound collection device M1E... Outdoor sound collection device M1N... Indoor sound collection device MC... Management center NW... Communications network OP... Remote operator RC... Remote operation room S1... Boom angle sensor S2... Arm angle sensor S3... Bucket angle sensor S4... Machine body tilt sensor S5... Swing angular velocity sensor S6... Imaging device S6B... Rear camera S6F... Front camera S6L... Left camera S6R···Right camera S7···Spatial recognition device S7B···Rear spatial recognition device S7F···Front spatial recognition device S7L···Left spatial recognition device S7R···Right spatial recognition device S8···Positioning device SYS···Remote control system T1···Communication device T2···Communication device V1···Sound output device V1A···Sound output device Z1···First frequency range Z2···Second frequency range Z3···Third frequency range

Claims

1. A remote control system for a work machine including a lower traveling body, an upper rotating body rotatably mounted on the lower traveling body, and a sound collecting device, a control device; a sound output device; the control device generates data of a reproduced sound based on data of an original sound collected by the sound collection device and registration information of a sound generated by a predetermined sound generation source, and outputs the reproduced sound from the sound output device; A remote control system for a work machine.

2. the control device generates data of the reproduced sound such that a sound generated by the predetermined sound generation source and included in the reproduced sound is different from a sound generated by the predetermined sound generation source and included in the original sound.

2. The remote control system for a work machine according to claim 1.

3. the control device generates the data of the reproduced sound based on the data of the original sound and the registration information when a predetermined condition is satisfied, and generates the data of the reproduced sound based on the data of the original sound without using the registration information when the predetermined condition is not satisfied.

2. The remote control system for a work machine according to claim 1.

4. a display device that is visually recognized by a user of the sound output device; an imaging device is attached to the work machine; the control device causes the display device to display the image captured by the imaging device, while causing the sound output device to output the reproduced sound.

2. The remote control system for a work machine according to claim 1.

5. The control device is configured to be able to switch between a first mode in which the reproduction sound data is generated based on the data of the original sound without using the registration information, and a second mode in which the reproduction sound data is generated based on the data of the original sound and the registration information.

2. The remote control system for a work machine according to claim 1.

6. the control device identifies the position of a sound source based on the data of the original sound, and generates the data of the reproduced sound so as to increase or decrease the volume of a sound generated by a sound source that is in a predetermined direction as viewed from the center of the work machine.

2. The remote control system for a work machine according to claim 1.

7. the control device generates the data of the reproduced sound by replacing a sound generated by the predetermined sound generation source included in the original sound with a sound created in advance.

2. The remote control system for a work machine according to claim 1.

8. the control device generates the reproduced sound data based on the original sound data and the registration information when a predetermined operation is being performed on the work machine, and generates the reproduced sound data based on the original sound data without using the registration information when the predetermined operation is not being performed on the work machine.

2. The remote control system for a work machine according to claim 1.

9. a sound output device separate from the sound output device used by the operator of the work machine; 2. The remote control system for a work machine according to claim 1.

10. the control device, based on data of an original sound collected by the sound collection device and registration information of a sound generated by a predetermined sound generation source, identifies a first portion of the data of the original sound corresponding to the sound generated by the predetermined sound generation source, and generates data of the reproduced sound based on data of another sound corresponding to the first portion and the remaining portion of the data of the original sound; 2. The remote control system for a work machine according to claim 1.

11. The registration information is a frequency range of the sound generated by the predetermined sound source.

2. The remote control system for a work machine according to claim 1.

12. The registration information is dynamically set information.

2. The remote control system for a work machine according to claim 1.

Images