Display device for work machine, work machine and remote control system for work machine

Abstract

In a display device of a working machine, the display device is configured to capture image information taken by an imaging device provided on the working machine, and to capture information about a sound signal captured by each of several sound collectors provided on the working machine, and the display device includes a control that is configured to display on a display section a noise generation situation around the working machine based on the information about the sound signal captured by each of the several sound collectors, superimposed on the image information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a display device of a work machine, a work machine, and a remote control system for the work machine. STATE OF THE ART

[0002] Conventionally, it was necessary to provide an operator operating a work machine with appropriate information about the surrounding environment in which the work machine operates. For example, a technology has been proposed that calculates a hazard level for each position outside a machine, extracts sounds at the position with a high hazard level, and outputs the sound (see Patent Document 1). This allows the operator to react immediately to avoid dangerous situations. RELATED PRIOR ART PATENT DOCUMENT

[0003] Patent Document 1: Japanese Unexamined Patent Publication No. 2012-58314 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] However, the system described in Patent Document 1 does not consider the display of the noise source. Therefore, when displaying image information representing the surroundings of a work machine, it is believed that safety can be further improved if the position of the noise source in the surroundings of the work machine can be visually recognized.

[0005] Considering the above problem, safety is improved by superimposing a noise situation around a working machine with the image information captured by an imaging device. MEANS TO SOLVE THE PROBLEMS

[0006] According to one aspect of the present disclosure, a display device of a work machine is provided. The display device is configured to acquire image information acquired by an imaging device provided on the work machine and information about a noise signal acquired by each of a plurality of noise collectors provided on the work machine. The display device includes a controller configured to display, on a display section, a noise generation situation around the work machine based on the information about the noise signal acquired by each of the plurality of noise collectors superimposed on the image information. EFFECTS OF THE INVENTION

[0007] According to one aspect of the present disclosure, a noise generation situation around a work machine can be detected, thereby improving safety. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a schematic diagram showing an example of a remote control system according to a first embodiment. Fig. 2 is a side view showing an excavator (an excavating machine) according to the first embodiment. Fig. 3 is a diagram showing an example of an arrangement of microphones in a microphone array provided on the excavator according to the first embodiment. Fig. 4 is a diagram showing a configuration example of a drive control system of the excavator according to the first embodiment. Fig. 5 is a diagram schematically showing an example of a configuration of a hydraulic system of the excavator according to the first embodiment. Fig. 6 is a functional block diagram showing a configuration example of a remote control system according to the first embodiment. Fig. 7 is a diagram showing an arrangement example of a remote control room RC according to the first embodiment. Fig. 8 is a conceptual diagram showing a situation around the excavator according to the first embodiment. Fig. 9 is a conceptual diagram showing generation of a noise heat map by a noise signal of the microphone array in a map generator according to the first embodiment. Fig. 10 is a diagram showing an example of a display screen displayed on a display device according to the first embodiment. Fig. 11 is a diagram showing an example of a display screen displayed on the display device according to the first embodiment. Fig. 12 is a diagram showing an example of a display screen displayed on a display device of a remote control room according to a variant. Fig. 13 is a conceptual diagram showing a situation around an excavator according to a second embodiment. Fig. 14 is a conceptual diagram showing a configuration for outputting sound in a remote control room in the second embodiment. Fig. 15 is a side view of an excavator as a working machine according to a third embodiment. Fig. 16 is a side view of an excavator as a working machine according to a fourth embodiment. Fig. 17 is a diagram showing a configuration example of an image display section and an operation section of a display device of the excavator according to the fourth embodiment. EMBODIMENT OF THE INVENTION

[0008] Embodiments of the present disclosure will be described below with reference to the drawings. The embodiments described below do not limit the invention, but are illustrative examples, and not all features or combinations thereof described in the embodiments are necessarily essential to the invention. In each drawing, the same or corresponding configurations may be denoted by the same or corresponding reference numerals, and explanations thereof may be omitted. [First embodiment]

[0009] First, an overview of a remote control system SYS according to a first embodiment will be given with reference to Fig. 1 described. Fig. 1 is a schematic diagram showing an example of the remote control system SYS according to the first embodiment. <Vorrichtungen, die das Fernbedienungssystem bilden>

[0010] As in Fig. 1, the remote control system SYS according to the first embodiment includes an excavator 100 and a remote control room RC.

[0011] The excavator 100 and the remote control room RC are connected to enable data transmission and reception via a communication line NW.

[0012] The excavator 100 is capable of wireless communication. The excavator 100 can send and receive data to and from a device connected to the communication line NW (e.g., the remote control room RC).

[0013] The excavator 100 transmits information about a construction site to the remote control room RC. This allows the remote control room RC to inspect the construction site in response to the information from the excavator 100. In the present embodiment, the device for surveying the construction site is not limited to the excavator 100, but may be any other type of device, such as a drone that flies over the construction site, a fixed-point camera, or an imaging device that can be worn by a user.

[0014] For example, the excavator 100 is equipped with an imaging device S6. The excavator 100 transmits imaging information indicating imaging results of the construction site to the remote control room RC through the imaging device S6.

[0015] The number of excavators 100 included in the remote control system SYS can be one or more. The remote control system SYS can provide information about the construction site to the remote control room RC via the multiple excavators 100. < Remote control room configuration example>

[0016] The remote control room RC includes a communication device T2, a remote control 40, an operating device 42, an operating sensor 43, a display device D1, and a loudspeaker A2. The remote control room RC includes an operating seat DS on which an operator OP sits, who remotely operates the excavator 100.

[0017] The communication device T2 is configured to communicate with a communication device T1 mounted on the excavator 100 (see Fig. 2) controls.

[0018] The remote controller 40 is an information processing device that performs various calculations. In the present embodiment, the remote controller 40 is constructed of a microcomputer comprising a CPU and a memory. Various functions of the remote controller 40 are implemented by the CPU, which executes a program stored in the memory.

[0019] The display device D1 displays a screen based on the information transmitted from the excavator 100, allowing the operator OP in the remote control room RC to view the surroundings of the excavator 100. According to the display device D1, the operator can check the situation at the construction site, including the surroundings of the excavator 100, while in the remote control room RC.

[0020] The operating device 42 is equipped with an operation sensor 43 for detecting details of the operation of the operating device 42. The operation sensor 43 is, for example, an inclination sensor for detecting an inclination angle of an operating lever, an angle sensor for detecting a pivot angle about the pivot axis of the operating lever, or the like. The operation sensor 43 may also include other sensors, such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor 43 outputs information about the detected details of the operations of the operating device 42 to the remote controller 40. The remote controller 40 generates an operation signal based on the received information and transmits the generated operation signal to the excavator 100. The operation sensor 43 may be configured to generate the operation signal.In this case, the operation sensor 43 can output the operation signal to the communication device T2 without passing through the remote controller 40. This allows the excavator 100 to be remotely controlled from the remote control room RC.

[0021] The loudspeaker A2 outputs the sound information transmitted from the excavator 100 so that the operator OP in the remote control room RC can recognize the sound generated around the excavator 100. <Konfigurationsbeispiel des Baggers>

[0022] Next, an overview of the excavator 100 according to the present embodiment will be described with reference to Fig. 2 described. Fig. 2 is a side view of the excavator 100 as a work machine according to the first embodiment. An upper rotating body 3 is rotatably mounted on a lower traveling body 1 of the excavator 100 via a rotating mechanism 2. A boom 4 is attached to the upper rotating body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6 is attached to the tip of the arm 5 as an end attachment. The end attachment may be a slope bucket, a backhoe bucket, or the like.

[0023] In the Fig. In the example shown in Figure 2, the movement direction (forward and backward direction) of the excavator 100 is indicated by the X-axis, the width direction of the excavator 100 is indicated by the Y-axis, and the height direction of the excavator 100 is indicated by the Z-axis.

[0024] The boom 4, the arm 5, and the bucket 6 constitute an excavator attachment, which is an example of an attachment, and are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively. A boom angle sensor S1 is attached to the boom 4, an arm angle sensor S2 is attached to the arm 5, and a bucket angle sensor S3 is attached to the bucket 6. The excavation attachment may be provided with a bucket tilting mechanism.

[0025] The boom angle sensor S1 detects the rotation angle of the boom 4. In the present embodiment, the boom angle sensor S1 is an acceleration sensor and can detect the boom angle, which is the rotation angle of the boom 4 with respect to the upper rotating body 3. For example, the boom angle becomes the minimum angle when the boom 4 is lowered the most and increases as the boom 4 is raised.

[0026] The arm angle sensor S2 detects the rotation angle of the arm 5. In the present embodiment, the arm angle sensor S2 is an acceleration sensor and can detect the arm angle, which is the rotation angle of the arm 5 with respect to the boom 4. For example, the arm angle becomes the minimum angle when the arm 5 is most closed and increases as the arm 5 is opened.

[0027] The bucket angle sensor S3 detects the rotation angle of the bucket 6. In the present embodiment, the bucket angle sensor S3 is an acceleration sensor and can detect the bucket angle, which is the rotation angle of the bucket 6 with respect to the arm 5. For example, the bucket angle becomes the minimum angle when the bucket 6 is most closed and increases as the bucket 6 is opened.

[0028] The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 can each be a variable resistance potentiometer, a stroke sensor that detects the stroke amount of the corresponding hydraulic cylinder, a rotary encoder that detects the rotation angle around the connecting pin, or the like. The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 form a position sensor that detects the position of the excavation attachment.

[0029] The upper rotating body 3 is equipped with a cab 10 as a driver's cab, an engine 11, a body inclination sensor S4, a rotation angular velocity sensor S5, an imaging device S6, a positioning device S7, a microphone array A1, a communication device T1, and the like.

[0030] An excavator control system 30 is installed in the cabin 10. A driver's seat, an operating device, and the like are installed in the cabin 10.

[0031] The excavator controller 30 is a computing device that performs various calculations. The excavator controller 30 is provided, for example, in the cab 10 and controls the drive of the excavator 100. The function of the excavator controller 30 can be realized by any hardware, software, or a combination thereof. For example, the excavator controller 30 is mainly composed of a microcomputer including a storage device such as a central processing unit (CPU) and a random access memory (RAM); a non-volatile auxiliary storage device such as a read-only memory (ROM); various input / output interface devices; and the like. The excavator controller 30 performs various functions, for example, by executing various programs installed in the non-volatile auxiliary storage device on the CPU.

[0032] The engine 11 is a power source of the excavator 100. In the present embodiment, the engine 11 is a diesel engine. The output shaft of the engine 11 is connected to the respective input shafts of a main pump 14 and a pilot pump 15.

[0033] The body tilt sensor S4 is configured to detect the tilt of the upper revolving body 3 with respect to a predetermined plane. In the present embodiment, the body tilt sensor S4 is an acceleration sensor that detects the tilt angle about the front-rear axis and the tilt angle about the left-right axis of the upper revolving body 3 with respect to the horizontal plane. For example, the front-rear axis and the left-right axis of the upper revolving body 3 pass orthogonally to each other through the center of the excavator, which is a point on the rotation axis of the excavator 100.

[0034] The rotation angle sensor S5 is configured to detect the rotation angular velocity of the upper rotating body 3. In the present embodiment, the rotation angular velocity sensor S5 is a gyroscopic sensor. The rotation angular velocity sensor S5 may be a resolver, a rotary encoder, or the like. The rotation angular velocity sensor S5 can detect the rotational velocity. The rotational velocity can be calculated from the rotational angular velocity.

[0035] The imaging device S6 is configured to capture an image around the excavator 100. In the present embodiment, the imaging device S6 includes a front camera S6F for imaging the space in front of the excavator 100, a left camera S6L for imaging the space to the left of the excavator 100, a right camera S6R for imaging the space to the right of the excavator 100, and a rear camera S6B for imaging the space behind the excavator 100.

[0036] The imaging device S6 is, for example, a monocular camera with an imaging element such as a CCD or CMOS and can output the captured image to a display device D1.

[0037] For example, the front camera S6F is mounted on the roof of the cabin 10. The left camera S6L is mounted on the left end of the upper surface of the upper rotating body 3. The right camera S6R is mounted on the right end of the upper surface of the upper rotating body 3. The rear view camera S6B is mounted on the rear end of the upper surface of the upper rotating body 3.

[0038] In the present embodiment, by providing the imaging device S6 with the above-described arrangement, an object present around the excavator 100 can be imaged. A camera (for example, RGBD cameras or stereo cameras) capable of detecting the distance to the object to be imaged can be used as the imaging device S6.

[0039] The positioning device S7 is configured to acquire information about the position of the excavator 100. In the present embodiment, the positioning device S7 is configured to measure the position and orientation of the excavator 100 in the reference coordinate system. Specifically, the positioning device S7 is a GNSS receiver that includes an electronic compass and measures the latitude, longitude, and altitude of the current position of the excavator 100, as well as the orientation of the excavator 100. The reference coordinate system according to the present embodiment is, for example, a geodetic world system. The geodetic world system is a three-dimensional orthogonal XYZ coordinate system in which the origin is located at the Earth's center of gravity, the X-axis extends toward the intersection of the Greenwich meridian and the equator, and the Y-axis extends toward the 90th degree.east longitude and the Z-axis runs towards the North Pole.

[0040] The communication device T1 is configured to control communication with devices external to the excavator 100. In the present embodiment, the communication device T1 is configured to control communication between the communication device T1 and devices external to the excavator 100 via a wireless communication network. The communication device T1 includes, for example, a mobile communication module conforming to a mobile communication standard such as Long Term Evolution (LTE), 4th Generation (4G), and 5th Generation (5G), a satellite communication module for connecting to a satellite communication network, or the like.

[0041] The communication device T1 controls, for example, wireless communication between the external GNSS (GNSS; Global Navigation Satellite System) positioning system and the excavator 100.

[0042] The microphone array A1 includes a plurality of microphones and is configured to collect sounds generated around the excavator 100. In the present embodiment, the microphone array A1 consists of a plurality of microphones mounted on the upper rotary body 3.

[0043] Fig. 3 is a diagram showing an arrangement example of microphones in the microphone array A1 according to the present embodiment. As shown in Fig. As shown in Figure 3, the microphone array A1 is mounted at the front of the booth 10. Specifically, a total of 15 microphones forming the microphone array A1 are arranged in a 3x5 array, with three microphones aligned along the Z axis and five microphones aligned along the Y axis. The distance between the microphones forming the microphone array A1 is a pitch of 1. The distance between the microphones is shown as an example and is determined depending on the implementation.

[0044] Each of the microphones constituting the microphone array A1 is capable of collecting, for example, sounds in a predetermined direction within a range of -90 degrees to +90 degrees centered on the front. The range in which the microphone array A1 can collect sounds is determined so that it is capable of detecting at least the sounds detected within the imaging range of the front camera S6F. This allows the sound emitted in the imaging range to be displayed in a recognizable manner when displaying the image information captured by the front camera S6F. The manner of displaying the image information will be described later. In the present embodiment, the sound collection range of the microphone array A1 is illustrated as an example, and it is sufficient that it includes at least the imaging range of the front camera S6F.

[0045] In the present embodiment, an example was prepared using the Fig. 3 as an example of multiple noise collectors. The present embodiment describes an example of multiple noise collectors and is not limited to the embodiment shown in Fig. 3. In particular, the number of microphones is not limited to the number shown in Fig. 3. For example, the number of microphones in the width direction (Y-axis direction) may be other than five, and the number of microphones in the height direction (Z-axis direction) may be other than three. Furthermore, the number of microphones is not limited to a two-dimensional arrangement in the Y-axis and Z-axis directions, and may, for example, be a one-dimensional arrangement in the Y-axis direction. [Excavator drive control system]

[0046] Fig. 4 is a diagram showing a configuration example of a drive control system of the Fig. 2 shows the excavator 100. In Fig. 4, a mechanical power transmission system is shown by a double line, hydraulic fluid lines are shown by thick solid lines, pilot lines are shown by dashed lines, and electrical drive and control system is shown by dashed lines.

[0047] The drive system of the excavator 100 according to the present embodiment includes the engine 11, a governor 13, the main pump 14, and a control valve unit 17. As described above, the hydraulic drive system of the excavator 100 according to the present embodiment includes hydraulic actuators such as the travel hydraulic motors 1L and 1R, a rotary hydraulic motor 2A, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9, for hydraulically driving the lower traveling body 1, the upper rotary body 3, the boom 4, the arm 5, and the bucket 6, respectively.

[0048] The engine 11 is the main power source of the hydraulic drive system and is mounted, for example, on the rear of the upper rotary body 3. Specifically, the engine 11 rotates at a predetermined target speed under direct or indirect control by the excavator controller 30 described later to drive the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses light oil as fuel.

[0049] The controller 13 controls the discharge volume of the main pump 14. For example, the controller 13 adjusts the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the excavator controller 30. The controller 13 includes, for example, the controllers 13L and 13R described later.

[0050] The main pump 14, like the engine 11, is mounted, for example, on the rear of the upper rotary body 3 and supplies hydraulic fluid to the control valve unit 17 via a high-pressure hydraulic line. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable-displacement hydraulic pump. As described above, the stroke length of the piston is adjusted by adjusting the tilt angle of the swash plate through the regulator 13 under the control of the excavator controller 30, and the discharge rate (discharge pressure) is controlled. The main pump 14 includes, for example, main pumps 14L and 14R, as described below.

[0051] The control valve unit 17 is a hydraulic control device that controls the hydraulic system in the excavator 100. In the present embodiment, the control valve unit 17 includes control valves 171 to 176. The control valve 175 includes a control valve 175L and a control valve 175R, and the control valve 176 includes a control valve 176L and a control valve 176R. The control valve unit 17 is configured to selectively supply hydraulic fluid discharged from the main pump 14 to one or more hydraulic actuators via the control valves 171 to 176. The control valves 171 to 176 control, for example, the flow rate of hydraulic fluid flowing from the main pump 14 to the hydraulic actuator and the flow rate of hydraulic fluid flowing from the hydraulic actuator to the hydraulic fluid tank.The hydraulic actuator includes the boom cylinder 7, the arm cylinder 8, the bucket cylinder 9, the travel hydraulic motors 1L and 1R, and the rotary hydraulic motor 2A. More specifically, the control valve 171 corresponds to the left travel hydraulic motor 1L, the control valve 172 corresponds to the right travel hydraulic motor 1R, and the control valve 173 corresponds to the rotary hydraulic motor 2A. The control valve 174 corresponds to the bucket cylinder 9, the control valve 175 corresponds to the boom cylinder 7, and the control valve 176 corresponds to the arm cylinder 8. The control valve 175 includes, for example, the control valves 175L and 175R described later, and the control valve 176 includes, for example, the control valves 176L and 176R described later. Details of the control valves 171 to 176 will be described later.

[0052] The pilot pump 15 is an example of a pilot pressure generating device and is configured to supply hydraulic fluid to hydraulic control devices via the pilot lines. In the present embodiment, the pilot pump 15 is a fixed-capacity hydraulic pump. The pilot pressure generating device may be implemented by the main pump 14. That is, the main pump 14 may have both a function of supplying hydraulic fluid to the control valve unit 17 via the hydraulic fluid lines and a function of supplying hydraulic fluid to various hydraulic control devices via the pilot lines. In this case, the pilot pump 15 may be omitted.

[0053] The operating device 26 is a device used by an operator to operate an actuator. The actuator includes at least one hydraulic actuator or one electric actuator.

[0054] A discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In the present embodiment, the discharge pressure sensor 28 outputs the detected value to the excavator controller 30. The discharge pressure sensor 28 includes, for example, discharge pressure sensors 28L and 28R, as described later.

[0055] An operation sensor 29 is configured to detect details of operations performed by the operator using the operation device 26. In the present embodiment, the operation sensor 29 detects the operation direction and operation amount of the operation device 26 corresponding to each of the actuators and outputs the detected values ​​to the excavator controller 30. In the present embodiment, the excavator controller 30 controls the opening area of ​​a proportional valve 31 depending on the output of the operation sensor 29. The excavator controller 30 then supplies the hydraulic fluid discharged from the pilot pump 15 to a pilot port of the corresponding control valve in the control valve unit 17.The pressure (control pressure) of the hydraulic fluid supplied to each of the control ports is, in principle, a pressure corresponding to the actuation direction and amount of the operating device 26 corresponding to each of the hydraulic actuators. Thus, the operating device 26 is configured to supply the hydraulic fluid discharged from the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17.

[0056] The proportional valve 31, which functions as a control valve for machine control, is arranged in a pipeline connecting the pilot pump 15 and the pilot port of the control valve in the control valve unit 17, and is configured to change the flow area of ​​the pipeline. In the present embodiment, the proportional valve 31 operates in response to a control command issued by the excavator controller 30. Therefore, the excavator controller 30 can supply the hydraulic fluid discharged from the pilot pump 15 to the pilot port of the control valve in the control valve unit 17 via the proportional valve 31, regardless of the operator's operation of the operating device 26.

[0057] With this configuration, the excavator controller 30 can operate the hydraulic actuator corresponding to the specific operating device 26 even when no operation is performed on the specific operating device 26.

[0058] For example, the excavator controller 30 sets a target rotational speed based on a work mode or the like set in advance by a predetermined operation by the operator or the like, and performs drive control to rotate the engine 11 at a constant rotational speed.

[0059] For example, the excavator control 30 issues a control command to the controller 13 when required and changes the delivery rate of the main pump 14.

[0060] The excavator controller 30 controls, for example, a machine guidance function that guides manual operation of the excavator 100 by the operator using the operating device 26. The excavator controller 30 controls, for example, a machine control function that automatically supports manual operation of the excavator 100 by the operator using the operating device 26.

[0061] Some of the functions of the excavator controller 30 can be performed by other controllers (control devices). That is, the functions of the excavator controller 30 can be achieved in a distributed manner by multiple controllers. For example, the machine guidance function and the machine control function can be achieved by a dedicated controller (control device). [Excavator's hydraulic system]

[0062] Next, a hydraulic system of the excavator 100 according to the present embodiment will be described with reference to Fig. 5 described.

[0063] Fig. 5 is a diagram schematically showing an example of a configuration of the hydraulic system of the excavator 100 according to the present embodiment.

[0064] In Fig. 5, a mechanical drive system, hydraulic fluid lines, pilot lines and an electrical control system are shown by double lines, solid lines, dashed lines and dotted lines, respectively.

[0065] In the hydraulic system implemented by the hydraulic circuit, the hydraulic fluid from each of the main pumps 14L and 14R driven by the engine 11 is circulated to the hydraulic fluid tank via center bypass oil passages C1L and C1R and parallel oil passages C2L and C2R, respectively.

[0066] The middle bypass oil line C1L starts from the main pump 14L, passes through the control valves 171, 173, 175L and 176L, which are arranged in this order in the control valve unit 17, and reaches the hydraulic fluid tank.

[0067] The middle bypass oil line C1R starts from the main pump 14R, passes through the control valves 172, 174, 175R and 176R arranged in this order in the control valve unit 17 and reaches the hydraulic fluid tank.

[0068] The control valve 171 is a spool valve for supplying the hydraulic fluid discharged from the main pump 14L to the travel hydraulic motor 1L and for discharging the hydraulic fluid discharged from the travel hydraulic motor 1L to the hydraulic fluid tank.

[0069] The control valve 172 is a spool valve for supplying the hydraulic fluid discharged from the main pump 14R to the travel hydraulic motor 1R and for discharging the hydraulic fluid discharged from the travel hydraulic motor 1R to the hydraulic fluid tank.

[0070] The control valve 173 is a spool valve for supplying the hydraulic fluid discharged from the main pump 14L to the rotary hydraulic motor 2A and for discharging the hydraulic fluid discharged from the rotary hydraulic motor 2A to the hydraulic fluid tank.

[0071] The control valve 174 is a spool valve for supplying the hydraulic fluid discharged from the main pump 14R to the bucket cylinder 9 and for discharging the hydraulic fluid in the bucket cylinder 9 to the hydraulic fluid tank.

[0072] The control valves 175L and 175R are spool valves for supplying the hydraulic fluids discharged from the main pumps 14L and 14R, respectively, to the boom cylinder 7 and for discharging the hydraulic fluid in the boom cylinder 7 to the hydraulic fluid tank.

[0073] The control valves 176L and 176R supply the hydraulic fluids discharged from the main pumps 14L and 14R to the arm cylinder 8 and direct the hydraulic fluid in the arm cylinder 8 to the hydraulic fluid tank.

[0074] Each of the control valves 171, 172, 173, 174, 175L, 175R, 176L and 176R adjusts the flow rate of hydraulic fluid supplied to and discharged from the hydraulic actuator or switches the flow direction depending on the pilot pressure acting on the pilot port.

[0075] The parallel oil passage C2L supplies hydraulic fluid of the main pump 14L to the control valves 171, 173, 175L, and 176L in parallel with the center bypass oil passage C1L. Specifically, the parallel oil passage C2L branches from the center bypass oil passage C1L upstream of the control valve 171 and is configured to supply the hydraulic fluid of the main pump 14L in parallel to each of the control valves 171, 173, 175L, and 176R. Accordingly, the parallel oil passage C2L can supply the hydraulic fluid to the control valve further downstream, even if the flow of hydraulic fluid through the center bypass oil passage C1L is restricted or blocked by one of the control valves 171, 173, and 175L.

[0076] The parallel oil passage C2R supplies the hydraulic fluid of the main pump 14R to the control valves 172, 174, 175R, and 176R in parallel with the center bypass oil passage C1R. Specifically, the parallel oil passage C2R branches from the center bypass oil passage C1R upstream of the control valve 172 and is configured to supply the hydraulic fluid of the main pump 14R to each of the control valves 172, 174, 175R, and 176R in parallel. Accordingly, the parallel oil passage C2R can supply the hydraulic fluid to the control valve further downstream, even if the flow of hydraulic fluid through the center bypass oil passage C1R is restricted or blocked by one of the control valves 172, 174, and 175R.

[0077] The controllers 13L and 13R adjust the discharge rate of the main pumps 14L and 14R by adjusting the inclination angle of the swash plates of the main pumps 14L and 14R, respectively, under the control of the excavator control system 30.

[0078] The discharge pressure sensor 28L detects the discharge pressure of the main pump 14L, and the detection signal corresponding to the detected discharge pressure is received by the excavator controller 30. The same applies to the discharge pressure sensor 28R. Accordingly, the excavator controller 30 can control the controllers 13L and 13R depending on the discharge pressures of the main pumps 14L and 14R.

[0079] Negative control throttles 18L and 18R are provided in the middle bypass oil passages C1L and C1R, respectively, between the downstream control valves 176L and 176R and the hydraulic fluid reservoir. Accordingly, the flow of hydraulic fluid discharged from the main pumps 14L and 14R is restricted by the negative control throttles 18L and 18R. The negative control throttles 18L and 18R generate pilot pressure to control the regulators 13L and 13R.

[0080] The negative control pressure sensors 19L and 19R detect the negative control pressure, and the detection signal corresponding to the detected negative control pressure is acquired by the excavator controller 30.

[0081] The excavator controller 30 can control the regulators 13L and 13R and adjust the discharge of the main pumps 14L and 14R depending on the discharge pressure of the main pumps 14L and 14R detected by the discharge pressure sensors 28L and 28R. For example, the excavator controller 30 can reduce the discharge by controlling the regulator 13L and adjusting the inclination angle of the swash plate of the main pump 14L when the discharge pressure of the main pump 14L increases. The same applies to the regulator 13R. Accordingly, the excavator controller 30 can control the total power of the main pumps 14L and 14R so that the absorbed power of the main pumps 14L and 14R, expressed as the product of the discharge pressure and the discharge, does not exceed the output power of the engine 11.

[0082] The excavator controller 30 can adjust the discharge volume of the main pumps 14L and 14R by controlling the regulators 13L and 13R depending on the negative control pressure detected by the negative control pressure sensors 19L and 19R. For example, the excavator controller 30 decreases the discharge volume of the main pumps 14L and 14R when the negative control pressure is high and increases the discharge volume of the main pumps 14L and 14R when the negative control pressure is low.

[0083] In particular, when the excavator 100 is in a standby state (which is Fig. 5) in which none of the hydraulic actuators are actuated, the hydraulic fluid discharged from the main pumps 14L and 14R flows through the middle bypass oil passages C1L and C1R and reaches the negative control throttles 18L and 18R. The flow of the hydraulic fluid discharged from the main pumps 14L and 14R increases the negative control pressure generated in front of the negative control pressure throttles 18L and 18R. As a result, the excavator controller 30 reduces the discharge volume of the main pumps 14L and 14R to the minimum allowable discharge volume and suppresses pressure loss (pumping loss) when the discharged hydraulic fluid passes through the middle bypass oil passages C1L and C1R.

[0084] When one of the hydraulic actuators is operated via the operating device 26, the hydraulic fluid discharged from the main pumps 14L and 14R flows into the hydraulic actuator to be operated via the control valve corresponding to the hydraulic actuator to be operated. The flow of hydraulic fluid discharged from the main pumps 14L and 14R reduces or eliminates the amount that reaches the negative control throttles 18L and 18R, thereby reducing the negative control pressure generated upstream of the negative control throttles 18L and 18R. As a result, the excavator controller 30 can increase the discharge volume of the main pumps 14L and 14R, circulate sufficient hydraulic fluid to the hydraulic actuator to be operated, and reliably drive the hydraulic actuator to be operated.

[0085] The operating device 26 includes a left operating lever 26L, a right operating lever 26R, and a travel lever 26D. The travel lever 26D includes a left travel lever 26DL and a right travel lever 26DR.

[0086] The left operating lever 26L and the right operating lever 26R are operated in one or more of the front-to-back direction and the left-to-right direction.

[0087] The left control lever 26L is used for turning and operating the arm 5. When the left control lever 26L is operated in the front-to-back direction, the pilot pressure corresponding to the lever operation amount is introduced into the pilot ports of the control valves 176L and 176R using the hydraulic fluid discharged from the pilot pump 15. When the left control lever 26L is operated in the left-to-right direction, the pilot pressure corresponding to the lever operation amount is introduced into the pilot port of the control valve 173 using the hydraulic fluid discharged from the pilot pump 15.

[0088] Specifically, when the left control lever 26L is operated in the arm closing direction, the excavator controller 30 causes the hydraulic fluid to be introduced into the right pilot port of the control valve 176L and the hydraulic fluid to be introduced into the left pilot port of the control valve 176R. When the left control lever 26L is operated in the arm opening direction, the excavator controller 30 causes the hydraulic fluid to be introduced into the left pilot port of the control valves 176L and the hydraulic fluid to be introduced into the right pilot port of the control valve 176R.When the left control lever 26L is operated counterclockwise, the excavator controller 30 causes hydraulic fluid to be introduced into the left pilot port of the control valve 173, and when it is operated clockwise, the excavator controller 30 causes hydraulic fluid to be introduced into the right pilot port of the control valve 173. Outputs from the operation sensors 29LA and 29LB are used to enable the excavator controller 30 to perform control.

[0089] The right operation lever 26R is used to operate the boom 4 and the bucket 6. When the right operation lever 26R is operated in the front-to-rear direction, the pilot pressure corresponding to the lever operation amount is introduced into the pilot port of the control valve 175 using the hydraulic fluid discharged from the pilot pump 15. When operated in the left-to-right direction, the pilot pressure corresponding to the lever operation amount is introduced into the pilot port of the control valve 174 using the hydraulic fluid discharged from the pilot pump 15.

[0090] Specifically, when the right operation lever 26R is operated in the boom lowering direction, the excavator controller 30 causes the hydraulic fluid to be introduced into the left pilot port of the control valve 175R. When the right operation lever 26R is operated in the boom raising direction, the excavator controller 30 causes the hydraulic fluid to be introduced into the right pilot port of the control valve 175L and the hydraulic fluid to be introduced into the left pilot port of the control valve 175R. When the right operation lever 26R is operated in the bucket closing direction, the excavator controller 30 causes the hydraulic fluid to be introduced into the right pilot port of the control valve 174, and when it is operated in the bucket opening direction, the excavator controller 30 causes the hydraulic fluid to be introduced into the left pilot port of the control valve 174.In order for the excavator control 30 to perform the control, outputs from operating sensors 29RA and 29RB are used.

[0091] The left travel lever 26DL is used to operate a left crawler 1CL and can be configured to operate in conjunction with a left travel pedal. When the left travel lever 26DL is operated in the front-to-rear direction, the excavator controller 30 causes the control pressure corresponding to the lever operation amount to be introduced into the pilot port of the control valve 171 using the hydraulic fluid discharged from the pilot pump 15. The right travel lever 26DR is used to operate a right crawler 1CR and can be configured to operate in conjunction with a right travel pedal. When the right travel lever 26DR is operated in the front-to-rear direction, the excavator controller 30 causes the pilot pressure corresponding to the lever operation amount to be introduced into the pilot port of the control valve 172 by using the hydraulic fluid discharged from the pilot pump 15.In order for the excavator control system 30 to perform the control, outputs from operating sensors 29DL and 29DR described later are used.

[0092] The operation sensor 29 is configured to detect the details of the operations performed by the operator using the operation device 26. In the present embodiment, the operation sensor 29 detects the operation direction and operation amount of the operation device 26 corresponding to each of the actuators and outputs the detected value to the excavator controller 30.

[0093] The operation sensor 29 includes the operation sensors 29LA, 29LB, 29RA, 29RB, 29DL, and 29DR. The operation sensor 29LA detects the details of operations in the front-to-rear direction with respect to the left operation lever 26L performed by the operator and outputs the detected value to the excavator controller 30. The details of the operations include, for example, a lever operation direction, a lever operation amount (a lever operation angle), and the like.

[0094] Similarly, the operation sensor 29LB detects the details of operations in the left-right direction with respect to the left operation lever 26L performed by the operator and outputs the detected value to the excavator controller 30. The operation sensor 29RA detects the details of operations in the front-rear direction with respect to the right operation lever 26R performed by the operator and outputs the detected value to the excavator controller 30. The operation sensor 29RB detects the details of operations in the left-right direction with respect to the right operation lever 26R performed by the operator and outputs the detected value to the excavator controller 30.

[0095] The operation sensor 29DL detects the details of the operations performed by the operator in the front-to-rear direction with respect to the left travel lever 26DL and outputs the detected value to the excavator controller 30. The operation sensor 29DR detects the details of the operations performed by the operator in the front-to-rear direction with respect to the right travel lever 26DR and outputs the detected value to the excavator controller 30.

[0096] The excavator controller 30 receives the output of the actuation sensor 29, outputs a control command to the regulator 13 as needed, and changes the discharge rate of the main pump 14. The excavator controller 30 receives the output of the control pressure sensor 19, which is provided upstream of the throttle 18, and outputs a control command to the regulator 13 as needed to change the discharge rate of the main pump 14. The throttle 18 includes a left throttle 18L and a right throttle 18R, and the control pressure sensor 19 includes negative control pressure sensors 19L and 19R. <Blockkonfiguration des Fernbedienungssystems>

[0097] Fig. 6 is a functional block diagram showing a configuration example of a remote control system SYS according to the present embodiment. Fig. In the example shown in Figure 6, block configurations of the remote control room RC and the excavator 100 included in the remote control system SYS are shown. The hardware configuration of the excavator 100 is as described above, and explanation thereof is omitted. <Konfiguration des Fernbedienungsraums RC>

[0098] The remote control compartment RC includes the remote controller 40, the communication device T2, the speaker A2, the operation sensor 43, and the display device D1. The speaker A2, the communication device T2, and the operation sensor 43 have already been described above, and explanations thereof are omitted.

[0099] Next, the remote control room RC is described. Fig. 7 is a diagram showing an example of the layout of the remote control room RC. In the remote control room RC, a plurality of control elements 42 are arranged around the control seat DS.

[0100] In the present embodiment, as shown in Fig. As shown in Figure 7, the display device D1 is a multi-screen consisting of nine monitors arranged in three vertical rows and three horizontal columns. Specifically, the display device D1 includes a central monitor D1a, an upper monitor D1b, a lower monitor D1c, a left monitor D1d, a right monitor D1e, an upper left monitor D1f, an upper right monitor D1g, a lower left monitor D1h, and a lower right monitor D1i.

[0101] In the present embodiment, the speaker A2 is mounted on top of the display device D1. The speaker A2 is provided, for example, for outputting sound collected by the microphone array A1 of the excavator 100. <<Funktionsblock des Baggers> >

[0102] Back to Fig. 6, each functional block in the excavator controller 30 of the excavator 100 is described. Each functional block in the excavator controller 30 is conceptual and does not necessarily have to be physically configured as shown. All or part of each functional block may be functionally or physically distributed and integrated into arbitrary units. All or part of the processing function performed in each functional block is achieved by a program executed by a CPU. Alternatively, each functional block may be implemented in hardware using wired logic. The excavator controller 30 includes a sensing part 301, an excavator state identifier 302, a map generator 303, a transmission controller 304, a reception controller 305, and an actuator driver 306, by executing a program.

[0103] Fig. Fig. 8 is a conceptual diagram showing a situation around the excavator 100 according to the present embodiment. Fig. The example shown in Figure 8 is an example in which the excavator 100, a worker 801 and a truck 802 are included.

[0104] Worker 801 is talking to excavator 100. Since truck 802 is in operation, its engine noise reaches excavator 100.

[0105] The imaging range of the front camera S6F of the excavator 100 includes the worker 801 and the truck 802.

[0106] In addition, the microphone array A1 collects the utterances of worker 801 and the engine noise of truck 802.

[0107] The excavator controller 30 transmits a sound signal indicative of the sound collected by the microphone array A1 of the excavator 100 to the communication device T2 of the remote control room RC via the communication device T1. Furthermore, the excavator 100 generates information (corresponding to a sound heat map described later) indicating the sound generation situation around the excavator 100 based on the sound signal collected by the microphone array A1, and transmits the generated information from the communication device T1 to the communication device T2 of the remote control room RC. A specific configuration of the excavator 100 for achieving the information transmission is described below.

[0108] Back to Fig. 6, the detection part 301 detects signals from various detection devices provided on the excavator 100. For example, the detection part 301 detects the detection results of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3. The detection part 301 also detects the measurement results of the position, orientation, and the like of the excavator 100 from the positioning device S7. The detection part 301 also detects the image information from the imaging device S6. The detection part 301 also detects the sound signal collected by each of the microphones constituting the microphone array A1.

[0109] The excavator state identifier 302 identifies the state of the excavator 100 based on the signal detected by the detection part 301. In the present embodiment, the state of the excavator 100 includes the position and orientation of the excavator 100 and the state of the attachment (for example, positions of the boom 4, arm 5, and bucket 6) of the excavator 100. The position of the excavator 100 is, for example, the position in the reference coordinate system of the excavator 100 (the latitude, longitude, and altitude of the reference point of the excavator 100). The excavator state identifier 302 specifies the position and orientation of the excavator 100 based on the output of the positioning device S7.

[0110] The condition of the attachment (for example, positions of the boom 4, the arm 5 and the bucket 6) can be determined from the detection result of the angle sensor (the boom angle sensor S1, the arm angle sensor S2 and the bucket angle sensor S3) and the size of the boom 4, the arm 5 and the bucket 6.

[0111] The map generator 303 generates a noise heatmap indicating the intensity of the noise around the excavator 100 based on the noise signal collected by each of the microphones forming the microphone array A1.

[0112] The noise heatmap according to the present embodiment is a two-dimensional map in the height direction (Z-axis direction) of the excavator 100 and the width direction (Y-axis direction) of the excavator 100, and is a map indicating the intensity of the collected noise in each noise collection direction within the noise collection area of ​​the microphone array A1. The noise intensity indicates the magnitude of the amplitude of the noise signal.

[0113] Fig. 9 is a conceptual diagram illustrating generation of the noise heat map by the noise signal of the microphone array A1 in the map generator 303 according to the present embodiment. Fig. For the sake of simplicity, in the example shown in Figure 9, an example is described in which the microphone array A1 consists of the microphones A1a to A1e.

[0114] As in Fig. As shown in Figure 9, the microphones A1a to A1e of the microphone array A1 output sound signals collected in each sound collection direction within the sound collection area to the excavator controller 30. Then, the sound signals detected by the detection part 301 of the excavator controller 30 are output to the map generator 303.

[0115] The map generator 303 sets one or more delay units corresponding to the noise collection direction for the noise signals collected by each of the microphones A1a to A1e.

[0116] In the Fig. In the example shown in Figure 9, in order to determine the intensity of the noise signal collected from a direction inclined from the front by an angle θ in the width direction (Y-axis direction), the map generator 303 allocates a delay unit 303a corresponding to the microphone A1a, a delay unit 303b corresponding to the microphone A1b, a delay unit 303c corresponding to the microphone A1c, and a delay unit 303d corresponding to the microphone A1d. The delay units 303a to 303d are determined according to the angle θ indicating the noise collection direction.

[0117] For example, the delay unit 303a generates a noise signal taking into account the delay time t4 by calculating the noise signal input from the microphone A1a using “e -t4's". s is the Laplacian operator. The time t4 indicates the delay time between the time at which the microphone A1a collects the sound and the time at which the microphone A1e collects the sound when the sound collection direction is inclined by the angle θ in the width direction. Specifically, the time t4 can be calculated from the equation "t4 = 4 l sinθ / v". The distance l is the distance between the microphones. The sound velocity is defined as v. The specific calculations by the delay units 303a to 303d can be performed using publicly known methods, and the explanation thereof is omitted. The sound velocity v can be a fixed value or calculated based on the detected air temperature or the like.In the present embodiment, an example of the delay units 303a to 303d is shown, and any publicly known circuit may be used as long as the delay of the noise signal is taken into consideration.

[0118] The delay unit 303b generates a noise signal taking into account the delay time t3 by calculating the noise signal input from the microphone A1b using “e -t3's ". Time t3 represents the delay time between the time when microphone A1b collects the sound and the time when microphone A1e collects the sound when the sound collection direction is tilted by the angle θ in the width direction. Specifically, time t3 can be calculated using the equation "t3 = 3 l sinθ / v".

[0119] The delay unit 303c generates a noise signal taking into account the delay time t2 by calculating the noise signal input from the microphone A1c using “e -t2·3 ". Time t2 indicates the delay time between the time when microphone A1c collects the sound and the time when microphone A1e collects the sound when the sound collection direction is tilted by the angle θ in the width direction. Specifically, time t2 can be calculated using the equation "t2 = 2 l sinθ / v".

[0120] The delay unit 303d generates a noise signal taking into account the delay time t1 by calculating the sound signal input from the microphone A1d using “e -t1·s". Time t1 indicates the delay time between the time at which microphone A1d picks up the sound and the time at which microphone A1e picks up the sound when the sound collection direction is tilted by the angle θ in the width direction. Specifically, time t1 can be calculated using the equation "t1 = l sinθ / v".

[0121] Based on the values ​​output from the delay units 303a to 303d and the noise signal output from the microphone A1e, a calculator 303f generates a noise signal in which the noise from the direction inclined by the angle θ is emphasized among the noise signals collected by the microphones A1a to A1e.

[0122] The map generator 303 generates a noise heatmap 303g based on the amplitude of the noise signal generated by the calculator 303f. Specifically, a corresponding region (e.g., pixels) is predetermined in the noise heatmap 303g for each noise collection direction. The map generator 303 assigns a color to the region (e.g., pixels) in the noise heatmap 303g. The color reflects the amplitude of the noise signal by emphasizing the sound in the noise collection direction corresponding to the region. For example, the map generator 303 assigns colors such that the larger the amplitude, the redder (or darker) the color, and the smaller the parameter, the whiter the color.

[0123] The map generator 303 generates, for each noise collection direction in the noise collection area of ​​the microphone array A1, a noise signal in which the noise in the direction is highlighted and assigns a color corresponding to the amplitude of the noise signal, thereby generating the noise heatmap 303g.

[0124] In the Fig. In the example shown in Figure 9, for the sake of simplicity, an example will be described in which the sound is collected from a direction inclined by the angle θ in the Y-axis direction, but the direction of sound collection is a combination of the Y-axis direction and the Z-axis direction. Regarding the combination of the direction, the intensity of the sound can be detected by the method described above, and the explanation thereof will be omitted.

[0125] The transmission controller 304 performs control to transmit various information based on the detection result of the detection part 301 to the remote control room RC via the communication device T1. For example, the transmission controller 304 performs control to transmit image information acquired by the imaging device S6, position information indicating the position and orientation of the excavator 100, state information indicating the status of the attachment, the sound signal acquired by the microphone array A1, and the sound heat map to the remote control room RC. The transmission by the transmission controller 304 is performed every predetermined time. The predetermined time can be any time, but it is set to a time interval that allows the user to recognize changes in the situation due to the operation of the excavator 100.For example, the transmission controller 304 may transmit the information at one-second intervals.

[0126] The receiving controller 305 performs control to receive various information from the remote control room RC via the communication device T1. For example, the receiving controller 305 receives an operation signal for controlling the operation of the excavator 100 from the remote control room RC.

[0127] The actuator driver 306 is configured to drive an actuator mounted on the excavator 100. In the present embodiment, the actuator driver 306 generates an operation signal for each of the plurality of solenoid valves included in the proportional valve 31 based on the operation signal transmitted from the remote control room RC.

[0128] Each of the solenoid valves receiving the operating signal increases or decreases the pilot pressure acting on the pilot port of the corresponding control valve in the control valve unit 17. As a result, the hydraulic actuator associated with each control valve operates at a speed corresponding to the stroke amount of the control valve. <<Funktionsblock des Fernbedienungsraums> >

[0129] Each functional block in the remote controller (an example of a controller) 40 of the remote control room RC will be described. Each functional block in the remote controller 40 is conceptual and may not necessarily be physically configured as shown. All or part of each functional block may be functionally or physically distributed and integrated into arbitrary units. All or part of the processing function performed in each functional block is achieved by a program executed by a CPU. Alternatively, each functional block may be implemented in hardware using wired logic. The remote controller 40 includes a reception controller 401, a display screen generator 402, an output controller 403, a signal generator 404, and a transmission controller 405 by executing a program.

[0130] In the present embodiment, a case will be described where the remote controller 40 of the remote control room RC is a controller of the display device D1 (an example of a display section). That is, the configuration in the remote control room RC functions as the display device for supporting remote control of the excavator 100.

[0131] The reception controller 401 performs control of reception of various information from the excavator 100 via the communication device T2.

[0132] As another example, the receiving controller 401 receives (acquires) from the excavator 100 the image information acquired by the imaging device S6, the position information, the state information, the sound signal acquired by the microphone array A1, and the sound heatmap (an example of information about the sound signal). The image information is information acquired by the imaging device S6. The position information is information indicating the position and orientation of the excavator 100 in the reference coordinate system measured by the positioning device S7. The state information is information indicating the state of the attachment (for example, positions of the boom 4, the arm 5, and the bucket 6). The sound heatmap is map information indicating the intensity of the sound around the excavator 100, which is generated by the map generator 303.

[0133] The display screen generator 402 generates a display image by superimposing the image information with a noise heat map indicating the noise generation situation around the excavator 100. Specifically, the display screen generator 402 generates a display image by superimposing the position in the image information and the noise collection direction of the heat map so that they correspond to each other.

[0134] The output controller 403 outputs the sound signal from the microphone array A1 through the speaker A2. Any method, including well-known methods, can be used as a method for outputting sound signals from multiple microphones from the speaker A2.

[0135] The output controller 403 outputs the display image generated by the display screen generator 402 to the display device D1.

[0136] The signal generator 404 generates an operation signal for controlling the operation of the excavator 100 in response to the operation received from the operation sensor 43. The transmission controller 405 transmits the generated operation signal to the excavator 100.

[0137] Fig. 10 is a diagram showing an example of a display screen displayed on the display device D1 according to the present embodiment. Fig. In the example shown in Figure 10, the worker 801 and the truck 802 are displayed in the image information captured by the front camera S6F. Furthermore, noise detection areas 1001 and 1002, represented as the noise heatmap, are superimposed on the image information. The noise detection areas 1001 and 1002 are displayed semi-transparently (the transmissivity can be freely determined).

[0138] The noise detection area 1001 is an area displayed based on the noise heat map. The noise detection area 1001 includes a dark red area 1001a, a red area 1001b, an orange area 1001c, a yellow area 1001d, and a light yellow area 1001e. In the noise detection area 1001, it is shown that the noise decreases as it transitions from the area 1001a to the area 1001e. In other words, the area 1001a is close to the noise source, indicating that the worker 801 is currently speaking.

[0139] Similarly, the noise detection area 1002 includes a red area 1002a, an orange area 1002b, a yellow area 1002c, and a light yellow area 1002d. Similarly, the noise detection area 1002 shows that the noise decreases as it moves from area 1002a to area 1002d. In other words, area 1002a is close to the noise source, indicating that the engine of the truck 802 is running.

[0140] As in the Fig. 10, the output controller 403 in the present embodiment displays the intensity of the sound detected from the direction corresponding to each area displayed on the screen in a recognizable representation of what the sound generation situation is around the excavator 100. In the present embodiment, an example in which the intensity of the sound is represented by a color is shown, but it is not limited to the example in which the intensity of the sound is represented by a color, and other configurations are also possible. For example, the intensity of the sound may be represented by shading, or by changing the luminance, or the like. In the present embodiment, the operator can recognize the source of the sound by displaying the screen in such a way that the intensity of the sound can be recognized.This means that when the operator hears a noise through speaker A2, the operator can identify the source of the noise and thus understand the current situation. The operator can then perform operations according to the current situation. Therefore, safety can be improved.

[0141] In the noise heat map used to generate the display screen according to the present embodiment, the intensity of the noise detected in each direction included in the noise collection area of ​​the microphone array A1 is represented by a color. Thus, when the microphone array A1 collects noise from multiple sources, the sources of the multiple noises are represented as a plurality of noise detection areas in the display image output by the output controller 403. That is, when the noise signal detected by the microphone array A1 includes multiple noise sources, the output controller 403 can display each of the multiple noise sources in a recognizable display manner. Accordingly, the operator can perform the operation after recognizing the multiple noise sources. Therefore, safety can be improved.

[0142] There may be a difference between the imaging range of the front camera S6F and the noise collection range of the microphone array A1. In this case, the microphone array A1 may detect a noise signal from a noise source that is not included in the imaging range of the front camera S6F. In this case, the display screen generator 402 may generate a display image by overlaying the noise detection range with the image information, even if the source is not included in the image information.

[0143] Fig. 11 is a diagram showing an example of a display screen displayed on the display device D1 according to the present embodiment. Fig. In the example shown in Figure 11, the noise source is not displayed in the image information captured by the front camera S6F, but the noise detection area 1101 indicating that the noise source is located on the right side is superimposed into the image information.

[0144] The noise detection area 1101 is an area displayed based on the noise heat map. The noise detection area 1101 includes an orange area 1101a, a yellow area 1101b, and a light yellow area 1101c. When noise is output from the speaker A2, the operator can determine from the noise detection area 1101 that the noise source is on the right side.

[0145] The Fig. The noise detection area 1101 shown in Figure 11 is an example of a display method and is not limited to this display method. That is, any display method can be used as long as it is possible to recognize that the noise signal is detected from a direction outside the area represented by the image information.

[0146] Thus, when the noise signal is detected from a direction outside the area represented by the image information captured by the front camera S6F, the output controller 403 according to the present embodiment displays that the noise signal is detected from outside the area represented by the image information on the display device D1. Accordingly, the operator can estimate the direction of the noise source even if it is not visible on the display device D1. The operator can perform work after estimating the direction of the noise source. Therefore, safety can be improved.

[0147] In the present embodiment, an example in which the excavator controller 30 generates the sound heat map (an example of information about the sound signal) is described. However, it is not limited to the embodiment in which the sound heat map (an example of information about the sound signal) is generated by the excavator controller 30, and the sound heat map may be generated by the remote controller 40, for example. (Variant 1 of the first embodiment)

[0148] In the above-described embodiment, an example is described in which the sound detection area indicating the intensity of the sound is superimposed on the image information. However, the display method is not limited to the above-described embodiment. The content of the utterance can be displayed, for example, as text information.

[0149] The display screen generator 402 of the remote control 40 extracts the voice contained in the sound signal received by the receiving controller 401 and converts the voice into text information. A well-known method can be used as a voice recognition method for converting the voice into text.

[0150] The display screen generator 402 generates a screen by superimposing the converted text information together with the sound detection area on the image information.

[0151] The output controller 403 displays the display screen on which the text information is superimposed. Fig. 12 is a diagram showing an example of a display screen displayed on the display device D1 according to the present embodiment. Fig. In the example shown in Fig. 12, the character string information 1201 is displayed together with the noise detection areas 1001 and 1002 superimposed on the image information captured by the front camera S6F.

[0152] In the present embodiment, the sound represented by the character string information 1201 is not only output from the speaker A2 but also displayed as the character string information 1201. Thus, even if the operator misses or does not hear the voice output from the speaker A2, the operator can recognize the content of the utterance, so that the operator can operate the excavator 100 taking the content of the utterance into account. Therefore, safety can be improved. (Variant 2 of the first embodiment)

[0153] In the above-described embodiment, an example is described in which noise signals received from a specific direction are synthesized based on the delay sum of the noise signals collected by the microphones constituting the microphone array A1, thereby detecting the intensity of the noise in the specific direction and generating the detection result as the noise heatmap. However, the method for detecting the noise generation situation is not limited to the method of the above-described embodiment.

[0154] For example, the noise signals collected by the microphones constituting the microphone array A1 are a combination of noise signals from different sources. Therefore, the map generator 303 can separate the noise signals collected by each of the microphones into different noise signals for each source and determine the source of the noise based on the differences in the arrival time of the separated noise signals at the microphones. Then, the transmission controller 304 transmits the information about the noise signal representing the noise source and the intensity of the separated noise signal to the remote controller 40. The display screen generator 402 generates a display image by superimposing a noise detection area representing the noise source and the intensity of the noise on the image information. Thus, a display screen similar to the embodiment described above can be displayed.As a method for extracting noise signals for each source, for example, a well-known method that combines noise signal doubling and filtering processing can be used. (Second embodiment)

[0155] In the above-described embodiment and variants, a method for displaying the noise heat map indicating the noise generation situation around the excavator 100 will be described. The case where the noise from the worker 801 and the truck 802 is collected as the noise around the excavator 100 for display will be described. However, the object whose noise is collected by the microphone array A1 is not limited to the object around the excavator 100, but also includes an object inside the excavator 100. Therefore, in the second embodiment, the case where the noise from the object inside the excavator 100 is considered will be described.

[0156] Fig. 13 is a conceptual diagram showing a situation around the excavator 100 according to the present embodiment. In the Fig. In the example shown in Figure 13, the worker 801 and the truck 802 are located around the excavator 100. In the example shown in Fig. In the example shown in Figure 13, the engine noise of the engine 11 installed in the excavator 100 is collected by the microphone array A1.

[0157] In the remote control room RC, if the sound output from the speaker A2 includes the engine noise of the engine 11, it may become difficult for the operator to understand the situation around the excavator 100. Therefore, in the present embodiment, the engine noise emitted by the engine 11 of the excavator 100 is controlled to be reduced. In the present embodiment, an example of filtering the engine noise will be described as a method for reducing the engine noise. In the present embodiment, an example of filtering the engine noise will be described as a method for reducing the engine noise, but other methods can also be applied as a method for reducing the engine noise.

[0158] Fig. 14 is a conceptual diagram showing a configuration for outputting sound in the remote control room RC in the present embodiment. Fig. In the example shown in Figure 14, an example is described in which the excavator 100 is equipped with an excavator controller 30A. Furthermore, the remote control room RC is equipped with a remote control 40A.

[0159] In the Fig. 14, the remote controller 40A includes the receiving controller 401, an acoustic separator 1401, an attenuator 1402, an adder 1403, a map generator 303A, the output controller 403, and the display screen generator 402.

[0160] In the present embodiment, an example in which the remote controller 40A filters the engine noise is described. That is, in the above-described embodiment, an example in which the map generator 303 is provided in the excavator controller 30 is described, but in the present embodiment, the map generator 303A is provided in the remote controller 40A.

[0161] In the Fig. In the example shown in Fig. 14, the excavator controller 30A transmits a sound signal indicative of the sound detected by the microphone array A1 to the communication device T2 of the remote control room RC via the communication device T1.

[0162] The acoustic separator 1401 separates the sound signals captured by each of the microphones and contained in the noise signal received by the receiving controller 401 into different noise signals for each source. Fig. In the example shown in Figure 14, the noise signals are separated into an engine noise signal, a worker noise signal 801, and a truck noise signal 802.

[0163] The acoustic separator 1401 according to the present embodiment contains characteristic information (e.g., frequency band and waveform) indicative of the engine noise of the engine 11. Therefore, the acoustic separator 1401 can identify the noise signal indicative of the engine noise from the separated noise signal.

[0164] The acoustic separator 1401 outputs the sound signal indicative of the engine sound to the attenuator 1402. Meanwhile, the acoustic separator 1401 outputs the sound signal other than the engine sound to the map generator 303A and the adder 1403.

[0165] The map generator 303A generates a noise heatmap based on the noise signal detected by removing the engine noise from the noise signals detected by each of the microphones constituting the microphone array A1. The generation method is the same as in Variation 2 of the first embodiment, and the explanation thereof is omitted.

[0166] The attenuator 1402 reduces the amplitude of the noise signal indicative of the engine noise. That is, in the present embodiment, the attenuator 1402 can attenuate a predetermined frequency component corresponding to the engine noise by reducing the amplitude of the noise signal indicative of the engine noise. The degree of reduction can be determined depending on the implementation.

[0167] The adder 1403 adds the sound signal indicating the engine sound attenuated by the attenuator 1402 and the other sound signal. Thus, the adder 1403 generates a sound signal in which the engine sound generated by the engine (a power source) of the excavator 100 (an example of a work machine) is attenuated.

[0168] The output controller 403 controls the output of the noise signal after summation from the speaker A2. At this time, the output controller 403 can synthesize the noise signals from the microphones constituting the microphone array A1 to output them through the speaker A2.

[0169] In the present embodiment, the remote controller 40 attenuates the engine noise through the control described above, so that the operator can easily understand the situation around the excavator 100 from the sound output from the speaker A2.

[0170] In the present embodiment, since the engine noise is attenuated when the sound is output from the speaker A2, the noise burden on the operator can be reduced.

[0171] Since a predetermined frequency component such as the engine noise is attenuated from the sound output from the speaker A2 and the sounds around the excavator 100 are easier to hear, it becomes easier to understand the surrounding situation by hearing, for example, people talking around the excavator 100 or any abnormal sounds. (Variant 1 of the second embodiment)

[0172] In the above-described embodiment, an example is described in which the acoustic separator 1401 separates into different noise signals for each source. However, a method for reducing engine noise is not limited to the method of extracting the noise signal indicative of the engine noise and attenuating the noise signal. In the present variant, an example is described in which a band-stop filter is provided instead of the acoustic separator 1401, the attenuator 1402, and the adder 1403. In the present variant, the map generator can be provided in the excavator controller or the remote controller.

[0173] In the present embodiment, the band-stop filter attenuates the frequency band corresponding to the engine noise from the noise signals captured by each of the microphones and included in the noise signal received by the receiving controller 401. Then, the output controller 403 outputs the noise signal obtained by attenuating the frequency band corresponding to the engine noise from the speaker A2.

[0174] In this variant, an example is described in which a band-stop filter is used as a filter to attenuate the frequency band corresponding to engine noise. However, the filter to attenuate the frequency band corresponding to engine noise is not limited to a band-stop filter; a notch filter can also be used. That is, any filter can be used as long as the frequency band to be attenuated is defined to attenuate the frequency band corresponding to engine noise and does not attenuate other noises.

[0175] In the present variant, by performing the control described above, the similar effect to that of the second embodiment can be achieved.

[0176] In the above-described embodiment and the present variation, an example in which the amplitude of the noise signal indicative of the engine noise is reduced as a filtering method is described. However, the present embodiment is an example of filtering and is not limited to the filtering method. In other words, any filtering that attenuates a noise that hinders recognition of the situation around the excavator 100 when the operator in the remote control room RC operates the excavator 100 based on a predetermined condition may be used. Next, another example of filtering based on a predetermined condition will be described. (Variant 2 of the second embodiment)

[0177] In the above-described embodiment, an example is described in which the engine noise of the engine 11 is muffled to help the operator recognize the surrounding situation. However, there are other methods to help the operator recognize the surrounding situation. For example, when multiple noise sources are present, it may be difficult to hear the human voice. Therefore, in the present variant, a case is described in which hearing the human voice is made easier.

[0178] The frequency band of human voice is approximately 100 Hz to 1,000 Hz. Therefore, when the frequency band is extracted and output from the speaker A2, the voice can be easily heard.

[0179] Therefore, in the present variant, the remote control 40 of the remote control room RC can receive the instruction as to whether or not filtering should be performed in the frequency band. Any method can be used to implement the instruction-receiving control device. For example, a button can be provided for switching whether or not filtering should be performed.

[0180] When the remote control 40 receives a command to filter, it performs filtering to extract the frequency band from 100 Hz to 1,000 Hz from the received audio signal.

[0181] The output controller 403 outputs the filtered sound signal from the speaker A2. In this variant, the frequency band of the voice is extracted, making it easier to hear human voices. As a result, the situation around the excavator 100 can be understood, and safety can be improved.

[0182] In this embodiment, an example is described in which the remote controller 40 of the remote control room RC receives the operation and switches whether or not to extract the frequency band corresponding to human voice from 100 Hz to 1000 Hz. However, in this embodiment, the frequency band whose extraction is switched depending on the operation is not limited to the frequency band corresponding to human voice. For example, as described in the above embodiment, whether or not to attenuate the frequency band corresponding to engine noise can be switched depending on the operation.

[0183] This variant describes an example in which filtering is switched depending on the operation. However, the condition for switching whether filtering is to be performed is not limited to the presence or absence of the operation. For example, filtering can be switched automatically depending on whether a human voice is detected.

[0184] In the above-described embodiment and the present variant, an example is described in which filtering is performed based on a predetermined condition, and either one or more of the generation of the noise heatmap and the output from the speaker A2 are performed based on the filtered noise signal. By displaying the information based on the filtered noise signal to the operator, the operator can thus recognize the necessary information. Therefore, comfort can be improved. (Third embodiment)

[0185] In the above-described embodiment, an example in which the microphone array A1 is provided on the front side of the excavator 100 is described. However, it is not limited to the example in which the microphone array A1 is provided on the front side of the excavator 100 as in the above-described embodiment. Therefore, in the third embodiment, an example in which the microphone array A1 is provided in each of the four directions is described.

[0186] Fig. 15 is a side view of the excavator 100A as a work machine according to the third embodiment. The components other than the microphone array A1 are the same as those in the above-described embodiment, and explanations thereof are omitted.

[0187] The microphone array A1 is configured to collect sounds around the excavator 100. In the present embodiment, the microphone array A1 includes a front microphone array A1F for collecting sounds in front of the excavator 100, a left microphone array A1L for collecting sounds to the left of the excavator 100, a right microphone array A1R for collecting sounds to the right of the excavator 100, and a rear microphone array A1B for collecting sounds behind the excavator 100.

[0188] The imaging device S6 is, for example, a monocular camera with an imaging element such as a CCD or CMOS and can output the captured image to the display device D1.

[0189] For example, the front camera S6F is mounted on the roof of the cabin 10. The left camera S6L is mounted on the left end of the upper surface of the upper rotating body 3. The right camera S6R is mounted on the right end of the upper surface of the upper rotating body 3. The rear camera S6B is mounted on the rear end of the upper surface of the upper rotating body 3.

[0190] For example, the left microphone array A1L is mounted on the left frame of the excavator 100. The right microphone array A1R is mounted on the right frame of the excavator 100. The rear microphone array A1B is mounted on the rear frame of the excavator 100.

[0191] The map generator 303 of the excavator controller 30 generates noise heatmaps for each of the four microphone arrays A1. The transmission controller 304 transmits the four noise heatmaps to the remote controller 40.

[0192] Thus, when the image information of each of the four imaging devices S6 is displayed in the remote control room RC, the noise detection area can be superimposed. Therefore, when an operation to switch the display of the image information acquired by the four imaging devices S6 is received from the operator in the remote control room RC, the noise detection area can be superimposed on the display image displayed after the switching.

[0193] In the present embodiment, the four microphone arrays A1 are arranged to correspond to the installation of the imaging devices S6. Therefore, each image information captured by the four imaging devices S6 can be displayed with the noise detection area superimposed on the image information. Thus, the operator in the remote control room RC can understand the noise source. Therefore, safety can be improved.

[0194] When one of the image information captured by the four imaging devices S6 is displayed on the display device D1, the display screen generator 402 may generate a display image considering each of the noise heatmaps of the microphone array A1 provided in directions that do not correspond to the displayed image information. For example, when the display screen generator 402 generates a display image based on the image information of the front camera S6F, in a case where the noise heatmap of the left microphone array A1L indicates that the noise is loud, information indicating that the noise source is on the left side may be superimposed on the image information of the front camera S6F.In this case, the display screen generator 402 can display the character string information converted from the voice collected by the left microphone array A1L and superimposed on the image information from the front camera S6F. Accordingly, the operator can understand the direction in which the noise source is outside the range displayed on the screen. Thus, the operator can recognize the direction in which the noise source is located. This can improve safety. (Fourth embodiment)

[0195] In the above-described embodiments and variants, a case where the operator performs the operation in the remote control room RC was described. However, the case where remote control is performed as in the above-described embodiment is not limited to an implementation of displays of the sound detection area superimposed on the image information. Therefore, in the fourth embodiment, a case where the display is provided to an operator working on the excavator 100B will be described.

[0196] Fig. 16 is a side view of the excavator 100B as a work machine according to the fourth embodiment. The components other than the imaging device S6 and the microphone array A1 are the same as those in the above-described embodiment, and explanations thereof are omitted.

[0197] The imaging device S6 according to the present embodiment includes the left camera S6L for imaging the area to the left of the excavator 100, the right camera S6R for imaging the area to the right of the excavator 100, and the rear camera S6B for imaging the area behind the excavator 100. Thus, in the present embodiment, the front camera S6F is not provided because the operator can visually check the front direction.

[0198] Similarly, the microphone array A1 includes the left microphone array A1L for collecting sounds on the left side of the excavator 100, the right microphone array A1R for collecting sounds on the right side of the excavator 100, and the rear microphone array A1B for collecting sounds on the rear of the excavator 100. The mounting locations are the same as those of the above-described embodiment, and a description thereof is omitted.

[0199] The map generator 303 of the excavator controller 30 generates noise heatmaps corresponding to each of the three microphone arrays A1. The excavator controller 30 then generates a display image by superimposing the noise heatmaps indicating the noise generation situation around the excavator 100 with the image information and displaying them on a display device DI.

[0200] Next, a configuration example of an image display section 51 and an operation section 52 of the display device DI will be described with reference to Fig. 17 described. Fig. Fig. 17 is a diagram showing a configuration example of the image display section 51 and the operation section 52 of the display device DI. Fig. In the example shown in Fig. 17, a start-up screen including a right image RG, a rear image BG, and a bird's-eye view image TG is displayed on the image display section 51 as a screen displayed before a push switch 52a of the operation section 52 is operated. The start-up screen is displayed during operation of the excavator 100.

[0201] First, the image display area 51 will be described. As shown in Fig. 17, the image display section 51 includes a date and time display area 51a, a travel mode display area 51b, an attachment display area 51c, a fuel consumption display area 51d, an engine control state display area 51e, an engine running time display area 51f, a cooling water temperature display area 51g, a remaining fuel amount display area 51h, a rotation speed level display area 51i, a remaining urea water amount display area 51j, a hydraulic fluid temperature display area 51k, an air conditioning operation state display area 51m, an image display area 51n, and a switching image display area 51p.

[0202] The travel mode display area 51b, the attachment display area 51c, the engine control state display area 51e, the rotation speed level display area 51i, and the air conditioning operation state display area 51m are areas for displaying setting state information, that is, information about the setting state of the excavator 100. The fuel consumption display area 51d, the engine running time display area 51f, the cooling water temperature display area 51g, the fuel remaining amount display area 51h, the urea water remaining amount display area 51j, and the hydraulic fluid temperature display area 51k are areas for displaying operation state information, which is information about the operation state of the excavator 100.

[0203] Specifically, the date and time display area 51a is an area for displaying the current date and time. The travel mode display area 51b is an area for displaying the current travel mode. The attachment display area 51c is an area for displaying an image representing the currently installed attachment. The fuel consumption display area 51d is an area for displaying fuel consumption information calculated by the excavator controller 30. The fuel consumption display area 51d includes an average fuel consumption display area 51d1 for displaying average lifetime fuel consumption or average segmental fuel consumption, and an instantaneous fuel consumption display area 51d2 for displaying instantaneous fuel consumption.

[0204] The engine control status display area 51e is an area for displaying the control status of the engine 11. The engine runtime display area 51f is an area for displaying the cumulative operating time of the engine 11. The cooling water temperature display area 51g is an area for displaying the current temperature status of the engine cooling water. The remaining fuel quantity display area 51h is an area for displaying the remaining quantity of fuel stored in the fuel tank. The rotation speed level display area 51i is an area for displaying an image of the current level adjusted with a scale 75. Fig. 17 shows a state where a first level is selected. The urea water remaining amount display area 51j is an area for displaying an image of the remaining amount of urea water stored in the urea water tank. The hydraulic fluid temperature display area 51k is an area for displaying the temperature state of the hydraulic fluid in the hydraulic fluid tank.

[0205] The air conditioning operating status display area 51m includes an outlet display area 51m1 for displaying the current outlet position, an operation mode display area 51m2 for displaying the current operation mode, a temperature display area 51m3 for displaying the currently set temperature, and an air volume display area 51m4 for displaying the currently set air volume.

[0206] The image display area 51n is an area for displaying an image captured by the imaging device S6. Fig. In the example shown in Figure 17, the image display area 51n displays the right image RG, the rear image BG, and the image TG from the bird's eye view.

[0207] The right image RG is an image of the area to the right of the excavator 100 and includes an image GC1 of the right end of the surface of the upper rotary body 3. The right image RG is an image generated by the excavator controller 30 by superimposing a noise heat map generated from a noise signal detected by the right microphone array A1R on an image captured by the right camera S6R.

[0208] The right image RG shows a worker 1501. Furthermore, a noise detection area 1511, represented by the noise heatmap, is superimposed on the right image RG. The noise detection area 1511 is an area displayed based on the noise heatmap. The noise detection area 1511 includes a dark red area 1511a, a red area 1511b, an orange area 1511c, a yellow area 1511d, and a light yellow area 1511e.

[0209] The rear image BG is an image of the area behind the excavator 100 and includes a counterweight image GC2. The rear image BG is an image generated by the excavator controller 30 by superimposing a noise heatmap generated from a noise signal detected by the rear microphone array A1B onto an image captured by the rearview camera S6B.

[0210] A truck 1502 is visible in the rear image BG. Furthermore, a noise detection area 1512, represented by the noise heatmap, is superimposed on the rear image BG. The noise detection area 1512 is an area displayed based on the noise heatmap. The noise detection area 1521 includes a yellow area 1512a and a light yellow area 1512b.

[0211] The bird's-eye view image TG is a virtual viewpoint image generated by the excavator controller 30, based on images captured by the rear view camera S6B, the left camera S6L, and the right camera S6R. A figure of an excavator corresponding to the excavator 100 is arranged in the center of the bird's-eye view image. This allows the operator to more intuitively understand the positional relationship between the excavator 100 and the objects around the excavator 100.

[0212] Worker 1501 and truck 1502 are also visible from a bird's eye view in image TG. This allows the operator of excavator 100 to recognize the surrounding situation.

[0213] The image display area 51n comprises a first image display area 51n1 arranged at the top and a second image display area 51n2 arranged at the bottom. Fig. In the example shown in Figure 17, the right image RG and the rear image BG are arranged in the first image display area 51n1, and the bird's-eye view image TG is arranged in the second image display area 51n2. In the image display area 51n, the bird's-eye view image TG may be arranged in the first image display area 51n1, and the rear image BG and the right image RG may be arranged in the second image display area 51n2.

[0214] The image display area 51n may be configured to also display the left image. In this case, in the image display area 51n, the left image and the right image RG may be arranged in the first image display area 51n1, and the rear image BG and the bird's-eye view image TG may be arranged in the second image display area 51n2. In this case, the left image may be arranged on the left side of the first image display area 51n1, and the right image RG may be arranged on the right side of the first image display area 51n1.

[0215] In the Fig. In the example shown in Figure 17, the right image RG and the rear image BG can be arranged side by side on the right and left sides, but can also be spaced apart. Fig. In the example shown in Fig. 17, the image display area 51n is a vertically long area, but the image display area 51n may also be a horizontally long area. When the image display area 51n is a horizontally long area, in the image display area 51n, the bird's-eye view image TG may be arranged as the first image display area 51n1 on the left side, and the rear image BG and the right image RG may be arranged as the second image display area 51n2 on the right side. In this case, the left and right images may be spaced apart, and the position of the bird's-eye view image TG and the position of the rear image BG and the right image RG may be interchanged.

[0216] The circuit diagram display area 51p includes circuit diagrams 51p1 to 51p7. Fig. In the example shown in Figure 17, circuit diagrams 51p1 to 51p7 are arranged at the bottom of the image display area 51 at left and right intervals from each other. Circuit diagrams 51p1 to 51p7 display icons representing the functions of the corresponding push buttons 52a1 to 52a7.

[0217] A detail icon for displaying menu detail items is displayed on circuit diagram 51p1. When the operator presses push button 52a1, which corresponds to circuit diagram 51p1, the icons displayed on circuit diagrams 51p2 to 51p7 are switched to the icons associated with the menu detail items.

[0218] In the Fig. In the example shown in Fig. 17, an icon for displaying information about the digital plane is displayed on the circuit diagram 51p4. When the operator operates the push button 52a4 corresponding to the circuit diagram 51p4, the bird's-eye view image TG displayed in the second image display area 51n2 is switched to an image (sub-screen) displaying information about the digital plane. Meanwhile, the rear image BG and the right image RG displayed in the first image display area 51n1 are continuously displayed in the same size. The image (sub-screen) displaying the information about the digital plane may be displayed in the first image display area 51n1. In this case, the rear image BG and the right image RG displayed in the first image display area 51n1 may be displayed in the second image display area 51n2 instead of the bird's-eye view image TG.

[0219] In the Fig. In the example shown in Figure 17, no icons are displayed in circuit diagrams 51p2, 51p3, 51p5, 51p6, and 51p7. Therefore, even if the operator presses the push buttons 52a2, 52a3, 52a5, 52a6, and 52a7 corresponding to circuit diagrams 51p2, 51p3, 51p5, 51p6, and 51p7, no changes occur in the image displayed on the image display section 51.

[0220] The icons displayed on circuit diagrams 51p1 to 51p7 are not limited to the above example, and icons for displaying other information can be displayed.

[0221] Next, the operating section 52 will be described. As shown in Fig. 17, the operating section 52 includes push buttons 52a, which are button-like switches corresponding to each of the circuit diagrams 51p1 to 51p7. Fig. In the example shown in Figure 17, the operation section 52 includes seven push buttons 52a1 to 52a7 arranged in the upper row and seven push buttons 52a8 to 52a14 arranged in the lower row. The push buttons 52a8 to 52a14 are arranged below each of the push buttons 52a1 to 52a7. However, the number, configuration, and arrangement of the push buttons 52a of the operation section 52 are not limited to the above example and may also be implemented in the form of a jog wheel, a jog switch, or the like that combines the functions of multiple push buttons into a single one. The operation section 52 may be formed as a separate element from the display device DI. The circuit diagrams 51p1 to 51p7 may be directly touch-controlled by a touch panel in which the image display section 51 and the operation section 52 are integrated.

[0222] The push buttons 52a1 to 52a7 are arranged below the circuit diagrams 51p1 to 51p7, which correspond to the circuit diagrams 51p1 to 51p7, and each function as a push button for selecting the circuit diagrams 51p1 to 51p7. Since the push buttons 52a1 to 52a7 are arranged below the circuit diagrams 51p1 to 51p7, which correspond to the circuit diagrams 51p1 to 51p7, the operator can select the circuit diagrams 51p1 to 51p7 intuitively.

[0223] The push switch 52a8 is a switch for switching the captured image displayed in the image display area 51n. Each time the push switch 52a8 is pressed, the captured image displayed in the first image display area 51n1 of the image display area 51n is switched, for example, between the rear image, the left image, the right image, and the bird's-eye view image. Each time the push switch 52a8 is pressed, the captured image displayed in the second image display area 51n2 of the image display area 51n can be switched, for example, between the rear image, the left image, the right image, and the bird's-eye view image. Each time the push switch 52a8 is operated, the captured image displayed in the first image display area 51n1 of the image display area 51n and the captured image displayed in the second image display area 51n2 can be switched.Thus, the push switch 52a8 can be used as the operation section 52 for switching the captured image displayed in the first image display area 51n1 or the second image display area 51n2, or can be used for switching the captured image displayed in the first image display area 51n1 and the captured image displayed in the second image display area 51n2. A switch for switching the screen displayed in the second image display area 51n2 can be provided separately.

[0224] The pressure switches 52a9 and 52a10 are switches for adjusting the air volume of the air conditioning system. Fig. In the example shown in Figure 17, the air volume of the air conditioner decreases when the pressure switch 52a9 is operated and increases when the pressure switch 52a10 is operated.

[0225] The pressure switch 52a11 is a switch for switching the cooling and heating functions on and off. Fig. In the example shown in Figure 17, the cooling and heating functions are switched on and off each time the pressure switch 52a11 is operated.

[0226] The pressure switches 52a12 and 52a13 are switches for setting the set temperature of the air conditioning system. Fig. In the example shown in Figure 17, the set temperature decreases when the pressure switch 52a12 is operated and increases when the pressure switch 52a13 is operated.

[0227] The pressure switch 52a14 is a switch for switching the display of the engine running time display area 51f.

[0228] Push buttons 52a2 to 52a6 and 52a9 to 52a13 are configured to allow input of numbers displayed on or next to the respective buttons. Push buttons 52a3, 52a4, 52a5, and 52a11 are configured so that when a cursor is displayed in the image display area 51n, the cursor can be moved left, up, right, or down, respectively.

[0229] The functions assigned to the pressure switches 52a1 to 52a7 and 52a8 to 52a14 are examples described above, and other functions may be assigned.

[0230] As described above, when the push button switch 52a1 corresponding to the circuit diagram 51p1 is operated while the right image RG, the rear image BG, and the bird's-eye view image TG are displayed in the image display area 51n, new icons (icons representing the functions newly assigned to the push buttons 52a1 to 52a7) are displayed on the circuit diagrams 51p2 to 51p7 while the right image RG and the rear image BG are displayed. Therefore, the operator can check the new icons while checking the right image RG and the rear image BG.

[0231] In the above example, if one of the push buttons 52a1 to 52a7 corresponding to the circuit diagrams 51p2 to 51p7 is operated while the right image RG, the rear image BG, and the bird's-eye view image TG are displayed in the image display area 51n, the bird's-eye view image TG is switched to a menu image displaying information corresponding to the selected circuit diagram. Thus, the menu image is displayed while the right image RG and the rear image BG are displayed. Accordingly, the operator can continue to monitor the surroundings (the rear and right areas) even while the menu image is displayed. Therefore, the operator can operate the excavator 100 with the displayed menu image.

[0232] In the present embodiment, the noise detection areas are superimposed on the right image RG and the rear image BG included in the image display area 51n of the display device DI.

[0233] That is, in the present embodiment, the operator of the excavator 100 can visually check the situation in front of the excavator 100 and recognize the noise source together with the noise heard from the front of the excavator 100.

[0234] However, it may be difficult for the operator of the excavator 100 to detect the noise source for areas that are not visible. Therefore, in the excavator controller 30 according to the present embodiment, the above-described screen is displayed on the display device DI.

[0235] Accordingly, the operator of the excavator 100 can detect the noise source around the excavator 100 even in an invisible area. Since the noise source can be identified, for example, by referring to the right image RG and the rear image BG, the source of the noise heard by the operator can be identified, and the action corresponding to the heard noise can be performed. Therefore, safety can be improved. <wirkung>

[0236] With the embodiments and variations described above, the operator can identify the noise source by visually checking the display image, making it easier to check the situation on the construction site. Because the operator can perform operations according to the construction site situation, safety can be improved. Furthermore, work efficiency can be improved because the operator can perform operations according to the construction site situation.

[0237] In the embodiments and variations described above, a case where an excavator is used as an example of a work machine was described. However, the configuration described in the embodiments and variations is not limited to the example where the excavator is used as the work machine, and a crane, a forklift, and the like can be used, for example.

[0238] Although the embodiments illustrating an example of a display device of a work machine, a work machine, and a remote control system for the work machine have been described above, the present invention is not limited to these embodiments. Various changes, modifications, substitutions, additions, deletions, and combinations are possible within the scope of the claims, and these naturally fall within the technical scope of the present invention. DESCRIPTION OF REFERENCE NUMBERS 100 excavators 1 Lower chassis 2 rotating mechanism 3 Upper rotating body 4 booms 5 arms 6 shovels s6 imaging device T1 communication device DI display device A1 microphone array 30 excavator control 301 Recording part 302 Excavator status identification 303 Card Generator 304 Transmission control 305 Reception control 306 Actuator Driver 31 Proportional valve RC remote control room T2 communication device D1 Display device A2 speakers 40 Remote control 401 Reception control 402 Display Screen Generator 403 Output control 404 Signal Generator 405 Transmission control QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2012-58314

[0003] < / wirkung>

Claims

[1] Display device for a working machine, wherein the display device is configured to acquire image information acquired by an imaging device provided on the work machine and to acquire information about a noise signal acquired by each of a plurality of noise collectors provided on the work machine, and the display device comprises a controller configured to display, on a display section, a noise generation situation around the work machine obtained based on the information about the noise signal acquired by each of the plurality of noise collectors, wherein the noise generation situation is superimposed on the image information. [2] The display device of the work machine according to claim 1, wherein the controller displays an intensity of a sound detected from a direction corresponding to each area in the image information in a recognizable display manner as the sound generation situation. [3] The display device of the work machine according to claim 2, wherein, when the noise signal detected by each of the plurality of noise collectors includes a plurality of noise sources, the controller displays the plurality of noise sources in a recognizable display manner. [4] The display device of the work machine according to claim 1, wherein, when the noise signal is detected from a direction outside a range in the image information, the controller displays that the noise signal is detected from outside the range in the image information. [5] The display device of the working machine according to claim 1, wherein the controller converts a voice included in the sound signal into text and displays the converted text on the display section. [6] The display device of the work machine according to claim 1, wherein the controller performs filtering based on a predetermined condition on the sound signal detected by each of the plurality of sound collectors and outputs the filtered sound signal. [7] The display device of the working machine according to claim 6, wherein the controller performs filtering to attenuate a predetermined frequency component and outputs the filtered noise signal. [8] The display device of the work machine according to claim 7, wherein the controller performs filtering to attenuate a noise signal generated by a drive source of the work machine. [9] The display device of the work machine according to claim 6, wherein the controller switches whether to perform filtering according to an operation received from an operation device as the predetermined condition. [10] The display device of the work machine according to any one of claims 6 to 9, wherein the controller outputs the filtered noise signal via a loudspeaker. [11] Working machine comprising: an imaging device; several sound collectors; and a control device configured to acquire image information from the imaging device, acquire a noise signal from each of the plurality of noise collectors, and output to a display device a noise generation situation around the work machine based on the noise signal acquired by each of the plurality of noise collectors superimposed on the image information. [12] The work machine according to claim 11, wherein the plurality of sound collectors are provided to detect sound emitted from an area including an image area to be imaged by the imaging device. [13] Remote control system for a work machine, the remote control system comprising: a working machine; and an information processing device for operating the work machine, wherein the work machine is configured to capture image information from an imaging device provided on the work machine, capture a sound signal from each of a plurality of sound collectors provided on the work machine, and transmit the image information and information about the sound signal, and the information processing device is configured to display on a display device a noise generation situation around the work machine based on the information about the noise signal superimposed on the received image information.

Images