Work machine and remote operation support system for work machine

Abstract

A work machine (100) is provided with a work attachment (101). The work machine (100) comprises a position and orientation detecting device (102), a storage device (103), an operating device (104), a force sense presenting device (105), a drive device (106), and a control device (107). The position and orientation detecting device (102) detects position and orientation information relating to the work machine (100). The storage device (103) stores an avoidance condition, which is a position and orientation condition to be avoided by the work machine (100). The operating device (104) generates an operation signal in response to an operation performed by an operator of the work machine (100). The force sense presenting device (105) presents a force sense to the operator via the operating device (104). The drive device (106) operates the work machine (100) in accordance with the operation signal. The control device (107) actuates the force sense presenting device (105) when the position and orientation information of the work machine (100) satisfies the avoidance condition.

Description

Work machine, work machine remote operation support system

[0001] The present disclosure relates to a work machine and a remote operation support system for the work machine.

[0002] Conventionally, safety devices for construction machinery have been known (for example, see Patent Document 1 below). The conventional safety device for construction machinery described in Patent Document 1 is provided on a construction machinery having a vehicle body that travels by operating an operating means and a front working implement that is mounted on the vehicle body and performs work by operating the operating means, and prevents the construction machinery from coming into contact with obstacles such as workers or objects around the construction machinery.

[0003] This conventional safety device for construction machinery includes a reaction force mechanism and a control means for the reaction force mechanism. The reaction force mechanism is capable of applying an operation reaction force, which is a force that resists the operation of the operating means, to the operating means when the operating means is operated. The control means for the reaction force mechanism controls the reaction force mechanism so that the operation reaction force is applied to the operating means when an obstacle is located within a predetermined danger zone.

[0004] With the above-described configuration, this conventional safety device for construction machinery applies an operation reaction force to the operating means when an obstacle such as a worker or an object is positioned within a predetermined danger zone while the operating means is being operated. This operation reaction force acts against the operation of the operating means and provides a sense of resistance to the operator operating the operating means, enabling the operator to sense that an obstacle is positioned within the danger zone, creating a dangerous situation.

[0005] Japanese Patent Application Laid-Open No. 2006-144349

[0006] The above-mentioned conventional safety devices for construction machinery use obstacle detectors such as laser scanning, ultrasonic transponders, camera images, or bumpers to determine whether the construction machinery is in a dangerous situation where it is approaching an obstacle. However, regardless of whether an obstacle is present, it is desirable to notify the operator of a work machine equipped with a work attachment of a situation that must be avoided by presenting a force feedback.

[0007] The present disclosure provides a work machine equipped with a work attachment that is capable of notifying an operator of a situation that the work machine must avoid by providing a force sense, regardless of whether an obstacle is present, and a remote operation support system for the work machine.

[0008] One aspect of the present disclosure is a work machine equipped with a work attachment, comprising: a position and attitude detection device that detects position and attitude information of the work machine; a storage device that stores avoidance conditions, which are conditions for the position and attitude that the work machine must avoid; an operation device that generates an operation signal in response to operation by an operator of the work machine; a force feedback device that presents a force feedback to the operator via the operation device; a drive device that operates the work machine in response to the operation signal; and a control device that activates the force feedback device when the position and attitude information satisfies the avoidance conditions.

[0009] Another aspect of the present disclosure is a remote operation assistance system for a work machine comprising a work machine including a work attachment and a remote operation assistance device that remotely operates the work machine, the remote operation assistance system comprising: a position and attitude detection device that detects position and attitude information of the work machine; a storage device that stores avoidance conditions that are conditions for the position and attitude that the work machine must avoid; an operation device that generates an operation signal in response to operation by an operator of the work machine; a force feedback device that presents a force feedback to the operator via the operation device; a drive device that operates the work machine in response to the operation signal; and a control device that activates the force feedback device when the position and attitude information satisfies the avoidance conditions.

[0010] According to the above aspects of the present disclosure, it is possible to provide a work machine and a remote operation support system for a work machine that are capable of notifying the operator of the work machine of a situation that the work machine must avoid by presenting a force sense to the operator, regardless of whether an obstacle is present or not.

[0011] FIG. 4 is a diagram illustrating an embodiment of a work machine and a remote operation support system according to the present disclosure. FIG. 4 is a top view and a left side view illustrating an example of the operation device of FIG. 1. FIG. 5 is a top view and a right side view illustrating another example of the operation device of FIG. 1. FIG. 6 is a side view of a shovel as an example of the work machine of FIG. 1. FIG. 7 is a diagram illustrating an example configuration of a drive system mounted on the shovel of FIG. 4. FIG. 8 is a diagram illustrating an example configuration of an electrical system mounted on the shovel of FIG. 4. FIG. 9 is a perspective view showing an example of an avoidance condition that should be avoided by the shovel of FIG. 4. FIG. 10 is a flow diagram showing the operation of a control device for the work machine and system of FIG. 1. FIG. 11 is a side view showing another example of an avoidance condition that should be avoided by the shovel of FIG. 4. FIG. 12 is a side view of a forestry machine as an example of the work machine of FIG. 1.

[0012] Hereinafter, an embodiment of the invention will be described with reference to the drawings.

[0013] 1 is a block diagram showing an embodiment of a work machine and a work machine remote operation support system according to the present disclosure. The work machine remote operation support system SYS of this embodiment includes a work machine 100 and a remote operation support device 200. The work machine remote operation support system SYS may also include, for example, a network server (not shown).

[0014] The work machine 100 is, for example, a work machine such as a shovel, forestry machine, wheel loader, forklift, or crane that is equipped with a work attachment 101 for performing a predetermined task such as excavation, transport, or lifting.

[0015] If the work machine 100 is a shovel, the work attachment 101 includes, for example, a boom, an arm, a bucket, etc. If the work machine 100 is a forestry machine, the work attachment 101 includes, for example, a boom, an arm, a harvester link, a roller harvester device, etc. If the work machine 100 is a wheel loader, the work attachment 101 includes, for example, an arm and a bucket. If the work machine 100 is a forklift, the work attachment 101 includes, for example, a fork, a backrest, a lift chain, a mast, etc. If the work machine 100 is a crane, the work attachment 101 includes, for example, a boom, a jib, a hook, a guy line, etc.

[0016] The work machine 100 includes, for example, a position and attitude detection device 102, a memory device 103, an operation device 104, a force feedback device 105, a drive device 106, and a control device 107. The work machine 100 also includes, for example, a communication device 108, a traveling device 109, and a turning device 110. If the work machine 100 is an excavator, forestry machine, or crane, the work machine 100 will include the turning device 110, but if the work machine 100 is a work machine other than an excavator, forestry machine, or crane, the work machine 100 may include a steering device instead of the turning device 110.

[0017] The position and orientation detection device 102 is a device that detects the position and orientation information of the work machine 100. The position and orientation detection device 102 detects the position and orientation of the work machine 100, including the position and orientation of the work attachment 101, for example, and outputs the detection result to the control device 107 as position and orientation information of the work machine 100.

[0018] The position and attitude detection device 102 includes, for example, a receiver for a global navigation satellite system (GNSS) that acquires position information of the work machine 100. The position and attitude detection device 102 also includes, for example, sensors such as an angle sensor, a stroke sensor, and an inertial sensor that are attached to the work machine 100 including the work attachment 101.

[0019] The storage device 103 includes, for example, a volatile storage device such as RAM, or a non-volatile storage device such as ROM, flash memory, or a magnetic storage device. Avoidance conditions are stored in the storage device 103. Details will be described later, but the avoidance conditions are conditions for the position and attitude (position and attitude) that the work machine 100 should avoid.

[0020] The operation device 104 generates, for example, an operation signal in response to an operation by the operator of the work machine 100, and outputs the operation signal to the control device 107. The operation device 104 includes, for example, an operation lever, an operation pedal, an operation button, an operation dial, an operation switch, and the like, which are installed in the driver's cab of the work machine 100. The operation device 104 may also be, for example, a portable controller or a joystick.

[0021] Figure 2 is a top view showing an example of the operation device 104 in Figure 1. The portable controller 104A shown in Figure 2 is the operation device 104 that is held with both hands by the operator of the work machine 100 and operated, and is connected to the control device 107 by wire or wirelessly. The portable controller 104A has, for example, a housing 104a, a left operation lever 104b, a right operation lever 104c, and a plurality of operation buttons 104d-104l. It should be noted that, for example, an off-the-shelf portable controller used in a game console or the like can also be used as the portable controller 104A.

[0022] The housing 104a has a longitudinal direction that is the left-right direction when held by an operator, and has a pair of gripping portions at each end on the left and right sides that the operator grips with their left and right hands. The pair of gripping portions of the housing 104a extend rearward from the center of the housing 104a so that the pair of gripping portions fits comfortably in the palm of the operator's hand when the operator holds the pair of gripping portions with the thumbs of their left and right hands placed on the left operation lever 104b and the right operation lever 104c, respectively.

[0023] The left operation lever 104b is provided, for example, at the front end of the grip on the left side of the operation device 104. The right operation lever 104c is provided, for example, at the front end of the grip on the right side of the operation device 104. The left operation lever 104b and the right operation lever 104c each detect the amount of operation by the operator in each of the operation directions of the portable controller 104A, that is, the right direction R, the left direction L, the forward direction F, and the backward direction B, and output the detection results to the control device 107.

[0024] The operation directions and operation amounts of the left operation lever 104b and the right operation lever 104c are associated with, for example, the operation and speed of the work attachment 101 of the work machine 100, and the turning device 110 or steering device.

[0025] Operation buttons 104d and 104e are, for example, provided to protrude forward from the front end surface of the grip located on the left side of the housing 104a and are arranged adjacent to each other above and below. Operation buttons 104f and 104g are, for example, provided to protrude forward from the front end surface of the grip located on the right side of the housing 104a and are arranged adjacent to each other above and below. These operation buttons 104d to 104g are associated with operations such as forward movement, reverse movement, and direction change of the traveling device 109 of the work machine 100, for example.

[0026] Operation buttons 104h to 104l are provided, for example, to protrude upward from the upper end surface of the center between the grips arranged on the left and right sides of housing 104a. Operation of these operation buttons 104h to 104l corresponds to, for example, turning on and off the start switch, lights, horn, and other accessories of work machine 100.

[0027] Figure 3 is a top view and a side view showing another example of the operation device 104 shown in Figure 1. The joystick 104B shown in Figure 3 is the operation device 104 that is held and operated in one hand by the operator of the work machine 100, and is connected to the control device 107 by wire or wirelessly. The operation device 104 shown in Figure 1 includes, for example, a pair of joysticks 104B for the left and right hands.

[0028] Each joystick 104B has, for example, a base 104m and a stick 104n attached to the base 104m so as to be tiltable. The joystick 104B may also have, for example, operation buttons, operation switches, operation dials, etc., similar to the portable controller 104A.

[0029] The joystick 104B detects, for example, the amount of operation by the operator of the stick 104n relative to the base 104m in each of the operation directions of rightward R, leftward L, forward F, and backward B, and outputs the detection results to the control device 107. Each operation direction and amount of operation of the joystick 104B is associated with, for example, the operation and speed of the work attachment 101, the slewing device 110, or the steering device of the work machine 100.

[0030] 1 presents a sense of force to the operator via the operation device 104. Here, the sense of force is the sensation that the operator of the work machine 100 feels from the operation device 104 when the operator operates the operation device 104. The force sense presentation device 105 includes, for example, at least one of a force sense presentation actuator and a pseudo force sense presentation actuator.

[0031] When the operation device 104 includes an operation lever installed in the driver's cab of the work machine 100, the force sense presentation device 105 includes a force sense presentation actuator that presents a force sense to the operator via the operation lever. The force sense presentation actuator of the force sense presentation device 105 presents to the operator, via the operation lever, for example, a force sense in the opposite direction to the operation direction of the operation lever by the operator, or a force sense in a direction that will cause the work machine 100 to escape the avoidance condition. The force sense presentation actuator can be configured, for example, by an electric motor, a hydraulic cylinder, or an air cylinder.

[0032] 2, when the operation device 104 is a portable controller 104A, the force feedback device 105 includes, for example, a pseudo-force feedback actuator 105a that provides a pseudo force feedback (hereinafter referred to as a "pseudo-force feedback") to an operator holding the housing 104a. The pseudo-force feedback actuator 105a is mounted on the portable controller 104A, for example, and provides a pseudo-force feedback in a predetermined direction to the operator of the portable controller 104A by generating pseudo-force feedback vibrations that are vibrations with a waveform asymmetric in the direction in which the pseudo-force feedback is provided.

[0033] The portable controller 104A may include, for example, a plurality of pseudo-force sense presentation actuators 105a. Specifically, the portable controller 104A may include, for example, pseudo-force sense presentation actuators 105a on the left and right gripping portions of the housing 104a, respectively. This allows, for example, the left and right pseudo-force sense presentation actuators 105a to generate pseudo-forces in the front and rear directions of the housing 104a, respectively, and presents the operator with a force sense in the rotational direction of the housing 104a (right-handed RHT or left-handed LHT).

[0034] Furthermore, the portable controller 104A may be provided with pseudo-force sense actuators 105a at the front and rear ends of the housing 104a, for example. This allows the front and rear pseudo-force sense actuators 105a to generate pseudo-forces in the up and down directions of the housing 104a, respectively. As a result, it is possible to present to the operator holding the operating device 104 a rotational force sense in which the front end of the housing 104a is lifted upward UT or pushed downward DT.

[0035] Furthermore, when the operation device 104 is a portable controller 104A, the force sense presentation device 105 includes a force sense presentation actuator 105b that presents a force sense to the operator via, for example, the left operation lever 104b and the right operation lever 104c. The force sense presentation actuator 105b presents, for example, a force sense in the opposite direction to the operation direction of the left operation lever 104b and the force sense presentation actuator 105b by the operator, or a force sense in the direction in which the work machine 100 will escape the avoidance condition, to the operator via the left operation lever 104b and the right operation lever 104c.

[0036] 3, when the operation device 104 is a joystick 104B, the force sense presentation device 105 includes a force sense presentation actuator 105b that presents a force sense to the operator via, for example, a stick 104n. The force sense presentation actuator 105b presents to the operator via the stick 104n, for example, a force sense in the opposite direction to the operation direction of the stick 104n by the operator, or a force sense in the direction in which the work machine 100 will escape the avoidance condition. In this case as well, the force sense presentation actuator 105b can be configured by, for example, an electric motor, a hydraulic cylinder, or an air cylinder.

[0037] Furthermore, when the operation device 104 is a joystick 104B, the force feedback device 105 may include, for example, a pseudo force feedback actuator 105a in the grip portion of the stick 104n. This enables the force feedback device 105 to present to the operator, via the grip portion of the stick 104n, for example, a pseudo force feedback in the opposite direction to the operation direction of the stick 104n by the operator, or a pseudo force feedback in a direction that will cause the work machine 100 to escape from the avoidance condition.

[0038] 1 operates the work machine 100 in response to an operation signal output from the operation device 104 and input to the control device 107. Specifically, when an operation signal is input from the operation device 104, the control device 107 outputs a control signal in response to the operation signal to the drive device 106. The drive device 106 drives the work machine 100 based on the control signal input from the control device 107.

[0039] The drive device 106 includes, for example, a power source such as an internal combustion engine or an electric motor, a hydraulic pump driven by the power source, a control valve that controls the direction and flow rate of hydraulic oil pumped from the hydraulic pump, and a hydraulic actuator driven by the hydraulic oil supplied from the control valve. The control valve is controlled, for example, in response to a control signal input from the control device 107 to the drive device 106. The hydraulic actuator includes, for example, a hydraulic motor and a hydraulic cylinder, and drives the work attachment 101, the traveling device 109, and the swing device 110 or the steering device.

[0040] The control device 107 is a controller configured with, for example, a central processing unit (CPU), memories such as ROM and RAM, a timer, and one or more microcontrollers including input / output units. The control device 107 executes programs stored in the memory or storage device 103, for example, to acquire detection results from various sensors including the position and orientation detection device 102, perform various calculations, and control each unit of the work machine 100.

[0041] In the work machine 100 of this embodiment, the control device 107 activates the force feedback device 105 when the position and orientation information of the work machine 100 based on the detection results of the position and orientation detection device 102 satisfies the specified avoidance conditions described below.

[0042] The communication device 108 is, for example, a wireless communication device mounted on the work machine 100, and is connected to the remote operation support device 200 via a wireless communication line so that information can be communicated with the remote operation support device 200. Furthermore, when the remote operation support device 200 is installed in a remote control room or the like that is located away from the work site of the work machine 100, the communication device 108 may be connected to the remote operation support device 200 so that information can be communicated with the remote operation support device 200 via, for example, a wireless communication line, a wired communication line, a server, or the like. In this case, the work machine remote operation support system SYS may include a server.

[0043] The traveling device 109 is driven by, for example, the drive device 106 to travel the work machine 100. Specifically, the traveling device 109 includes, for example, a plurality of wheels driven by a hydraulic motor of the drive device 106, or a lower traveling body equipped with crawlers.

[0044] The slewing device 110 is driven, for example, by the drive device 106 to slewing the work machine 100. Specifically, the slewing device 110 is driven, for example, by a slewing hydraulic motor of the drive device 106 to slewing an upper slewing body of the work machine 100 mounted on the traveling device 109.

[0045] The steering device is driven, for example, by the drive device 106, and changes the direction of travel of the work machine 100. Specifically, the steering device includes, for example, an articulation unit that tilts the front body of the work machine 100 relative to the rear body by extension and contraction of a steering cylinder of the drive device 106.

[0046] The remote operation support device 200 is a device that remotely controls the work machine 100. For example, as described above, the remote operation support device 200 is installed in a remote control room that is located away from the work site of the work machine 100. Furthermore, the remote operation support device 200 may be configured as a portable controller or joystick that is carried by an operator who operates the work machine 100 from outside the cab at the work site of the work machine 100, as shown in Figures 2 and 3 , for example.

[0047] The remote operation assistance device 200 includes, for example, a control device 201 , a communication device 202 , a storage device 203 , an operation device 204 , and a force feedback device 205 .

[0048] The control device 201 is a controller configured with, for example, one or more microcontrollers including a CPU, memories such as ROM and RAM, a timer, and an input / output unit. The control device 201 controls each unit of the remote operation assistance device 200 by, for example, executing a program stored in the memory or the storage device 203.

[0049] Furthermore, in the remote operation support system SYS for a work machine of this embodiment, the control device 201 activates the force feedback device 205, for example, when the position and posture information of the work machine 100 acquired from the work machine 100 via the communication device 202 satisfies a predetermined avoidance condition described below.

[0050] The communication device 202 is, for example, a wireless communication device provided in the remote operation assistance device 200, and is connected via a wireless communication line to be able to communicate information with the communication device 108 of the work machine 100. Furthermore, as described above, when the remote operation assistance device 200 is installed in a remote location such as a remote operation room, the communication device 202 may be connected via, for example, a wireless communication line, a wired communication line, a server, or the like to be able to communicate information with the communication device 108 of the work machine 100.

[0051] The storage device 203 includes, for example, a volatile storage device such as a RAM, or a non-volatile storage device such as a ROM, flash memory, or magnetic storage device. Similar to the storage device 103 of the work machine 100, the storage device 203 stores avoidance conditions, which are conditions for the position and attitude of the work machine 100 that should be avoided.

[0052] The operation device 204 generates an operation signal in response to operation by the operator of the work machine 100, for example, and outputs it to the control device 201. The control device 201 transmits the operation signal input from the operation device 204 to the communication device 108 of the work machine 100 via the communication device 202. The control device 107 of the work machine 100 controls the drive device 106 based on the operation signal received via the communication device 108, and operates the work attachment 101, traveling device 109, and swing device 110 or steering device of the work machine 100.

[0053] The operation device 204 has a configuration similar to the operation levers, operation pedals, operation buttons, operation dials, and operation switches installed in the driver's cab of the work machine 100, for example, and is installed in a remote operation room in a remote location away from the work site of the work machine 100. Furthermore, as shown in Figure 2 or Figure 3, the remote operation support device 200 may be configured as, for example, a portable controller or joystick for use in a remote location or at the work site.

[0054] 2, the operation device 204 can have, for example, a configuration similar to that of the portable controller 104A of the work machine 100. Specifically, the operation device 204 has, for example, a housing 204a, a left operation lever 204b, a right operation lever 204c, and a plurality of operation buttons 204d-204l. The configuration of each part of the operation device 204 is similar to the configuration of each part of the portable controller 104A, and therefore detailed description thereof will be omitted.

[0055] 3, the operation device 204 may have a configuration similar to that of the joystick 104B of the work machine 100. Specifically, the operation device 204 has, for example, a base 204m and a stick 204n. The configuration of each part of the operation device 204 is similar to that of the joystick 104B, and therefore detailed description thereof will be omitted. The remote operation support device 200 may also have, for example, operation buttons, operation switches, operation dials, etc., similar to that of the portable controller 104A.

[0056] The force sense presentation device 205 presents a force sense to the operator via the operation device 204. When the operation device 204 includes a configuration similar to an operation lever or the like installed in the driver's cab of the work machine 100, the force sense presentation device 105 includes a force sense presentation actuator that presents a force sense to the operator via the operation lever. The force sense presentation actuator of the force sense presentation device 205 presents to the operator, via the operation lever, for example, a force sense in the opposite direction to the operation direction of the operation lever by the operator, or a force sense in a direction that will cause the work machine 100 to escape the avoidance condition. The force sense presentation actuator can be configured, for example, by an electric motor, a hydraulic cylinder, an air cylinder, or the like.

[0057] 2, the force feedback device 205 includes a plurality of pseudo force feedback actuators 205a mounted on a housing 204a, similar to the portable controller 104A of the work machine 100. This enables the force feedback device 205 to present to the operator of the operation device 204 a pseudo force feedback in a predetermined direction, including a clockwise RHT or counterclockwise LHT of the housing 204a, or a direction in which the front end of the housing 204a is lifted upward UT or pushed downward DT.

[0058] 3, the force feedback device 205 includes a force feedback actuator 205b that provides a force feedback to the operator via a stick 204n. The force feedback actuator 205b provides the operator via the stick 204n with a force feedback in the opposite direction to the operation direction of the stick 204n by the operator, or a force feedback in the direction in which the work machine 100 will escape the avoidance condition. In this case as well, the force feedback actuator 205b can be configured using, for example, an electric motor, a hydraulic cylinder, or an air cylinder.

[0059] 3, the force feedback device 205 may include a pseudo force feedback actuator 205a in the gripping portion of the stick 204n. This enables the force feedback device 205 to present to the operator, via the gripping portion of the stick 204n, for example, a pseudo force feedback in the opposite direction to the operation direction of the stick 204n by the operator, or a pseudo force feedback in a direction that allows the work machine 100 to escape the avoidance condition.

[0060] Hereinafter, an embodiment in which the work machine 100 is a shovel or a forestry machine will be described.

[0061] Figure 4 is a side view of an excavator 100S as an example of the work machine 100. An upper rotating body 3 is rotatably mounted on a lower traveling body 1 of the excavator 100S, which corresponds to the traveling device 109 in Figure 1, via a rotating mechanism 2, which corresponds to the rotating device 110 in Figure 1. A boom 4 is attached to the upper rotating body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6 is attached to the tip of the arm 5 as an end attachment.

[0062] The boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of a work attachment 101 in Fig. 1. The boom 4 is driven by a boom cylinder 7, the arm 5 is driven by an arm cylinder 8, and the bucket 6 is driven by a bucket cylinder 9. The boom cylinder 7, arm cylinder 8, and bucket cylinder 9 are included in the drive device 106 in Fig. 1.

[0063] A boom angle sensor S1 is attached to the boom 4, an arm angle sensor S2 is attached to the arm 5, and a bucket angle sensor S3 is attached to the bucket link. A swing angular velocity sensor S4 is attached to the upper swing structure 3. The boom angle sensor S1, arm angle sensor S2, bucket angle sensor S3, and swing angular velocity sensor S4 are included in the position and attitude detection device 102 in FIG. 1 .

[0064] The boom angle sensor S1 is one of the attitude detection sensors and is configured to detect the rotation angle of the boom 4. In this embodiment, the boom angle sensor S1 is a stroke sensor that detects the stroke amount of the boom cylinder 7, and derives the rotation angle of the boom 4 around the boom foot pin that connects the upper rotating body 3 and the boom 4 based on the stroke amount of the boom cylinder 7.

[0065] The arm angle sensor S2 is one of the posture detection sensors and is configured to detect the rotation angle of the arm 5. In this embodiment, the arm angle sensor S2 is a stroke sensor that detects the stroke amount of the arm cylinder 8, and derives the rotation angle of the arm 5 around the connecting pin that connects the boom 4 and the arm 5 based on the stroke amount of the arm cylinder 8.

[0066] The bucket angle sensor S3 is one of the attitude detection sensors and is configured to detect the rotation angle of the bucket 6. In this embodiment, the bucket angle sensor S3 is a stroke sensor that detects the stroke amount of the bucket cylinder 9, and derives the rotation angle of the bucket 6 around the connecting pin that connects the arm 5 and the bucket 6 based on the stroke amount of the bucket cylinder 9.

[0067] The rotation angular velocity sensor S4 is configured to detect the rotation angular velocity of the upper rotating body 3. In this embodiment, the rotation angular velocity sensor S4 is a gyro sensor. The rotation angular velocity sensor S4 may be configured to calculate a rotation angle based on the rotation angular velocity. The rotation angular velocity sensor S4 may also be configured with another sensor such as a rotary encoder.

[0068] The upper rotating body 3 is equipped with a cabin 10 serving as a driver's room, an engine 11, a positioning device 18, a sound collection device A1, an imaging device C1, a communication device T1, etc. A controller 30 is also installed inside the cabin 10. A driver's seat, operating devices, etc. are also installed inside the cabin 10. However, the excavator 100S may be an unmanned excavator in which the cabin 10 is omitted.

[0069] The engine 11 is a drive source for the excavator 100S. In this embodiment, the engine 11 is a diesel engine. An output shaft of the engine 11 is coupled to input shafts of the main pump 14 and the pilot pump 15 (see FIG. 6 ). The engine 11 is included in the drive device 106 in FIG. 1 .

[0070] The positioning device 18 is configured to measure the position of the excavator 100S. In this embodiment, the positioning device 18 is a GNSS compass, and is configured to be able to measure the position and orientation of the upper rotating body 3. The positioning device 18 is included in the position and attitude detection device 102 in Figure 1 .

[0071] The sound collection device A1 is configured to collect sounds generated around the excavator 100S. In this embodiment, the sound collection device A1 is a microphone attached to the upper rotating body 3.

[0072] The imaging device C1 is configured to capture images of the surroundings of the excavator 100S. In this embodiment, the imaging device C1 includes a rear camera C1B attached to the rear end of the upper surface of the upper rotating body 3, a front camera C1F attached to the front end of the upper surface of the cabin 10, a left camera C1L attached to the left end of the upper surface of the upper rotating body 3, and a right camera C1R attached to the right end of the upper surface of the upper rotating body 3. The imaging device C1 may be a spherical camera installed at a predetermined position within the cabin 10. The predetermined position is, for example, a position corresponding to the eye position of an operator seated in a driver's seat installed within the cabin 10.

[0073] The communication device T1, which corresponds to the communication device 108 in Fig. 1, is configured to control communication with devices external to the shovel 100S. In this embodiment, the communication device T1 is configured to control wireless communication between the communication device T1 and devices external to the shovel 100S via a wireless communication network. The controller 30, which corresponds to the control device 107 in Fig. 1, is a calculation device that executes various calculations, as described above.

[0074] Fig. 5 is a block diagram showing an example of the configuration of a drive system mounted on the excavator 100S shown in Fig. 4. In Fig. 5, mechanical power transmission lines are indicated by double lines, hydraulic oil lines by thick solid lines, pilot lines by dashed lines, and electrical control lines by dotted lines.

[0075] 1 is composed of an engine 11, a regulator 13, a main pump 14, a pilot pump 15, a control valve unit 17, an engine control unit 74, a solenoid valve unit 45, etc. The engine 11 is drive-controlled by the engine control unit 74.

[0076] The main pump 14 supplies hydraulic oil to a control valve unit 17 via a hydraulic oil line 16. In this embodiment, the main pump 14 is a swash plate type variable displacement hydraulic pump.

[0077] The regulator 13 is configured to control the discharge rate of the main pump 14. In this embodiment, the regulator 13 is configured to adjust the swash plate tilt angle of the main pump 14 in response to the discharge pressure of the main pump 14 or a control signal from the controller 30. The discharge rate (displacement volume) of the main pump 14 per rotation is controlled by the regulator 13.

[0078] The pilot pump 15 is configured to supply hydraulic oil to various hydraulic control devices via a pilot line 25. In this embodiment, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pump 15 may be omitted. In this case, the function of the pilot pump 15 may be realized by the main pump 14. That is, the main pump 14 may have a function of supplying hydraulic oil to the solenoid valve unit 45 and the like via a throttle or the like, in addition to a function of supplying hydraulic oil to the control valve unit 17.

[0079] The control valve unit 17 is configured to selectively supply hydraulic oil received from the main pump 14 to one or more hydraulic actuators. In this embodiment, the control valve unit 17 includes a plurality of control valves corresponding to the plurality of hydraulic actuators. The control valve unit 17 is configured to selectively supply hydraulic oil discharged from the main pump 14 to one or more hydraulic actuators.

[0080] The hydraulic actuators include, for example, a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, a left-side traveling hydraulic motor 1L, a right-side traveling hydraulic motor 1R, and a swing hydraulic motor 2A. These hydraulic actuators are included in the drive unit 106 in FIG. 1 .

[0081] The controller 30 is configured to control the solenoid valve unit 45 based on an operation signal received through the communication device T1. In this embodiment, the operation signal may be transmitted from a remote operation support device 200 installed in a remote operation room at a remote location away from the work site WS, or may be input from an operation device 104 operated by an operator inside or outside the cabin 10 at the work site WS of the excavator 100S.

[0082] The solenoid valve unit 45 includes a plurality of solenoid valves arranged in the pilot lines 25 that connect the pilot pump 15 to the pilot ports of the control valves in the control valve unit 17 .

[0083] In this way, the controller 30 can raise and lower the boom 4, open and close the arm 5, open and close the bucket 6, rotate the upper rotating body 3, and move the lower traveling body 1, etc., in response to operation signals input from the operating device 204 of the remote operation support device 200 or the operating device 104 of the work machine 100.

[0084] Fig. 6 is a block diagram showing an example of the configuration of an electrical system mounted on the excavator 100S of Fig. 4. As shown in Fig. 5, the engine 11 is connected to an engine control unit 74. Various data indicating the state of the engine 11 is transmitted from the engine control unit 74 to the controller 30. The controller 30 is configured to be able to store various data indicating the state of the engine 11 in a memory 30a.

[0085] The battery 70 is configured to supply electric power to various electric loads mounted on the excavator 100S. The alternator 11a (generator), starter 11b, controller 30, electrical components 72, etc. are configured to operate on the electric power stored in the battery 70. The starter 11b is configured to be driven by the electric power stored in the battery 70 and start the engine 11. The battery 70 is also configured to be charged with the electric power generated by the alternator 11a.

[0086] The water temperature sensor 11c transmits data related to the temperature of the engine coolant to the controller 30. The regulator 13 transmits data related to the swash plate tilt angle to the controller 30. The discharge pressure sensor 14b transmits data related to the discharge pressure of the main pump 14 to the controller 30. The positioning device 18 transmits data related to the position of the excavator 100S to the controller 30.

[0087] An oil temperature sensor 14c is provided in a conduit 14-1 between the main pump 14 and a hydraulic oil tank that stores hydraulic oil to be sucked by the main pump 14. The oil temperature sensor 14c transmits data relating to the temperature of the hydraulic oil flowing through the conduit 14-1 to the controller 30.

[0088] A urea water remaining amount sensor 21a provided in the urea water tank 21 transmits data relating to the remaining amount of urea water to the controller 30. A fuel remaining amount sensor 22a provided in the fuel tank 22 transmits data relating to the remaining amount of fuel to the controller 30.

[0089] 1 is configured to transmit and receive information via wireless communication with the communication device 202 of the remote operation support apparatus 200. In this embodiment, the communication device T1 and the communication device 202 are configured to transmit and receive information via a fifth generation mobile communication line (5G line), an LTE line, a satellite line, or the like.

[0090] In a remote control room located away from the work site, for example, a remote operation support device 200, a sound output device, an indoor imaging device, an image generation device, a display device, etc., which are not shown, are installed.

[0091] The sound output device is configured to output sound. In this embodiment, the sound output device is a speaker, and is configured to reproduce sound collected by the sound collection device A1 attached to the excavator 100S.

[0092] The indoor image capturing device is configured to capture an image of the inside of the remote control room. In this embodiment, the indoor image capturing device is a camera installed inside the remote control room and configured to capture an image of the operator seated in the driver's seat.

[0093] The display device is configured to display information about the situation around the excavator 100S. The display device is, for example, a multi-display made up of nine monitors arranged in three rows and three columns, and is configured to be able to display the state of the space in front, to the left, and to the right of the excavator 100S. Each monitor is a liquid crystal monitor, an organic EL monitor, or the like. However, the display device may be made up of one or more curved monitors, or may be made up of a projector.

[0094] The display device may be a display device that can be worn by the operator. For example, the display device may be a head-mounted display configured to be able to transmit and receive information to and from the control device 201 via wireless communication. The head-mounted display may be connected to the control device 201 by wire. The head-mounted display may be a transparent head-mounted display or a non-transparent head-mounted display. The head-mounted display may be a monocular head-mounted display or a binocular head-mounted display.

[0095] The display device is configured to display an image that enables the operator in the remote control room to visually recognize the surroundings of the shovel 100S. In other words, the display device displays an image so that the operator can confirm the surrounding conditions of the shovel 100S as if he or she were inside the cabin 10 of the shovel 100S, even though he or she is in the remote control room.

[0096] As described above, the image generation device is a device that generates images to support the remote operation of the shovel 100S, which is the work machine 100 equipped with an excavation attachment as the work attachment 101. The image generation device generates images captured by the imaging device C1 at the work site WS where the shovel 100S is working, and displays the images on the display device.

[0097] The operation device 204 of the remote operation support device 200 installed in the remote operation room can have a configuration similar to that of the operation device 104 in the cabin 10 of the excavator 100S, for example. Specifically, a driver's seat similar to that in the cabin 10 of the excavator 100S is arranged in the remote operation room, and a left console box and a right console box are arranged on the left and right sides of the driver's seat, respectively. A left operation lever and a right operation lever are arranged at the front ends of the upper surfaces of the left and right console boxes, respectively. A travel lever and a travel pedal are also arranged in front of the driver's seat. Furthermore, an engine speed adjustment dial is arranged in the center of the upper surface of the right console box. The left operation lever, right operation lever, travel lever, travel pedal, and engine speed adjustment dial constitute the operation device 204 of the remote operation support device 200.

[0098] The remote operation support device 200 may be configured, for example, by a portable controller shown in Fig. 2 or a joystick shown in Fig. 3. In this case, the remote operation support device 200 may be located in a remote operation room at a remote location away from the work site WS of the shovel 100S, or may be carried by an operator who operates the shovel 100S at the work site WS.

[0099] Figure 7 is a perspective view showing an example of an avoidance condition that should be avoided by an excavator 100S, which is an example of the work machine 100 shown in Figure 1. In the example shown in Figure 7, at a work site WS where the excavator 100S performs excavation work, a three-dimensional appropriate range AR is set in advance as a range in which the excavator 100S can avoid contact with surrounding obstacles and can operate within the movable range of the work machine 100, including the work attachment 101. The appropriate range AR is stored in advance, for example, in at least one of the storage device 103 of the excavator 100S and the storage device 203 of the remote operation support device 200.

[0100] In the example shown in Fig. 7, the working range in which the shovel 100S can perform work while avoiding contact with surrounding obstacles is a range in front of the utility pole UP, which is an obstacle, and below the power line PL, which is also an obstacle. Also, in the example shown in Fig. 7, the working range of the shovel 100S includes a range below the ground surface ES. In such a case, the appropriate range AR is set, for example, to the outer edge of the working range of the shovel 100S and inside thereof, and also to the outer edge of the movable range of the shovel 100S, including the work attachment 101, and inside thereof.

[0101] Furthermore, thresholds Dth1 and Dth2 for determining whether or not an avoidance condition for the shovel 100S to avoid an accident is met are stored in advance in at least one of the storage device 103 of the shovel 100S and the storage device 203 of the remote operation support device 200. These thresholds Dth1 and Dth2 are thresholds for determining whether or not at least a part of the shovel 100S is included in a range near the outer edge of the working range of the shovel 100S or a range near the outer edge of the movable range of the shovel 100S, for example.

[0102] That is, the range near the outer edge of the working range of the shovel 100S is, for example, a range within the working range of the shovel 100S where the distance from the outer edge of the working range is equal to or less than the threshold values ​​Dth1, Dth2. Also, the range near the outer edge of the movable range of the shovel 100S is a range within the movable range of the shovel 100S where the distance from the outer edge of the movable range is equal to or less than the threshold values ​​Dth1, Dth2.

[0103] In the example shown in Fig. 7 , the avoidance conditions that the shovel 100S must avoid include, for example, at least a portion of the shovel 100S including the work attachment 101 being in a range close to the outer edge of the working range of the shovel 100S or in a range close to the outer edge of the movable range of the shovel 100S. In other words, the avoidance conditions that the shovel 100S must avoid are conditions that the distance between at least a portion of the shovel 100S including the work attachment 101 shown in Fig. 7 and the outer edge of the appropriate range AR is equal to or less than the threshold values ​​Dth1, Dth2.

[0104] Fig. 8 is a flow diagram showing an example of a force feedback processing flow KSP executed by the controller 30 of the excavator 100S or the control device 201 of the remote operation support device 200 as an example of the control device 107 of the work machine 100 shown in Fig. 1. Hereinafter, for the sake of simplicity of explanation, the "controller 30 or control device 201 as the control device 107" will be referred to as the "control device 107, 201".

[0105] When the control devices 107, 201 start the force feedback processing flow KSP shown in Figure 8, they first acquire position and orientation information and operation signals of the shovel 100S from the position and orientation detection device 102 and the operation devices 104, 204 (information acquisition processing P1).

[0106] Next, the control devices 107, 201 calculate the minimum distance Dmin between the shovel 100S including the work attachment 101 and the outer edge of the appropriate range AR based on the information acquired in the previous process P1 and the information on the appropriate range AR stored in the storage devices 103, 203. Furthermore, the control devices 107, 201 determine whether the minimum distance Dmin is equal to or less than the first threshold value Dth1 stored in the storage devices 103, 203 (determination process P2).

[0107] In this determination process P2, if the control device 107, 201 determines that the minimum distance Dmin is greater than the threshold value Dth1 (NO), i.e., that the avoidance condition is not satisfied, it terminates the force sense presentation process flow KSP in Fig. 8 and repeatedly executes the force sense presentation process flow KSP at a predetermined cycle. On the other hand, in this process P2, if the control device 107, 201 determines that the minimum distance Dmin is equal to or less than the threshold value Dth1 (YES), i.e., that the avoidance condition is satisfied, it executes the next determination process P3.

[0108] In this determination process P3, the control device 107, 201 determines whether the minimum distance Dmin is equal to or less than a second threshold value Dth2 stored in the storage device 103, 203. This second threshold value Dth2 may be, for example, a value smaller than the first threshold value Dth1 used in the initial determination process P2. In this process P3, if the control device 107, 201 determines that the minimum distance Dmin is greater than the threshold value Dth2 (NO), it executes the next force feedback process P4.

[0109] In process P4, the control devices 107 and 201 operate the haptic devices 105 and 205, respectively. Here, the control devices 107 and 201, for example, use the haptic devices 105 and 205 to present a force (operation reaction force) in the opposite direction to the operator's operation direction, which reduces the minimum distance Dmin, via the operation devices 104 and 204. At this time, the control devices 107 and 201 may, for example, fix the operation of the operation devices 104 and 204 using the haptic devices 105 and 205 to prevent the operator from operating them. Furthermore, the control devices 107 and 201, for example, use the haptic devices 105 and 205 to present vibration via the operation devices 104 and 204.

[0110] 7, the operator of the excavator 100S operates the operating devices 104, 204 in the arm opening direction to rotate the arm 5 of the work attachment 101 upward. As a result, the minimum distance Dmin between the bucket 6, which is part of the work attachment 101, and the outer edge of the appropriate range AR becomes equal to or less than the threshold value Dth1.

[0111] Here, the operation device 104 of the excavator 100S that operates the arm 5 includes, for example, a left operation lever in the cabin 10, a left operation lever 104b of the portable controller 104A in Fig. 2, and a stick 104n of the joystick 104B in Fig. 3. Also, the operation device 204 of the remote operation support device 200 that operates the arm 5 includes, for example, a left operation lever installed in a remote operation room, a left operation lever 204b of the remote operation support device 200 in Fig. 2, and a stick 204n of the remote operation support device 200 in Fig. 3.

[0112] In this case, the force sense presentation device 105 of the excavator 100S presents to the operator a force sense and vibration in the direction opposite to the operation direction via, for example, the left operation lever in the cabin 10, the left operation lever 104b of the portable controller 104A, or the stick 104n of the joystick 104B, using the force sense presentation actuator 105b. Furthermore, the force sense presentation device 105 may also fix the operation of the left operation lever in the cabin 10, the left operation lever 104b of the portable controller 104A, or the stick 104n of the joystick 104B, using, for example, the force sense presentation actuator 105b, to prevent inappropriate operation by the operator.

[0113] Similarly, the force feedback device 205 of the remote operation support device 200 uses, for example, the force feedback actuator 205b to provide the operator with a force feedback and vibration in the direction opposite to the operation direction via the left operation lever in the remote operation room, the left operation lever 204b, or the stick 204n of the remote operation support device 200. Furthermore, the force feedback device 205 may also use, for example, the force feedback actuator 205b to fix the operation of the left operation lever in the remote operation room, the left operation lever 204b, or the stick 204n of the remote operation support device 200, thereby preventing the operator from performing inappropriate operations.

[0114] As a result, the operator can recognize that the avoidance condition for the shovel 100S to avoid an accident has been satisfied, that is, that at least a part of the shovel 100S is in the vicinity of the outer edge of the appropriate range AR, through the force sense presented via the operation devices 104, 204. Therefore, the operator can stop operations that would reduce the minimum distance Dmin between the shovel 100S and the appropriate range AR, and prevent the shovel 100S from entering the vicinity of the outer edge of the working range and the vicinity of the outer edge of the movable range.

[0115] Thereafter, the control devices 107, 201 repeat processes P2 to P4 until the minimum distance Dmin between the shovel 100S and the outer edge of the appropriate range AR becomes greater than the threshold value Dth1. Furthermore, in process P3, if the control devices 107, 201 determine that the minimum distance Dmin between the shovel 100S and the appropriate range AR is equal to or less than the second threshold value Dth2 (YES), the control devices 107, 201 execute the next return direction presentation process P5.

[0116] In this process P5, the control devices 107, 201 present to the operator, via the operation devices 104, 204, a force sense in the return direction in which the shovel 100S escapes the avoidance condition. Specifically, the control devices 107, 201 present to the operator, via the operation devices 104, 204, a force sense in the direction in which the shovel 100S returns to the inside of the range near the outer edge of the work range and the range near the outer edge of the movable range by enlarging the minimum distance Dmin from the appropriate range AR beyond the threshold value Dth1.

[0117] More specifically, in the above-mentioned example, the force feedback device 105 of the shovel 100S uses, for example, the force feedback actuator 105b to provide the operator with a force feedback in the arm closing direction via the left operating lever in the cabin 10, the left operating lever 104b of the portable controller 104A, or the stick 104n of the joystick 104B.

[0118] Similarly, the force feedback device 205 of the remote operation support device 200 uses, for example, the force feedback actuator 205b to provide the operator with a force feedback in the arm closing direction via the left operating lever in the remote operation room, the left operating lever 204b of the remote operation support device 200, or the stick 204n.

[0119] This makes it possible to present to an inexperienced operator the appropriate operating direction of the operating device 104, 204. Furthermore, the control devices 107, 201 may, for example, use the force feedback devices 105, 205 to intermittently generate a force feedback in the arm bending direction at a predetermined cycle via the operating device 104 and the operating device 204, thereby presenting the operator with a force feedback like repeated poking. This makes it easier for the operator to recognize the appropriate operating direction of the operating device 104, 204.

[0120] Furthermore, the avoidance conditions, which are conditions for the position and posture that the excavator 100S should avoid, include, for example, fluctuations in the position and posture information due to slippage of the excavator 100S.

[0121] For example, in the example shown in Fig. 7, when the upper rotating body 3 of the excavator 100S is rotated and the bucket 6 of the work attachment 101 is pressed against the excavation surface from the side, a reaction force acts from the excavation surface to the upper rotating body 3 via the work attachment 101. This reaction force may cause the lower traveling body 1 of the excavator 100S to slip on the ground surface ES.

[0122] The control devices 107, 201 detect slippage of the shovel 100S, for example, based on fluctuations in position and orientation information acquired from the position and orientation detection device 102 and information acquired from various sensors of the shovel 100S. When the control devices 107, 201 detect slippage of the shovel 100S, they actuate the force feedback devices 105, 205 to present a force feedback in the slip direction of the shovel 100S to the operator via the operation devices 104, 204.

[0123] Specifically, the control devices 107, 201 actuate the left and right pseudo-force sense presentation actuators 105a, 205a of the force sense presentation devices 105, 205 in the portable controller 104A or remote operation support device 200 shown in Fig. 2 that constitute the operation devices 104, 204. As a result, the left and right pseudo-force sense presentation actuators 105a, 205a present pseudo-force senses in the forward and backward directions, which are opposite to each other, to the operator via the housings 104a, 204a.

[0124] This makes it possible to present to the operator a force sense of right-handedness RHT or left-handedness LHT according to the slip direction of the shovel 100S via the housings 104a, 204a of the portable controller 104A or the remote operation support device 200. As a result, the operator remotely operating the shovel 100S using the portable controller 104A or the remote operation support device 200 can easily recognize slip of the shovel 100S. Note that the pseudo force sense presentation actuators 105a, 205a can similarly make the operator recognize not only slip in the rotational direction of the shovel 100S, but also linear slip.

[0125] Figure 9 is a side view showing another example of an avoidance condition that should be avoided by the excavator 100S of Figure 4. As shown in Figure 9, the excavator 100S, which is an example of the work machine 100 shown in Figure 1, includes a lower traveling body 1 and an upper rotating body 3 that is rotatably mounted on the lower traveling body 1 and has a work attachment 101 attached thereto. In the example shown in Figure 9, the avoidance condition is a state in which the lower traveling body 1 is partially lifted up due to the work attachment 101 applying a force to the ground surface ES, which is the excavation target.

[0126] The control devices 107 and 201 detect a jack-up state in which a part of the undercarriage 1 is lifted up, for example, based on the position and posture information of the excavator 100S including the work attachment 101. The control devices 107 and 201 may detect a jack-up state, for example, using the detection result of a bottom pressure sensor provided in at least one of the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9. Furthermore, the control devices 107 and 201 may detect a jack-up state, for example, using the inclination angle of the ground surface ES recognized based on the image captured by the imaging device C1.

[0127] For example, when the control devices 107, 201 detect a jack-up, they determine that the shovel 100S has satisfied the avoidance conditions, and activate the force feedback device 105 to present the operator with a force feedback corresponding to the lift-up direction FD of the lower running body 1 via the operating devices 104, 204.

[0128] Specifically, the control devices 107, 201 operate the pseudo-force sense presentation actuators 105a, 205a in front of and behind the force sense presentation devices 105, 205, for example, in the portable controller 104A of Figure 2 that constitutes the operation device 104, 204 or the remote operation support device 200.

[0129] As a result, the pseudo-force sense presentation actuators 105a, 205a on the front side of the housings 104a, 204a present an upward pseudo-force sense to the operator via the front portions of the housings 104a, 204a, while the pseudo-force sense presentation actuators 105a, 205a on the rear side of the housings 104a, 204a present a downward pseudo-force sense to the operator via the rear portions of the housings 104a, 204a.

[0130] 2, it is possible to present to the operator a force sensation of an upward direction UT corresponding to the lift-up direction FD of the shovel 100S via the housings 104a, 204a of the portable controller 104A or the remote operation support device 200. As a result, it becomes possible for the operator who remotely operates the shovel 100S using the portable controller 104A or the remote operation support device 200 to easily recognize that the shovel 100S is being jacked up.

[0131] The control devices 107 and 201 also actuate, for example, the joystick 104B of the operation device 104 or 204 shown in FIG. 3 constituting the remote operation support device 200, or the force feedback actuators 105b and 205b of the force feedback devices 105 and 205.

[0132] Then, the force sense presentation actuators 105b, 205b of the force sense presentation devices 105, 205 present a force sense in the direction opposite to the operation direction of the stick 104n, 204n by the operator to the operator via the stick 104n, 204n. At this time, the force sense presentation actuators 105b, 205b may present vibration to the operator via the stick 104n, 204n, for example. Furthermore, the pseudo force sense presentation actuators 105a, 205a of the force sense presentation devices 105, 205 may present a pseudo force sense in the direction opposite to the operation direction of the stick 104n, 204n by the operator to the operator via the stick 104n, 204n.

[0133] This allows an operator operating the shovel 100S using the portable controller 104A or remote operation support device 200 shown in Figure 2, or the joystick 104B or remote operation support device 200 shown in Figure 3, to be presented with a pseudo-force sensation or force sensation in a direction that prevents the shovel 100S from floating up.

[0134] Figure 10 is a side view of a forestry machine 100H as an example of the work machine 100 of Figure 1. The forestry machine 100H shown in Figure 10 differs from the excavator 100S shown in Figure 4 mainly in that the end attachment that makes up the work attachment 101 has been changed from the bucket 6 to a roller harvester device 6H. As the other configuration of the forestry machine 100H shown in Figure 10 is similar to the configuration of the work machine 100 shown in Figure 4, similar parts are given the same reference numerals and description thereof will be omitted.

[0135] Like the excavator 100S, the forestry machine 100H includes a lower traveling body 1 and an upper rotating body 3 rotatably mounted on the lower traveling body 1 and having a work attachment 101 attached thereto. In the forestry machine 100H, for example, a harvester link 6L is attached to the tip of an arm 5 constituting the work attachment 101, and a roller harvester device 6H is attached to the tip of the harvester link 6L. The harvester link 6L is hydraulically driven, for example, by a harvester link cylinder 9H similar to the bucket cylinder 9. A guard is attached to the front of the cabin 10 so as to cover the window portion. The guard can be opened and closed, and is closed during lumber processing work to protect the operator from falling objects such as demolition materials, crushed stone, and branches.

[0136] The roller harvester 6H includes, for example, a gripping unit that grips and holds logs, a feeding unit that feeds the logs (feeds the logs lengthwise), a pruning unit that prunes the logs as they are fed, and a cutting unit that cuts the logs to a desired length. The roller harvester 6H also includes a tilting unit that changes the orientation of the main body (the angle relative to the horizontal plane). Note that "log" refers to the state before it is produced into lumber through lumber processing, and "lumber" refers to the state produced from the logs through lumber processing.

[0137] In this forestry machine 100H, the avoidance condition includes, for example, position and orientation information indicating the possibility that a log will collide with the forestry machine 100H when the work attachment 101 holding a log feeds the log in the lengthwise direction or rotates the log around a rotation axis in the vertical direction. Specifically, in the forestry machine 100H, the avoidance condition is, for example, the position and orientation of the roller harvester 6H that may cause a log held by the roller harvester 6H to collide with the cabin 10 when the roller harvester 6H is fed in the lengthwise direction. In addition, in the forestry machine 100H, the avoidance condition is, for example, the position and orientation of the roller harvester 6H that may cause a log held by the roller harvester 6H to collide with the cabin 10 when the roller harvester 6H is rotated around a rotation axis parallel to the vertical direction.

[0138] The above-mentioned avoidance conditions and the predetermined length of the log are stored in advance, for example, in at least one of the storage device 103 of the forestry machine 100H as the work machine 100 shown in Fig. 1 and the storage device 203 of the remote operation support device 200. Note that the length of the log may be calculated by the control devices 107, 201 or estimated by machine learning using, for example, an image of the log photographed by the imaging device C1 or information about the log acquired by LiDAR (laser radar).

[0139] For example, when the position and orientation information of the work attachment 101 acquired from the position and orientation detection device 102 satisfies the avoidance conditions stored in the storage devices 103 and 203, the control devices 107 and 201 operate the force feedback devices 105 and 205 to provide a force feedback to the operator via the operation devices 104 and 204, corresponding to the direction in which the log is fed or the direction in which the log is rotated.

[0140] Specifically, the control devices 107, 201, for example, operate the force feedback devices 105, 205 to provide a force feedback that vibrates back and forth in the feeding direction of the logs via the operation devices 104, 204. This notifies the operator that there is a risk of the logs gripped by the roller harvester device 6H colliding with the cabin 10 when being fed in the lengthwise direction.

[0141] The control devices 107, 201 may also, for example, operate the force feedback devices 105, 205 to provide a force feedback that vibrates in the rotational direction of the roller harvester 6H via the operation devices 104, 204. This notifies the operator that there is a risk that the logs held by the roller harvester 6H may collide with the cabin 10 when the logs are rotated around a vertical rotation axis. Note that the control devices 107, 201 may, for example, generate a warning sound or provide an audio notification via a sound output device at the same time as providing a force feedback via the operation devices 104, 204.

[0142] As described above, the work machine teleoperation support system SYS of this embodiment comprises a work machine 100 including a work attachment 101, and a teleoperation support device 200 that remotely operates the work machine 100. The work machine teleoperation support system SYS also comprises a position and orientation detection device 102, storage devices 103, 203, operation devices 104, 204, force feedback devices 105, 205, a drive device 106, and control devices 107, 201. The position and orientation detection device 102 detects position and orientation information of the work machine 100. The storage devices 103, 203 store avoidance conditions, which are conditions for the position and orientation that the work machine 100 should avoid. The operation devices 104, 204 generate operation signals in response to operations by the operator of the work machine 100. The force feedback devices 105, 205 present a force feedback to the operator via the operation devices 104, 204. The drive device 106 operates the work machine 100 in response to the operation signal. The control devices 107, 201 operate the force feedback devices 105, 205 when the position and orientation information of the work machine 100 satisfies the avoidance conditions.

[0143] With this configuration, the work machine remote operation support system SYS of this embodiment can operate the force feedback devices 105, 205 using the control devices 107, 201 when the position and attitude information of the work machine 100 satisfies a preset avoidance condition. The force feedback devices 105, 205 can then present a force feedback to the operator via the operation devices 104, 204, thereby notifying the operator that the position and attitude of the work machine 100 satisfies the avoidance condition. Therefore, with the work machine remote operation support system SYS of this embodiment, when the work machine 100 satisfies an avoidance condition, which is a condition for the position and attitude of the work machine 100 to be avoided, the operator operating the operation devices 104, 204 can be prompted to perform an operation to escape the avoidance condition. This can more reliably prevent, for example, the excavator 100S from moving out of the appropriate range AR into the outer area IR, as shown in FIG. 7 . Furthermore, for example, it can more reliably prevent a log gripped by the roller harvester device 6H of the forestry machine 100H shown in FIG. 10 from colliding with the cabin 10. Furthermore, unlike when the work machine 100 is brought to an emergency stop when the work machine 100 meets the avoidance conditions, the operator is not forced to interrupt work or have to take his or her eyes off the work site WS to check the content of the warning displayed on the display device. Therefore, even an inexperienced operator can improve work efficiency. Furthermore, the work machine remote operation support system SYS of this embodiment can determine the avoidance conditions without measuring the external environment using an external sensor. Therefore, the device configuration and data processing can be simplified.

[0144] The work machine 100 of this embodiment also includes a work attachment 101. The work machine 100 also includes a position and orientation detection device 102, a storage device 103, an operation device 104, a force feedback device 105, a drive device 106, and a control device 107. The position and orientation detection device 102 detects position and orientation information of the work machine 100. The storage device 103 stores avoidance conditions, which are conditions for the position and orientation that the work machine 100 must avoid. The operation device 104 generates an operation signal in response to operation by the operator of the work machine 100. The force feedback device 105 presents a force feedback to the operator via the operation device 104. The drive device 106 operates the work machine 100 in response to the operation signal. The control device 107 activates the force feedback device 105 when the position and orientation information of the work machine 100 satisfies the avoidance conditions.

[0145] With this configuration, the work machine 100 of this embodiment can operate the haptic device 105 via the control device 107 when the position and posture information of the work machine 100 satisfies a preset avoidance condition. The haptic device 105 can then present a force to the operator via the operation device 104, notifying them that the position and posture of the work machine 100 meets the avoidance condition. Therefore, with the work machine 100 of this embodiment, when the work machine 100 meets an avoidance condition, which is a condition for the position and posture that should be avoided, the operator operating the operation device 104 can be prompted to perform an operation to escape the avoidance condition. This can more reliably prevent the excavator 100S from moving out of the appropriate range AR into the outer area IR, for example, as shown in FIG. 7 . Furthermore, unlike when the work machine 100 is brought to an emergency stop when the work machine 100 meets the avoidance condition, the operator is not forced to interrupt work or take his or her eyes off the work site WS to check the content of a warning displayed on the display device. Therefore, even an inexperienced operator can improve work efficiency. Furthermore, according to the work machine 100 of this embodiment, it is possible to determine the avoidance conditions without measuring the external environment using an external sensor, thereby simplifying the device configuration and data processing.

[0146] Furthermore, in the work machine 100 of this embodiment, the operation device 104 is, for example, a portable controller 104A shown in Fig. 2. The force feedback device 105 includes a pseudo force feedback actuator 105a that is mounted on the portable controller 104A and that provides a pseudo force feedback in a predetermined direction.

[0147] With this configuration, the portable controller 104A can be used at the work site WS where the work machine 100 performs work to operate the work machine 100 from inside the cabin 10 of the work machine 100, or to remotely operate the work machine 100 from outside the cabin 10. Furthermore, it is possible to generate pseudo-force vibrations in any direction in the pseudo-force sense display actuator 105a and present a pseudo-force sense in any direction to the operator via the housing 104a. As a result, as shown in FIG. 2 , it is possible to present to the operator via the housing 104a of the portable controller 104A a pseudo-force sense in the clockwise direction RHT, counterclockwise direction LHT, upward direction UT, downward direction DT, forward / backward direction, left / right direction, or up / down direction, depending on the avoidance condition satisfied by the work machine 100.

[0148] Furthermore, in the work machine remote operation support system SYS of this embodiment, the remote operation support device 200 can be configured as a portable controller, as shown in Fig. 2. The force feedback device 205 includes a pseudo force feedback actuator 205a that is mounted on the remote operation support device 200 as a portable controller and that provides a pseudo force feedback in a predetermined direction.

[0149] With this configuration, the work machine 100 can be remotely operated using the remote operation support device 200 as a portable controller in a remote location away from the work site WS where the work machine 100 performs work. Furthermore, pseudo force sense vibrations in any direction can be generated in the pseudo force sense presentation actuator 205a, and a pseudo force sense in any direction can be presented to the operator via the housing 204a. As a result, as shown in FIG. 2 , a pseudo force sense in the right direction RHT, left direction LHT, upward direction UT, downward direction DT, front-to-back direction, left-to-right direction, or up-and-down direction can be presented to the operator via the housing 204a, depending on the avoidance condition satisfied by the work machine 100.

[0150] Furthermore, in the work machine 100 of this embodiment, the operation device 104 is a joystick 104B. The force feedback device 105 includes a force feedback actuator 105b that provides a force feedback in the opposite direction to the operation direction of the joystick 104B by the operator of the work machine 100, or a force feedback in a direction that will escape the avoidance condition.

[0151] With this configuration, the joystick 104B can be used at the work site WS where the work machine 100 performs work to operate the work machine 100 from inside the cabin 10 of the work machine 100, or to remotely operate the work machine 100 from outside the cabin 10. Furthermore, the force sense display actuator 105b can present a force sense in any direction to the operator via the stick 104n of the joystick 104B. This makes it possible to present a force sense in the opposite direction to the operator's operation direction or a force sense in a direction that will escape the avoidance condition to the operator via the joystick 104B, and to encourage the operator to perform an operation that will escape the avoidance condition.

[0152] Furthermore, in the work machine remote operation support system SYS of this embodiment, the remote operation support device 200 may be configured as a joystick as shown in Fig. 3. Furthermore, the force feedback device 205 includes a force feedback actuator 205b that provides a force feedback in the opposite direction to the operation direction of the stick 204n by the operator remotely operating the work machine 100, or a force feedback in a direction that will escape the avoidance condition.

[0153] With this configuration, the work machine 100 can be remotely operated using the remote operation support device 200 as a joystick in a remote location away from the work site WS where the work machine 100 performs work. Furthermore, the force sense display actuator 205b can present a force sense in any direction to the operator via the stick 204n of the operation device 204. This makes it possible to present a force sense in the opposite direction to the operator's operation direction or a force sense in a direction that will escape the avoidance condition to the operator in the remote location via the operation device 204 of the remote operation support device 200, and to encourage the operator to perform an operation that will escape the avoidance condition.

[0154] Furthermore, in the work machine 100 and work machine remote operation support system SYS of this embodiment, the avoidance conditions include at least a part of the work machine 100, including the work attachment 101, being within a range close to the outer edge of the work range of the work machine 100, or within a range close to the outer edge of the movable range of the work machine 100.

[0155] With this configuration, for example, as shown in Figure 7, when at least a portion of the work machine 100 enters a range near the appropriate range AR that indicates the outer edge of the work range or the outer edge of the movable range of the work machine 100, the control device 107, 201 activates the force feedback device 105, 205. As a result, a force feedback is presented to the operator via the operation device 104, 204. This enables the operator to recognize that at least a portion of the work machine 100 is in a range near the outer edge of the work range of the work machine 100 or in a range near the outer edge of the movable range of the work machine 100.

[0156] Therefore, the operator can operate the operation devices 104, 204 in a direction that will cause the work machine 100 to escape the avoidance condition, that is, in a direction that will cause the work machine 100 to move away from the outer edge of the appropriate range AR, which indicates the outer edge of the working range and the outer edge of the movable range. Also, unlike when the work machine 100 is brought to an emergency stop when the work machine 100 exceeds the working range or the movable range, the operator is not forced to interrupt work or need to take his or her eyes off the work site WS to check the content of the warning displayed on the display device. Therefore, even an inexperienced operator can improve work efficiency.

[0157] Furthermore, in the work machine 100 and work machine remote operation support system SYS of this embodiment, the avoidance conditions include fluctuations in position and attitude information due to slippage of the work machine 100. Furthermore, the control devices 107, 201 operate the force feedback devices 105, 205 to present a force feedback in the slip direction of the work machine 100 to the operator via the operation devices 104, 204.

[0158] With this configuration, a force sensation corresponding to the slip direction of the work machine 100 can be presented to the operator of the work machine 100 via the housing 104a of the portable controller 104A and the housing 204a of the remote operation support device 200 as a portable controller shown in Figure 2. This makes it possible for the operator remotely operating the work machine 100 using the portable controller 104A or the remote operation support device 200 to easily recognize that the work machine 100 has slipped. As a result, the operator remotely operating the work machine 100 can perform an operation to return the work machine 100 from a slipping state to the state it was in before the slip.

[0159] 4 to 9 , the work machine 100 of this embodiment is, for example, an excavator 100S including a lower traveling body 1 and an upper rotating body 3 rotatably mounted on the lower traveling body 1 and having a work attachment 101 attached thereto. The avoidance conditions, which are conditions for the position and posture that the excavator 100S must avoid, include a state in which a part of the lower traveling body 1 is lifted up due to the work attachment 101 applying a force to an excavation target. The control devices 107, 201 operate the force feedback devices 105, 205 to present a force corresponding to the direction in which the lower traveling body 1 is lifted up to the operator via the operation devices 104, 204.

[0160] With this configuration, for example, it is possible to present to the operator a force sense of an upward UT corresponding to the lift-up direction of the shovel 100S via the portable controller 104A or the housings 104a, 204a of the remote operation support device 200 shown in Fig. 2. Furthermore, it is possible to present to the operator a force sense in the direction opposite to the operation direction of the operator via the joystick 104B or the sticks 104n, 204n of the remote operation support device 20 shown in Fig. 3. As a result, it is possible to make the operator remotely operating the work machine 100 aware of the lift-up of the undercarriage 1, and to urge him or her to operate in a direction that will escape the avoidance condition.

[0161] 10 , the work machine 100 is, for example, a forestry machine 100H that includes a lower traveling body 1 and an upper rotating body 3 that is rotatably mounted on the lower traveling body 1 and has a work attachment 101 attached thereto. The avoidance conditions that the forestry machine 100H must avoid include position and posture information that indicates the possibility of the log colliding with the forestry machine 100H when the work attachment 101 holding the log feeds the log in the lengthwise direction or rotates the log about a vertical rotation axis. The control devices 107, 201 then operate the force feedback devices 105, 205 to present a force corresponding to the log feed direction or log rotation direction to the operator via the operation devices 104, 204.

[0162] With this configuration, for example, a force sense corresponding to the feed direction or rotation direction of the log gripped by the work attachment 101 can be presented to the operator via the portable controller 104A shown in Fig. 2 or the housings 104a, 204a of the remote operation support device 200. Also, for example, a force sense corresponding to the feed direction or rotation direction of the log gripped by the work attachment 101 can be presented to the operator via the joystick 104B shown in Fig. 3 or the sticks 104n, 204n of the remote operation support device 20. As a result, the operator of the work machine 100 can be notified of the risk of a collision between the log and the forestry machine 100H, and can be prompted to take action to avoid the collision.

[0163] As described above, according to this embodiment, it is possible to provide a work machine 100 and a work machine remote operation support system SYS that are capable of notifying the operator of the work machine 100 equipped with a work attachment 101 of a situation that must be avoided by the work machine 100 by presenting a force sense, regardless of the presence or absence of an obstacle.

[0164] The preferred embodiments of the present invention have been described above in detail. However, the present invention is not limited to the above-described embodiments. Various modifications or substitutions may be applied to the above-described embodiments without departing from the scope of the present invention. Furthermore, features described separately may be combined unless technical contradictions arise.

[0165] In the above-described embodiment, an example was described in which the work machine was a shovel, but the work machine according to the present disclosure is not limited to a shovel. If the work machine is a crane, for example, boom sway exceeding a predetermined amplitude can be set as the avoidance condition. This makes it possible for the force feedback device to present a force sensation corresponding to the boom sway to the operator via the operation device.

[0166] Furthermore, if the work machine is a crane, the avoidance conditions can be set to, for example, conditions that may cause the crane to tip over. Specifically, the avoidance conditions can include, for example, conditions regarding the tilt angle of the upper rotating body or the traveling body relative to the horizontal plane and the attitude of the boom and jib as work attachments. This allows an inexperienced crane operator to intuitively perceive the possibility of the crane tipping over and determine the appropriate operation.

[0167] Furthermore, for example, if the work machine is a shovel, crane, wheel loader, or forklift, an overload exceeding the rated load of the work attachment can be set as an avoidance condition. This makes it possible to make the operator aware of the overload by, for example, presenting a force sensation of being pushed down or pulled forward to the operator via the operating device using the force feedback device.

[0168] This application claims priority based on Japanese Patent Application No. 2023-206066, filed on December 6, 2023, the entire contents of which are incorporated herein by reference.

[0169] REFERENCE SIGNS LIST 1 Lower traveling body 3 Upper rotating body 100 Work machine 100S Excavator 101 Work attachment 102 Position and attitude detection device 103 Storage device 104 Operation device 104A Portable controller 104B Joystick 105 Force sense presentation device 105a Pseudo force sense presentation actuator 105b Force sense presentation actuator 106 Drive device 107 Control device 200 Teleoperation support device 201 Control device 203 Storage device 204 Operation device 205 Force sense presentation device FD Lift-up direction SYS Teleoperation support system for work machine

Claims

1. A work machine equipped with a work attachment, comprising: a position and attitude detection device that detects position and attitude information of the work machine; a memory device that stores avoidance conditions, which are conditions for the position and attitude of the work machine to avoid; an operation device that generates an operation signal in response to operation of an operator of the work machine; a force feedback device that presents a force feedback to the operator via the operation device; a drive device that operates the work machine in response to the operation signal; and a control device that activates the force feedback device when the position and attitude information satisfies the avoidance conditions.

2. A work machine as described in claim 1, wherein the operation device is a portable controller, and the force feedback device includes a pseudo force feedback actuator mounted on the portable controller for providing a pseudo force feedback in a predetermined direction.

3. A work machine as described in claim 1, wherein the operation device is a joystick, and the force feedback device includes a force feedback actuator that presents a force feedback in a direction opposite to the direction in which the operator operates the joystick or a force feedback in a direction that escapes the avoidance condition.

4. A work machine according to claim 1, wherein the avoidance condition includes that at least a part of the work machine including the work attachment is within a range adjacent to an outer edge of a working range of the work machine, or within a range adjacent to an outer edge of a movable range of the work machine.

5. A work machine as described in claim 2, wherein the avoidance condition includes a change in the position and attitude information due to slippage of the work machine, and the control device activates the force feedback device to present a force feedback in the slip direction of the work machine to the operator via the operation device.

6. The work machine is a shovel comprising a lower running body and an upper rotating body rotatably mounted on the lower running body and having the work attachment attached thereto, the avoidance condition includes a state in which a part of the lower running body is lifted up due to the work attachment applying a force to an excavation target, and the control device operates the force feedback device to present a force corresponding to the lifting direction of the lower running body to the operator via the operating device. The work machine of claim 2 or claim 3.

7. The work machine according to claim 2 or claim 3, wherein the work machine is a forestry machine comprising a lower running body and an upper rotating body rotatably mounted on the lower running body and to which the work attachment is attached, the avoidance condition includes the position and posture information indicating that the log may collide with the forestry machine when the work attachment holding the log feeds the log in the lengthwise direction or rotates the log around a vertical rotation axis, and the control device operates the force feedback device to present to the operator a force feedback corresponding to the feed direction of the log or the rotation direction of the log via the operating device.

8. A remote operation support system for a work machine comprising a work machine including a work attachment and a remote operation support device for remotely operating the work machine, comprising: a position and attitude detection device for detecting position and attitude information of the work machine; a storage device in which avoidance conditions, which are conditions for the position and attitude that the work machine must avoid, are stored; an operation device for generating an operation signal in response to operation of an operator of the work machine; a force feedback device for presenting a force feedback to the operator via the operation device; a drive device for operating the work machine in response to the operation signal; and a control device for actuating the force feedback device when the position and attitude information satisfies the avoidance conditions.

Images