Working machinery

Abstract

To provide a work machine which can enhance the skill of an operator.SOLUTION: A work machine includes: a sight line detection device for detecting a sight line of an operator operating an operation device; an object detection device for detecting an object 70 around the work machine; a reaction force device capable of applying a reaction force that is a force opposite to the operation force to the operation device, to the operation device; calculation means for calculating a recognition degree of the operator for the object 70 detected by the object detection device, on the basis of the detection result of the sight line detection device; reaction force applying means for applying the reaction force to the operation device from the reaction force device, when the operation device is operated so that at least a part of the work machine is moved to the object 70 detected by the object detection device; and reaction force change means for changing the magnitude of the reaction force applied by the reaction force device, according to the recognition degree calculated by the calculation means.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a work machine capable of applying a reaction force to an operating device operated by an operator. [Background technology]

[0002] Patent Document 1 discloses setting a gaze area based on the frequency distribution of gaze points that an operator (operator) of a work machine focuses on.

[0003] Furthermore, Patent Document 2 discloses a method for stopping the operation of a work machine when the operator (driver) operating the work machine is distracted while driving. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-133229 [Patent Document 2] Japanese Patent Publication No. 2017-53092 [Overview of the project] [Problems that the invention aims to solve]

[0005] Incidentally, when an operator operates an operating device to move at least a part of a work machine toward an object of which the operator has low awareness, restricting such an operation makes the operator aware that the operator has performed such an operation, which is effective for improving the operator's skill. However, in Patent Document 1, concentration of the gazing point does not occur on fluid objects such as people and vehicles in a work site that the operator does not gaze at more than a certain level. Therefore, it is impossible to grasp the operator's awareness of such objects. Further, as in Patent Document 2, simply stopping the operation of the work machine when the operator is driving while looking away cannot make the operator understand why the operation of the work machine has stopped or what kind of operation was wrong. Therefore, it is impossible to improve the operator's skill.

[0006] An object of the present invention is to provide a work machine capable of improving the operator's skill. [Means for Solving the Problems]

[0007] The present invention provides a work machine including an operating device operated by an operator, comprising: a line-of-sight detection device that detects the line of sight of the operator; an object detection device that detects an object present around the work machine; a reaction force device capable of applying a reaction force, which is a force opposite to an operation force applied to the operating device, to the operating device; a calculation means that calculates the awareness of the operator with respect to the object detected by the object detection device based on the detection result of the line-of-sight detection device; a reaction force applying means that causes the reaction force device to apply the reaction force to the operating device when the operating device is operated to move at least a part of the work machine toward the object detected by the object detection device; and a reaction force changing means that changes the magnitude of the reaction force applied by the reaction force device according to the awareness calculated by the calculation means.

[0008] Furthermore, the present invention relates to a work machine equipped with an operating device operated by an operator, comprising: a gaze detection device for detecting the operator's line of sight; a reaction force device capable of applying a reaction force to the operating device, which is a force opposite to the operating force applied to the operating device; a calculation means for calculating the distribution of the operator's awareness of the surroundings of the work machine based on the detection result of the gaze detection device; a reaction force application means for causing the reaction force to be applied from the reaction force device to the operating device when the operating device is operated; and a reaction force changing means for changing the magnitude of the reaction force applied by the reaction force device according to the distribution of awareness calculated by the calculation means. [Effects of the Invention]

[0009] The present invention provides a system in which the magnitude of the reaction force applied from a reaction force device to an operating device changes according to the operator's awareness of an object that at least a part of the work machine is approaching. Therefore, information regarding the operator's awareness can be conveyed to the operator through the reaction force. For example, the less the operator is aware of an object, the greater the reaction force applied to the operation toward that object. Also, for example, by reducing (preferably making zero) the reaction force applied to an object that the operator is aware of, the workability of the work machine can be prevented from decreasing. Furthermore, the reaction force applied to the operating device can make the operator aware of their lack of attention to the object. As a result, when the operator is operating the work machine to bring a part of it closer to an object, they will be more attentive to the object. Therefore, the operator's skill can be improved.

[0010] Furthermore, the present invention also provides a system in which the magnitude of the reaction force applied from the reaction force device to the operating device changes according to the distribution of the operator's awareness of the surroundings of the work machine. Therefore, information regarding the operator's awareness can be conveyed to the operator through the reaction force. For example, if the operator's awareness of the area around the work machine is low, the reaction force for operations toward the area with low awareness can be increased. Also, for example, by reducing (preferably making zero) the reaction force for operations toward the area with high awareness of the operator, the workability of the work machine can be prevented from decreasing. The reaction force to the operating force applied to the operating device can make the operator aware of insufficient attention to the direction in which the work machine is moving. As a result, when the operator moves a part of the work machine, they will pay sufficient attention to the direction of movement. Therefore, the operator's skill can be improved. [Brief explanation of the drawing]

[0011] [Figure 1] This is a side view of the work machine. [Figure 2] This is a functional configuration diagram of a work machine. [Figure 3] This diagram shows objects in the view from inside the cab, as well as the operator's point of focus. [Figure 4] This diagram shows the level of awareness for each point of focus. [Figure 5] This is a diagram showing the distribution of cognitive levels. [Figure 6] This diagram shows how to vary the speed and intensity of the reaction force application. [Figure 7] This diagram shows the range of motion of the attachment. [Figure 8] This figure is a report summarizing multiple situations when a reaction force is applied. [Figure 9] This diagram shows the operating area of ​​the attachment and the trajectory of its movement. [Figure 10] This figure shows the relationship between the variance and the adjustment value. [Figure 11] This figure shows the relationship between the normally distributed values ​​and the adjusted values. [Figure 12] This figure shows the relationship between normal distribution values ​​and thresholds. [Modes for carrying out the invention]

[0012] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0013] (Configuration of the work machine) The working machine according to an embodiment of the present invention is capable of applying a reaction force to an operating device operated by an operator. As shown in Figure 1, a side view of the working machine 1, the working machine 1 is a machine that performs work with an attachment 30, and is, for example, a hydraulic excavator. The working machine 1 has a machine body 25 equipped with a lower traveling body 21 and an upper rotating body 22, an attachment 30, and a cylinder 40.

[0014] The lower traveling body 21 is the part on which the work machine 1 travels, and is equipped with, for example, crawler tracks. The upper slewing body 22 is rotatably mounted on the upper part of the lower traveling body 21 via a slewing device 24. A cab (operator's cabin) 23 is provided at the front of the upper slewing body 22.

[0015] Attachment 30 is mounted on the upper slewing body 22 so as to be rotatable in the vertical direction. Attachment 30 comprises a boom 31, an arm 32, and a bucket 33. The boom 31 is mounted on the upper slewing body 22 so as to be rotatable (launchable) in the vertical direction. The arm 32 is mounted on the boom 31 so as to be rotatable in the vertical direction. The bucket 33 is the tip attachment of attachment 30 and is mounted on the arm 32 so as to be rotatable in the front-rear direction. The bucket 33 is the part that performs tasks such as excavating, leveling, and scooping soil and sand. Note that the work object held by the bucket 33 is not limited to soil and sand, but may also be stones or waste (industrial waste, etc.). Furthermore, the tip attachment is not limited to a bucket 33, but may also be a grapple or a lifting magnet, etc.

[0016] The cylinder 40 is capable of hydraulically rotating the attachment 30. The cylinder 40 is a hydraulic telescopic cylinder. The cylinder 40 comprises a boom cylinder 41, an arm cylinder 42, and a bucket cylinder 43.

[0017] The boom cylinder 41 rotates the boom 31 relative to the upper slewing body 22. The base end of the boom cylinder 41 is rotatably attached to the upper slewing body 22. The tip end of the boom cylinder 41 is rotatably attached to the boom 31.

[0018] The arm cylinder 42 rotates the arm 32 relative to the boom 31. The base end of the arm cylinder 42 is rotatably attached to the boom 31. The tip end of the arm cylinder 42 is rotatably attached to the arm 32.

[0019] The bucket cylinder 43 rotates the bucket 33 relative to the arm 32. The base end of the bucket cylinder 43 is rotatably attached to the arm 32. The tip end of the bucket cylinder 43 is rotatably attached to a link member 34, which is rotatably attached to the bucket 33.

[0020] Furthermore, the work machine 1 is equipped with an indoor camera 26 (see Figure 2). The indoor camera (gaze detection device) 26 is installed inside the cab 23. The indoor camera 26 detects the operator's gaze by capturing images of the operator's eyes when operating the work machine 1. Note that the gaze detection device for detecting the operator's gaze is not limited to the indoor camera 26; for example, it may be a wearable gaze detection device attached to the operator's head.

[0021] Furthermore, the work machine 1 includes a camera 27 and a LiDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) 28. The camera 27 and LiDAR 28 are mounted on the upper slewing body 22, but may also be mounted on the attachment 30 (especially the boom 31) or the cab 23. The camera (object detection device) 27 and LiDAR (object detection device) 28 detect objects around the work machine 1. Specifically, the camera 27 is fixed in a shooting direction such that its shooting range is located at least in front of the cab 23, and the LiDAR 28 is fixed in a scanning direction such that its scan range is located at least in front of the cab 23. The camera 27 acquires images of the shooting range, and the LiDAR 28 acquires point cloud data (three-dimensional point cloud) indicating the distance from the position where the LiDAR 28 is mounted to the object. Then, by analyzing the images captured by camera 27 (for example, by determining the shape of objects captured in the images), the presence or absence of objects and the type of objects (for example, dump trucks or fences) are identified. In addition, the position of objects is determined from the point cloud data acquired by LiDAR 28, relative to the position where LiDAR 28 is mounted.

[0022] Furthermore, the presence and location of an object may be determined from the position and size of the object in the image captured by the camera 27, or the presence and type of an object may be determined from the point cloud data acquired by the LiDAR 28, and it is not necessary for either the camera 27 or the LiDAR 28 to be installed.In this embodiment, the camera 27 acquires visible light images, but an infrared camera that acquires infrared night vision images may be used as the camera 27 so that it can function even at night.The LiDAR 28 only needs to acquire point cloud data (three-dimensional point cloud) indicating the distance to an object, and may be replaced with a TOF (Time of Flight) sensor or an ultrasonic sensor.

[0023] Furthermore, the work machine 1 has an operating lever 29 (see Figure 2). The operating lever (operating device) 29 is operated by an operator. By operating the operating lever 29, the lower traveling body 21 moves forward and backward, the upper rotating body 22 rotates, the attachment 30 rotates, etc.

[0024] Furthermore, the work machine 1 may be configured to be remotely controlled from outside the work machine 1. In this case, the view shown in Figure 3 is displayed on a monitor outside the work machine 1, and the operator remotely controls the work machine 1 while looking at the monitor. In addition, a gaze detection device is provided on the outside of the work machine 1 to detect the gaze of the operator remotely controlling the work machine 1.

[0025] (Functional configuration of the work machine) As shown in Figure 2, a functional configuration diagram of the work machine 1, the work machine 1 includes a controller 11, a storage device 13, and a reaction force device 14.

[0026] The controller 11 receives images captured by the indoor camera 26, images captured by camera 27, and point cloud data acquired by LiDAR 28 as input.

[0027] Furthermore, the controller 11 receives input for the amount of movement of the operating lever 29. The amount of movement of the operating lever 29 is the amount of displacement of the operating lever 29 from its neutral position.

[0028] The reaction force device 14 can apply a reaction force to the operating lever 29 that is in the opposite direction to the operating force applied to the operating lever 29. The reaction force device 14 applies a reaction force in the direction that returns the operating lever 29 to the neutral position when the operating lever 29 is displaced away from the neutral position. The reaction force device 14 applies the reaction force to the operating lever 29 by means of, for example, an electromagnetic force.

[0029] The following explanation will use the rotation operation of the upper rotating body 22 as an example, but this embodiment can also be applied when the lower traveling body 21 moves forward or backward, or when the attachment 30 is rotated up and down.

[0030] The controller (calculation means) 11 calculates the operator's awareness of the objects detected by the camera 27 and LiDAR 28 based on the detection results of the indoor camera 26. The detection results of the indoor camera 26 refer to the detection results of the operator's line of sight.

[0031] The operator's level of awareness is calculated as follows. First, the coordinates of the operator's gaze points and the gaze time Δt for each gaze point are calculated at a predetermined sampling interval. Figure 3 shows the object 70 and the operator's gaze points in the field of view looking forward from inside the cab 23. In Figure 3, the coordinates P1 to P4 for each of the four gaze points for the two objects 70 are calculated. Each gaze point is the coordinate of the two objects 70 that the operator has gazed at from a predetermined time ago to the current sampling. In Figure 3, the gaze point was at coordinate P4 in the sampling a predetermined time ago, and it is shown that the gaze point changed from coordinate P4 to coordinate P3, then from coordinate P3 to coordinate P2, and then from coordinate P2 to the current gaze point, coordinate P1. The coordinates P1 to P4 for each gaze point are, for example, two-dimensional coordinates (x,y). x is the coordinate in the left-right direction in Figure 3, and y is the coordinate in the depth direction of the paper in Figure 3. Below, the coordinate P i (x i ,y i )

[0032] Next, cognitive level C for each point of focus i However, it is calculated by the following equation (1). Here, K is a coefficient for the gaze time Δt, which adjusts the cognitive sensitivity per unit gaze time. Cognitive level C for each gaze point i This is shown in Figure 4. The horizontal axis in Figure 4 represents the range of motion of the rotation of the upper rotating body 22, and corresponds to the left-right coordinate x of each point of focus. The vertical axis in Figure 4 represents the cognitive level. In Figure 4, the area to the right of the vertical axis is the rightward direction of the work machine 1, and the area to the left of the vertical axis is the leftward direction of the work machine 1. Cognitive level C i This information is updated after a certain period of time has elapsed, for example, at predetermined sampling intervals. C i =K×Δt...Equation (1)

[0033] Next, the recognition level C i is reset as a recognition range C i (x i , y i )_ Lev _ i based on a normal distribution centered on coordinates P 2 , where μ represents the mean and σ C Lev _ i = 1 / (2πσ 2 ) 1 / 2 EXP(-(P i -μ) 2 / 2σ 2 )×C i ···Equation (2)

[0034] Next, at each gaze point, with the mean μ set to zero and the variance σ 2 set to an arbitrary value, the recognition range C i (x i , y i )_ Lev _ i is calculated centered on coordinates P Lev _ i . From the sum of each recognition range C Lev , a recognition level distribution C Lev is calculated as the operator's recognition degree. The recognition level distribution C calculated in this manner is shown in Fig. 5.

[0035] When the operation lever 29 is operated to move at least a part of the work machine 1 toward the object 70 detected by the camera 27 and the LiDAR 28, the controller (reaction force applying means) 11 causes the reaction force device 14 to apply a reaction force to the operation lever 29. At this time, the controller (reaction force changing means) 11 changes the magnitude of the reaction force applied from the reaction force device 14 to the operation lever 29 in accordance with the recognition degree calculated by the controller itself. Specifically, the magnitude of the reaction force F applied from the reaction force device 14 is determined by the following equation (3) in accordance with the recognition level distribution C Lev (the operator's recognition degree). F = 1 / exp(C Lev ) ···Equation (3)

[0036] In this way, the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 changes according to the operator's awareness of the object 70 that at least a part of the work machine 1 is approaching. Therefore, information about the operator's awareness can be conveyed to the operator through the reaction force. For example, the less the operator is aware of the object 70, the greater the reaction force can be applied to the operation toward that object 70. Also, for example, by reducing (preferably making zero) the reaction force applied to the operation toward an object 70 that the operator is aware of, the workability of the work machine 1 can be prevented from decreasing. Furthermore, the reaction force in response to the operating force on the operating lever 29 can make the operator aware of their lack of attention to the object 70. As a result, when the operator is operating the work machine 1 to bring a part of it closer to the object 70, they will be able to operate it with sufficient attention to the object 70. Therefore, the operator's skill can be improved.

[0037] In this embodiment, as shown in equation (3), the lower the operator's awareness of the object 70, the greater the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29. Conversely, the higher the operator's awareness of the object 70, the smaller the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29.

[0038] Thus, the lower the operator's awareness of object 70, the greater the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29. This effectively alerts the operator to their lack of attention to object 70. Conversely, the higher the operator's awareness of object 70, the smaller the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29. This prevents the operation of the operating lever 29 from being restricted by the reaction force, thus preventing a decrease in the work efficiency of the work machine 1.

[0039] As described above, in this embodiment, cognitive level distribution C LevThe force is calculated for the left-right direction of the work machine 1, and when the work machine 1 is operated in such a way that at least a part of it approaches the object 70, a reaction force is applied from the reaction force device 14 to the operating lever 29 according to the operator's awareness of the object 70 that at least a part of the work machine 1 is approaching. Operations that cause at least a part of the work machine 1 to approach the object 70 include operations that cause the attachment 30 or the upper slewing body 22 to move in the left-right direction of the work machine 1, such as the slewing operation of the upper slewing body 22 or the travel operation of the lower traveling body 21.

[0040] Furthermore, in this embodiment, the cognitive level distribution C Lev The force is calculated in the front-rear direction of the work machine 1 (the depth direction in Figure 3), and when an operation is performed that causes at least a part of the work machine 1 to approach the object 70, a reaction force is applied from the reaction force device 14 to the operating lever 29 according to the operator's awareness of the object 70 that at least a part of the work machine 1 is approaching. An operation that causes at least a part of the work machine 1 to approach the object 70 includes operations that move the attachment 30 or the upper slewing body 22 in the forward direction of the work machine 1, such as operations of the attachment 30 (operations that rotate the boom 31, operations that rotate the arm 32, operations that rotate the bucket 33) or operations that move the lower traveling body 21.

[0041] Here, the controller (reaction force mode means) 11 may change the mode of application, which is the mode of temporal change in the magnitude of the reaction force, when applying a reaction force from the reaction force device 14 to the operating lever 29. For example, this mode of application may be a mode in which the speed of application of the reaction force is varied, or a mode in which the reaction force is applied while vibrating.

[0042] Figure 6 shows a method for varying the speed of reaction force application. As shown in Figure 6, varying the speed of reaction force application means using a high-response method, which applies the target reaction force over a relatively short period of time, and a low-response method, which applies the target reaction force over a relatively long period of time, thereby applying the reaction force rapidly or gradually.

[0043] The method of applying a reaction force while oscillating means alternating between applying a target reaction force and reducing the reaction force to zero.

[0044] In the case where the speed of applying the reaction force is varied, for example, the shorter the distance between the work machine 1 and the object 70, the more likely it is that the operator will interfere with the object 70 if the reaction force is applied too quickly. Also, in the case where the reaction force is applied while vibrating, for example, the shorter the distance between the work machine 1 and the object 70, the more likely it is that the operator will interfere with the object 70 if the reaction force is vibrated too much. By changing the manner in which the reaction force is applied in this way, it is possible to make the operator more clearly aware of their lack of attention to the object 70.

[0045] Furthermore, the controller (reaction force moderation means) 11 may change the manner in which it applies the reaction force from the reaction force device 14 to the operating lever 29 according to the level of awareness it has calculated. For example, in the case of varying the speed of application of the reaction force, the reaction force can be applied more rapidly to objects 70 that the operator is less aware of, while the reaction force can be applied more gradually to objects 70 that the operator is more aware of. Also, for example, in the case of applying the reaction force while vibrating, the vibration can be increased to objects 70 that the operator is less aware of, while the vibration can be decreased to objects 70 that the operator is more aware of. In this way, by changing the manner in which the reaction force is applied according to the operator's level of awareness of the object 70, it is possible to make the operator more clearly aware of when their attention to the object 70 is insufficient.

[0046] Furthermore, as shown in Figure 3, the controller (reaction force changing means) 11 may change the magnitude of the reaction force according to the distance L(L1,L2) between the object 70 and the attachment 30 (bucket 33). For example, as shown in equation (4) below, the shorter the distance between the object 70 and the bucket 33, the larger the reaction force may be for the operation of moving the attachment 30 toward the object 70. F=(1 / exp(C Lev ))×1 / L...Equation (4)

[0047] In this way, the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 changes depending on the distance between the object 70 and the attachment 30. For example, the shorter the distance between the object 70 and the attachment 30, the greater the reaction force applied to the operation toward the object 70. In this case, the operator can be made aware that there is a high possibility that the attachment 30 will interfere with the object 70.

[0048] If the LiDAR 28 does not detect the object 70, the controller (reaction force applying means) 11 will not apply a reaction force from the reaction force device 14 to the operating lever 29. However, even if the LiDAR 28 does not detect the object 70, the reaction force device 14 may still apply a reaction force to the operating lever 29.

[0049] Specifically, in Figure 3, the coordinates the operator had been fixating on object 70 were set as the fixation point. However, regardless of whether object 70 is present or not, the coordinates the operator has been fixating on are set as the fixation point, and the cognitive level C for each fixation point is set. i The cognitive level distribution C is calculated and Lev By resetting the settings, the distribution of the operator's awareness in the area surrounding the work machine 1 is calculated. Then, the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 is changed according to the operator's awareness of the point of focus. For example, if the operator's awareness of the area surrounding the work machine 1 in which at least a part of the work machine 1 is approaching is high, the magnitude of the reaction force applied to the operating lever 29 is set to be small, and if the awareness is low, the magnitude of the reaction force applied to the operating lever 29 is set to be large. With this configuration, even when no object 70 is present, operation in the direction of low awareness is restricted, and the operator can be made aware that they are not paying enough attention to the direction in which the work machine 1 is moving (they are driving while distracted).

[0050] Here, the controller (range setting means) 11 sets the movable range 80 of the attachment 30, as shown in Figure 7. The top of Figure 7 is the front of the upper rotating body 22, and the left and right directions in Figure 7 are the left and right directions of the upper rotating body 22.

[0051] In Figure 7, there is an object 70 located outside the movable range 80 of the attachment 30. The attachment 30 is unlikely to reach the object 70 located outside the movable range 80. Therefore, the possibility of the attachment 30 interfering with such an object 70 is low. Accordingly, when the operating lever 29 is operated to move the attachment 30 toward the object 70 located outside the movable range 80, the controller (reaction force applying means) 11 does not apply a reaction force from the reaction force device 14 to the operating lever 29. This prevents the workability of the work machine 1 from being unnecessarily reduced.

[0052] Furthermore, in Figure 7, the controller (calculation means) 11 does not need to calculate the recognition level for objects 70 located outside the movable range 80. This reduces the computational load on the controller 11.

[0053] Returning to Figure 2, the controller 11 stores in the memory device 13 the conditions relating to at least one of the working machine 1 and the object 70 when it applies a reaction force to the reaction force device 14. Specifically, the conditions when a reaction force is applied to the reaction force device 14 include the time the reaction force was applied, the operation performed when the reaction force was applied, the posture of the working machine 1 when the reaction force was applied, the magnitude of the reaction force, and the type of object 70. The posture of the working machine 1 is, for example, the horizontal distance between the upper rotating body 22 and the bucket 33. After the work is completed, the controller (output device) 11 outputs a report summarizing the multiple conditions stored in the memory device 13 to a monitor inside the cab 23 or an external monitor of the working machine 1. Figure 8 shows the report summarizing the multiple conditions when a reaction force was applied. In Figure 8, along with the multiple conditions, a comment summarizing them is included. By checking the contents of this report, it is possible to confirm the conditions under which the reaction force was applied. Therefore, it can be used to improve the operator's operations in the future.

[0054] In this case, the work machine 1 often performs similar tasks repeatedly. When performing similar tasks repeatedly, the attachment 30 operates along almost the same trajectory. Therefore, the controller (area setting means) 11 sets the operating area of ​​the attachment 30 based on the operating trajectory of the attachment 30. The operating area 90 of the attachment 30 and the operating trajectory 91 of the attachment 30 are shown in Figure 9. The top of Figure 9 is the front of the work machine 1, and the left and right directions in Figure 9 represent the left and right directions of the work machine 1. In Figure 9, the operating trajectory 91 forms a loop in the front and rear directions of the work machine 1.

[0055] The operating area of ​​attachment 30 is specifically set as follows: First, the coordinates P of the trajectory of attachment 30. Ti Normal distribution value H Ti This is calculated using the following equation (5) and stored in the memory device 13. H Ti =(1 / (2πσ 2 ) 1 / 2 )exp(-(P Ti -μ)2 / 2σ 2 )...Equation (5)

[0056] Next, the normal distribution value H is calculated and stored each time the operation is repeated. Ti Each coordinate P Ti The sum in is the threshold T h The above range is set as the operating area 90 in which the operation of attachment 30 is repeated and the operation of attachment 30 is limited.

[0057] Furthermore, if the operating area 90 is limited and can be identified in advance, the operator may manually set the main operating area 90.

[0058] The controller (reaction force applying means) 11 applies a reaction force from the reaction force device 14 to the operating lever 29 when the operating lever 29 is operated so that the attachment 30 moves outside the operating area 90. When the work machine 1 repeatedly performs similar tasks, the movement of the attachment 30 usually remains within the operating area 90. By applying a reaction force from the reaction force device 14 to the operating lever 29 when the operating lever 29 is operated so that the attachment 30 moves outside the operating area 90, it is possible to suppress operations that would cause the attachment 30 to move outside the operating area 90.

[0059] Furthermore, the controller (reaction force changing means) 11 increases the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 as the attachment 30 gets closer to the boundary of the operating area 90. This further suppresses operations that cause the attachment 30 to move outside the operating area 90.

[0060] Furthermore, the controller (reaction force changing means, calculation means) 11 increases the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 as awareness decreases, and calculates awareness outside the operating area 90 as zero. This ensures that a large reaction force is applied when the attachment 30 reaches the boundary of the operating area 90. This suppresses operations that would cause the attachment 30 to move outside the operating area 90.

[0061] Furthermore, the controller (calculation means) 11 may limit the range for calculating awareness to within the operating area 90. When the work machine 1 repeatedly performs similar tasks, the movement of the attachment 30 usually remains within the operating area 90. Therefore, there is little need to calculate the operator's awareness of an object 70 located outside the operating area 90. Thus, by limiting the range for calculating the operator's awareness of an object 70 to within the operating area 90, the computational load on the controller 11 can be reduced.

[0062] Here, in equation (5) above, the normal distribution value H Ti The variance value σ used in the calculation 2 The sensitivity when calculating the operating area 90 may be set in steps by the operator. For example, the relationship between the variance value and the adjustment value is shown in Figure 10, where the variance value σ is set to five adjustment values. 2 Set the value to 5 levels. Normal distribution value H Ti The relationship between the adjustment value and the variance value σ is shown in Figure 11. 2 By gradually changing this, the operating area 90 can be made wider or narrower. Also, the normal distribution value H Ti and threshold T h The relationship is shown in Figure 12, which shows the threshold T h You can widen or narrow the operating area 90 by changing this setting.

[0063] (effect) As described above, according to the working machine 1 of this embodiment, the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 changes according to the operator's awareness of the object 70 that at least a part of the working machine 1 is approaching. Therefore, information regarding the operator's awareness can be conveyed to the operator through the reaction force. For example, the less the operator is aware of the object 70, the greater the reaction force can be made to the operation toward that object 70. Also, for example, by making the reaction force toward an object 70 that the operator is aware of smaller (preferably zero), the workability of the working machine 1 can be prevented from decreasing. Furthermore, the reaction force to the operating force on the operating lever 29 can make the operator aware that they are not paying enough attention to the object 70. As a result, when the operator is operating the working machine 1 to bring a part of it closer to the object 70, they will be able to operate it while paying sufficient attention to the object 70. Therefore, the operator's skill can be improved.

[0064] Furthermore, the lower the operator's awareness of object 70, the greater the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29. This effectively alerts the operator to their lack of attention to object 70. Conversely, the higher the operator's awareness of object 70, the smaller the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29. This prevents the operation of the operating lever 29 from being restricted by the reaction force, thus preventing a decrease in the work efficiency of the work machine 1.

[0065] Furthermore, the manner in which the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 changes over time is also changed. For example, the reaction force can be applied in a manner that varies in speed. In this case, for example, the shorter the distance between the work machine 1 and the object 70, the more rapidly the reaction force is applied, which makes the operator aware that there is a high possibility of interference with the object 70. Alternatively, for example, the reaction force can be applied while vibrating. In this case, for example, the shorter the distance between the work machine 1 and the object 70, the more violently the reaction force is vibrated, which makes the operator aware that there is a high possibility of interference with the object 70. By changing the manner in which the reaction force is applied in this way, it is possible to make the operator more clearly aware of their lack of attention to the object 70.

[0066] Furthermore, the manner in which the reaction force is applied changes depending on the operator's awareness of the object 70. For example, in the case of applying the reaction force with varying speeds, the reaction force can be applied more rapidly to objects 70 that the operator is less aware of, while the reaction force can be applied more gradually to objects 70 that the operator is more aware of. Also, for example, in the case of applying the reaction force while vibrating, the vibration can be increased to objects 70 that the operator is less aware of, while the vibration can be decreased to objects 70 that the operator is more aware of. In this way, by changing the manner in which the reaction force is applied according to the operator's awareness of the object 70, it is possible to make the operator more clearly aware of their lack of attention to the object 70.

[0067] Furthermore, the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 changes depending on the distance between the object 70 and the attachment 30. For example, the shorter the distance between the object 70 and the attachment 30, the greater the reaction force applied to the operation toward the object 70. In this case, the operator can be made aware that there is a high possibility that the attachment 30 will interfere with the object 70.

[0068] Furthermore, if the operating lever 29 is operated to move the attachment 30 toward an object 70 located outside the movable range 80 of the attachment 30, no reaction force is applied to the operating lever 29 from the reaction force device 14. The attachment 30 has difficulty reaching an object 70 located outside the movable range 80 of the attachment 30. Therefore, the possibility of the attachment 30 interfering with such an object 70 is low, and no reaction force is applied even if the operating lever 29 is operated to move the attachment 30 toward such an object 70. This prevents the workability of the work machine 1 from being unnecessarily reduced.

[0069] Furthermore, the operator's awareness of an object 70 located outside the movable range 80 of the attachment 30 is not calculated. The attachment 30 has little chance of reaching an object 70 located outside its movable range 80. Therefore, the possibility of the attachment 30 interfering with such an object 70 is low, and the operator's awareness of such an object 70 is not calculated. This reduces the computational load on the controller 11.

[0070] Furthermore, a report is output detailing the conditions of at least one of the working machine 1 and the object 70 when the controller 11 applies a reaction force to the reaction force device 14. For example, after completing the work, the operator can review the contents of the report to understand the conditions under which the reaction force was applied. This information can then be used to improve the operator's future operations.

[0071] Furthermore, if the operating lever 29 is operated so that the attachment 30 moves outside the operating area 90, a reaction force is applied to the operating lever 29 from the reaction force device 14. For example, when the work machine 1 repeatedly performs the same operation, the movement of the attachment 30 usually remains within the operating area 90. By applying a reaction force to the operating lever 29 from the reaction force device 14 when the operating lever 29 is operated so that the attachment 30 moves outside the operating area 90, it is possible to suppress operations that cause the attachment 30 to move outside the operating area 90.

[0072] Furthermore, the closer the attachment 30 is to the boundary of the operating area 90, the greater the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29. This makes it possible to further suppress operations that cause the attachment 30 to move outside the operating area 90.

[0073] Furthermore, the lower the operator's awareness of object 70, the greater the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29, and the awareness outside the operating area 90 is calculated as zero. For example, when the work machine 1 repeatedly performs the same task, the movement of the attachment 30 usually remains within the operating area 90. Therefore, the awareness outside the operating area 90 is set to zero, and a large reaction force is applied when the attachment 30 reaches the boundary of the operating area 90. This makes it possible to suppress operations that cause the attachment 30 to go outside the operating area 90.

[0074] Furthermore, the range in which the operator's awareness of object 70 is calculated is limited to within the operating area 90. For example, when the work machine 1 repeatedly performs the same task, the movement of the attachment 30 usually remains within the operating area 90. Therefore, there is little need to calculate the operator's awareness of object 70 located outside the operating area 90. Thus, by limiting the range in which the operator's awareness of object 70 is calculated to within the operating area 90, the computational load on the controller 11 can be reduced.

[0075] Furthermore, the magnitude of the reaction force applied from the reaction force device 14 to the operating lever 29 is changed according to the distribution of the operator's awareness of the surroundings of the work machine 1. Therefore, information about the operator's awareness can be conveyed to the operator through the reaction force. For example, if the operator's awareness of the area around the work machine is low, the reaction force for operations toward the area with low awareness can be increased. Also, for example, by reducing (preferably making zero) the reaction force for operations toward the area with high awareness of the operator, the workability of the work machine 1 can be prevented from decreasing. The reaction force in response to the operating force on the operating lever 29 can make the operator aware of insufficient attention to the direction in which the work machine 1 is moving. As a result, the operator will pay sufficient attention to the direction of movement when operating to move a part of the work machine 1. Therefore, the operator's skill can be improved.

[0076] Although embodiments of the present invention have been described above, these are merely illustrative examples and do not particularly limit the present invention. Specific configurations and other aspects can be modified as appropriate. Furthermore, the actions and effects described in the embodiments of the invention are merely a list of the most preferred actions and effects resulting from the present invention, and the actions and effects according to the present invention are not limited to those described in the embodiments. [Explanation of Symbols]

[0077] 1. Working Machinery 11 Controller (calculation means, reaction force application means, reaction force change means, reaction force mode means, range setting means, output device, area setting means) 13 Storage device 14 Reaction device 21 Lower running body 22 Upper rotating body 23 Cab 24 Swivel device 25 Machine body 26. Indoor camera (with eye-tracking device) 27. Camera (object detection device) 28. LiDAR (Object Detection System) 29. Operating lever (operating device) 30 Attachments 31 Boom 32 Arms 33 buckets 34 Link members 40 cylinders 41 Boom Cylinder 42 Arm Cylinder 43 Bucket Cylinder 70 Object 80 Range of motion 90 Operating Area 91 Motion trajectory

Claims

1. A work machine equipped with an operating device that is operated by an operator, A gaze detection device for detecting the operator's line of sight, An object detection device for detecting objects in the vicinity of the aforementioned work machine, A reaction force device capable of applying a reaction force to the aforementioned operating device, which is a force opposite to the operating force applied to the aforementioned operating device, A calculation means for calculating the operator's awareness of the object detected by the object detection device based on the detection results of the gaze detection device, When the operating device is operated to move at least a part of the work machine toward the object detected by the object detection device, the reaction force applying means for applying the reaction force from the reaction force device to the operating device, A reaction force changing means that changes the magnitude of the reaction force applied by the reaction force device according to the recognition level calculated by the calculation means, It has, The calculation means is Based on the time the operator has gazed at each of the gaze points, which are coordinates that the operator has gazed at on the object within a specified sampling interval, the cognitive level of each gaze point is calculated. Based on the coordinates of each of the gaze points and the respective cognitive levels of each of the gaze points, a cognitive level distribution representing the distribution of the cognitive levels in the coordinates surrounding the work machine is calculated as the cognitive level. The reaction force changing means is characterized in that it changes the magnitude of the reaction force applied by the reaction force device according to the value of the cognitive level distribution in the coordinates of the object to which at least a part of the work machine approaches.

2. The work machine according to claim 1, characterized in that the reaction force changing means increases the magnitude of the reaction force as the value of the recognition level distribution in the coordinates of the object to which at least a part of the work machine approaches by operating the operating device decreases, and decreases the magnitude of the reaction force as the value increases.

3. The work machine according to claim 1 or 2, characterized in that it has a reaction force mode means for changing the mode of application, which is the mode of change in the magnitude of the reaction force over time.

4. The work machine according to claim 3, characterized in that the reaction force mode means changes the mode of application according to the value of the recognition level distribution at the coordinates of the object to which at least a part of the work machine approaches by operation of the operating device.

5. Having an attachment, The working machine according to claim 1 or 2, characterized in that the reaction force changing means changes the magnitude of the reaction force according to the distance between the object and the attachment.

6. Attachments and, A range setting means for setting the movable range of the attachment, It has, The working machine according to claim 1 or 2, characterized in that the reaction force applying means does not apply a reaction force from the reaction force device to the operating device when the operating device is operated to move the attachment toward an object located outside the range of motion.

7. Attachments and, A range setting means for setting the movable range of the attachment, It has, The work machine according to claim 1 or 2, characterized in that the calculation means does not calculate the degree of recognition for the object located outside the range of motion.

8. The work machine according to claim 1 or 2, characterized in that it has an output device that outputs a report of the situation relating to at least one of the work machine and the object when the reaction force applying means applies the reaction force to the reaction force device.

9. A work machine equipped with an operating device operated by an operator, A gaze detection device for detecting the operator's line of sight, An object detection device for detecting objects in the vicinity of the aforementioned work machine, A reaction force device capable of applying a reaction force to the aforementioned operating device, which is a force opposite to the operating force applied to the aforementioned operating device, A calculation means for calculating the operator's awareness of the object detected by the object detection device based on the detection results of the gaze detection device, When the operating device is operated to move at least a part of the work machine toward the object detected by the object detection device, the reaction force applying means for applying the reaction force from the reaction force device to the operating device, A reaction force changing means that changes the magnitude of the reaction force applied by the reaction force device according to the recognition level calculated by the calculation means, Attachments and, Area setting means for setting the operating area of ​​the attachment based on the operating trajectory of the attachment, It has, The reaction force applying means is characterized in that it applies a reaction force from the reaction force device to the operating device when at least one of the following conditions is met: when the operating device is operated to move at least a part of the work machine toward the object detected by the object detection device, or when the operating device is operated to move the attachment outside the operating area.

10. The working machine according to claim 9, characterized in that the reaction force changing means increases the magnitude of the reaction force as the attachment is closer to the boundary of the operating area.

11. The reaction force changing means increases the magnitude of the reaction force as the level of awareness decreases. The work machine according to claim 9, characterized in that the calculation means calculates the recognition level outside the operating area as zero.

12. Attachments and, Area setting means for setting the operating area of ​​the attachment based on the operating trajectory of the attachment, It has, The work machine according to claim 1 or 2, characterized in that the calculation means limits the range for calculating the recognition level to within the operating area.

13. A work machine equipped with an operating device that is operated by an operator, A gaze detection device for detecting the operator's line of sight, A reaction force device capable of applying a reaction force to the aforementioned operating device, which is a force opposite to the operating force applied to the aforementioned operating device, A calculation means for calculating the distribution of the operator's awareness of the surroundings of the work machine based on the detection results of the gaze detection device, When the operating device is operated to move at least a part of the work machine, a reaction force applying means for applying the reaction force from the reaction force device to the operating device, A reaction force changing means that changes the magnitude of the reaction force applied by the reaction force device according to the distribution of recognition calculated by the calculation means, It has, The calculation means is Based on the operator's gaze time for each of the gaze points, which are coordinates that the operator has focused on in the area surrounding the work machine within a specified sampling interval, the cognitive level of each gaze point is calculated. Based on the coordinates of each of the gaze points and the respective cognitive levels of each of the gaze points, a cognitive level distribution representing the distribution of cognitive levels in the coordinates surrounding the work machine is calculated as the cognitive level distribution. The reaction force changing means is characterized in that it changes the magnitude of the reaction force applied by the reaction force device according to the value of the cognitive level distribution in the coordinates in the direction in which at least a part of the work machine is moved.

14. A working machine according to any one of claims 1, 9, and 13, The work machine is provided with a monitor located outside the work machine that displays a view including at least a portion of the area around the work machine, The eye-tracking device is characterized by detecting the gaze of the operator who remotely operates the work machine while looking at the monitor.

Images