Display system, display method, and non-volatile storage medium

Abstract

This invention provides a display system, display method, and non-volatile storage medium that enable further efficiency in attaching and detaching attachments. [Solution] The attachment 30 is detachable from the coupler 8. The external sensor 20 acquires sensor data related to the attachment 30. The controller 50 outputs a guidance image to the monitor 40 based on the sensor data acquired by the external sensor 20. The guidance image includes a display that allows recognition of the relative position between the gripping portion 8b of the coupler 8 and the first pin 31 of the attachment 30 that is attached to and detached from the gripping portion 8b.

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , , , ,

[0001] The present disclosure relates to a display system, a display method, and a non - volatile memory medium.

Background Art

[0002] Regarding the alignment between an attachment and a tool carrier, it is disclosed in European Patent Application Publication No. 4265849 (Patent Document 1). In Patent Document 1, in order to connect the tool carrier of a loader to an attachment, a visual movement instruction for moving the loader relative to the attachment is provided to an operator.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It may be difficult to align with the tool carrier when attaching an attachment, and further improvement in the efficiency of attachment and detachment of the attachment is desired.

[0005] An object of the present disclosure is to provide a display system, a display method, and a non - volatile memory medium that enable further improvement in the efficiency of attachment and detachment of an attachment.

Means for Solving the Problems

[0006] The display system of the present disclosure includes a sensor, a monitor, and a controller. The sensor acquires sensor data related to an attachment that is detachable from a coupler. The controller outputs a guidance image to the monitor based on the sensor data. The guidance image includes a display that can recognize the relative position between the gripping portion of the coupler and the pin of the attachment to be detached from the gripping portion.

[0007] The display method of this disclosure is a display method that displays images on a monitor to assist in attaching an attachment to a coupler. The display method in the work machine of this disclosure comprises the steps of acquiring sensor data related to the attachment and outputting a guidance image to a monitor based on the acquired sensor data. The guidance image includes a display that allows recognition of the relative position between the gripping portion of the coupler and the pins of the attachment to be attached to and detached from the gripping portion.

[0008] The non-transitory storage medium of this disclosure stores a program that causes the controller's processor to execute the following steps: acquiring sensor data related to an attachment, and outputting a guidance image to a monitor based on the acquired sensor data, which includes a display that allows recognition of the relative position between the gripping portion of the coupler and the pins of the attachment that are attached to and detached from the gripping portion. [Effects of the Invention]

[0009] According to this disclosure, a display system, a display method, and a non-volatile storage medium can be realized that enable further efficiency in attachment and detachment. [Brief explanation of the drawing]

[0010] [Figure 1] This is a perspective view showing the configuration of a work machine in one embodiment of the present disclosure. [Figure 2] This diagram illustrates the operation of the coupler when an attachment is mounted on the work machine shown in Figure 1. [Figure 3] Figure 1 is a functional block diagram showing the configuration of the controller in the display system of the work machine shown. [Figure 4] Figure 1 is a flowchart illustrating the display method for the work machine shown. [Figure 5] This is a diagram showing a guidance image displayed in three dimensions. [Figure 6]This diagram shows a guidance image that includes arrows (support arrows) indicating the direction of movement to the target location. [Figure 7] This figure shows an image of the target position and a guidance image that includes coupler rotation instructions. [Figure 8] This figure shows a guidance image including lines and dimensions for operation assistance. [Figure 9] This figure shows a guidance image including angles for operation assistance in a top view. [Figure 10] This diagram shows guidance images to assist in aligning the coupler directly with the attachment in a top view. [Modes for carrying out the invention]

[0011] The embodiments of this disclosure will be described below with reference to the drawings.

[0012] In the specification and drawings, the same reference numerals are used for identical or corresponding components, and redundant descriptions are avoided. Furthermore, in the drawings, components may be omitted or simplified for the sake of clarity. Also, at least some of the embodiments and modifications may be combined in any way.

[0013] In the following explanation, "up," "down," "front," "rear," "left," and "right" refer to directions relative to the operator seated in the driver's seat 4S within the cab 4 shown in Figure 1. A side view refers to a viewpoint from which the work machine 100 is viewed from the left or right. A top view refers to a viewpoint from which the work machine 100 is viewed from above and below.

[0014] <Configuration of the working machine>

[0015] As an example of a work machine covered by this disclosure, the configuration of a hydraulic excavator will be explained with reference to Figure 1. Note that the work machines to which this disclosure applies are not limited to hydraulic excavators, but may also include other work machines such as backhoes.

[0016] FIG. 1 is a diagram schematically showing the configuration of a hydraulic excavator as an example of a working machine in an embodiment of the present disclosure. As shown in FIG. 1, the hydraulic excavator 100 of the present embodiment has a main body 1 and a working machine 2 that operates hydraulically. The main body 1 has a revolving body 3 and a traveling body 5.

[0017] The traveling body 5 has a pair of crawlers 5Cr and a traveling motor 5M. The hydraulic excavator 100 can travel by the rotation of the crawlers 5Cr. The traveling motor 5M is provided as a drive source for the traveling body 5. The traveling motor 5M is a hydraulic motor that operates hydraulically. Note that the traveling body 5 may have wheels (tires).

[0018] The revolving body 3 is disposed on the traveling body 5 and supported by the traveling body 5. The revolving body 3 can revolve with respect to the traveling body 5 about a revolving axis RX by a revolving motor (not shown). The revolving axis RX is the center of revolution of the revolving body 3. The revolving motor is a hydraulic motor that operates hydraulically. The revolving axis RX is an imaginary straight line that is the center of revolution of the revolving body 3. Note that the traveling motor 5M or the revolving motor may be an electric motor.

[0019] The revolving body 3 has a driver's cab 4 (cab). Inside the driver's cab 4, a driver's seat 4S on which an operator sits is provided. The operator (occupant) boards the driver's cab 4 and can perform operations such as attaching and detaching the attachment 30 to the coupler 8, operating the working machine 2, revolving the revolving body 3 with respect to the traveling body 5, and traveling the hydraulic excavator 100 by the traveling body 5. The hydraulic excavator 100 may be remotely operated.

[0020] The working machine 2 is attached to the revolving body 3. The working machine 2 has a boom 6, an arm 7, a boom cylinder 10, an arm cylinder 11, and an attachment cylinder 12. The coupler 8 is attached to the working machine 2 and is attached to the tip of the arm 7.

[0021] The boom 6 is rotatably connected to the main body 1. Specifically, the base end of the boom 6 is rotatably connected to the slewing body 3 with the boom foot pin 13 as the pivot point. The arm 7 is rotatably connected to the boom 6. Specifically, the base end of the arm 7 is rotatably connected to the tip of the boom 6 with the boom top pin 14 as the pivot point. The coupler 8 is rotatably connected to the arm 7. Specifically, one end of the coupler 8 is rotatably connected to the tip of the arm 7 with the arm top pin 15 as the pivot point.

[0022] One end of the boom cylinder 10 is connected to the slewing body 3, and the other end is connected to the boom 6. The boom 6 is movable relative to the main body 1 by the boom cylinder 10. Due to the movement of the boom cylinder 10, the boom 6 can rotate vertically relative to the slewing body 3 with the boom foot pin 13 as the pivot point.

[0023] One end of the arm cylinder 11 is connected to the boom 6, and the other end is connected to the arm 7. The arm 7 is movable relative to the boom 6 by the arm cylinder 11. The movement of the arm cylinder 11 allows the arm 7 to rotate in the digging direction or the dumping direction, with the boom top pin 14 as the pivot point.

[0024] One end of the attachment cylinder 12 is connected to the arm 7, and the other end is connected to the link 17. The link 17 is connected to the other end of the coupler 8 by a link pin 16. The coupler 8 is movable relative to the arm 7 by the attachment cylinder 12. The movement of the attachment cylinder 12 allows the coupler 8 to rotate around the arm top pin 15 as a pivot point.

[0025] Each of the boom cylinder 10, arm cylinder 11, and attachment cylinder 12 is a hydraulic cylinder driven by hydraulic pressure, but other actuators such as electric cylinders may also be used.

[0026] The coupler 8 is configured to allow attachment 30 to be attached and detached. The coupler 8 makes it easy to change the attachment 30 that is attached to the tip of the work implement 2. The attachments 30 that can be attached to the tip of the work implement 2 include, for example, a bucket, breaker, grapple, large crusher, and small crusher. In Figure 1, a bucket is shown as an example of an attachment 30.

[0027] The coupler 8 includes, for example, a coupler body 8a and a locking pin 8c. The coupler body 8a has holes for supporting the arm top pin 15 and the link pin 16, respectively. The coupler body 8a also has a gripping portion 8b. The gripping portion 8b is, for example, a recess and has a hook shape. The first pin 31 of the attachment 30 can be inserted into the gripping portion 8b. By inserting the first pin 31 into the gripping portion 8b, the first pin 31 can be hooked onto the coupler 8.

[0028] The locking pin 8c is attached to the coupler body 8a. For example, two locking pins 8c are attached to the coupler body 8a. The locking pin 8c is controlled to move between, for example, a retracted state and an extended state within the coupler body 8a. When the locking pin 8c is extended from the coupler body 8a (locked state), the coupler body 8a and the locking pin 8c form a pin locking portion. The second pin 32 of the attachment 30 can be inserted into this pin locking portion. When the second pin 32 is inserted into the pin locking portion, the second pin 32 is sandwiched between the coupler body 8a and the locking pin 8c and locked into the pin locking portion. When the locking pin 8c is retracted into the coupler body 8a (unlocked state), the locking state of the second pin 32 is released.

[0029] An external sensor 20 is attached to the hydraulic excavator 100. The external sensor 20 is attached to the operator's cab 4 of the hydraulic excavator 100, for example. The external sensor 20 may be attached to the top of the operator's cab 4, or to the side of the operator's cab 4. The external sensor 20 may be attached to parts of the hydraulic excavator 100 other than the operator's cab 4, or it may be installed at a location away from the hydraulic excavator 100.

[0030] The external sensor 20 acquires sensor data related to the attachment 30. The external sensor 20 is, for example, an imaging device. The external sensor 20 may be, for example, a monocular camera that acquires a two-dimensional image of the attachment 30. Alternatively, the external sensor 20 may be, for example, a depth camera. As a depth camera, for example, a stereo camera, a structured light camera, or a time-of-flight camera may be used. The external sensor 20 is not limited to an imaging device as long as it can acquire sensor data related to the attachment 30, and may also be, for example, a LiDAR (Light Detection and Ranging). Thus, the external sensor 20 includes cameras, LiDAR, etc. Therefore, the sensor data related to the attachment 30 includes images of the attachment 30 captured by the camera, point cloud data of the attachment 30 acquired by the LiDAR, etc. Furthermore, the sensor that acquires sensor data related to the attachment 30 is not limited to the external sensor, but can be any sensor that can acquire sensor data related to the attachment 30.

[0031] The hydraulic excavator 100 has an attitude sensor 21 (Figure 3) and a position and orientation sensor 24 (Figure 3). The attitude sensor 21 detects the attitude of the work implement 2 and outputs an attitude signal indicating the attitude of the work implement 2. The attitude sensor 21 can detect the attitude of the boom 6, arm 7, and coupler 8. The attitude sensor 21 includes sensors located on each of the boom 6, arm 7, and coupler 8. The attitude sensor 21 may be, for example, an IMU (Inertial Measurement Unit), a stroke sensor, a potentiometer, an imaging device, etc.

[0032] The position and orientation sensor 24 (Figure 3) includes, for example, a GNSS (Global Navigation Satellite Systems) receiver and a GNSS antenna. The position and orientation sensor 24 includes two GNSS antennas 23a and 23b (Figures 1 and 3). Each of the two GNSS antennas 23a and 23b is installed at a different position on the rotating body 3. The GNSS receiver (not shown) calculates the position of the rotating body 3 in the global coordinate system and the direction in which the rotating body 3 is facing from the satellite positioning signals received by each of the GNSS antennas 23a and 23b from the satellite. The GNSS receiver outputs a position signal indicating the position of the rotating body 3 and an orientation signal indicating the direction in which the rotating body 3 is facing.

[0033] The position and orientation sensor 24 has a rotation angle sensor 22 (Figure 3). The rotation angle sensor 22 is fixed to the rotating body 3, for example. The rotation angle sensor 22 detects the rotation angle of the rotating body 3 relative to the traveling body 5 and outputs a rotation angle signal indicating the rotation angle of the rotating body 3. The rotation angle sensor 22 can detect the rotation angle in the machine coordinate system (local coordinate system). The rotation angle sensor 22 may be, for example, an IMU, a potentiometer, or an imaging device. The machine coordinate system is a Cartesian coordinate system with the rotation center of the rotating body 3 as the origin, and represented by axes extending in the front-rear direction, the left-right direction, and the up-down direction (rotation axis RX: Figure 1).

[0034] <Attachment installation procedure>

[0035] Next, the operation of attaching the attachment 30 to the work machine shown in Figure 1 will be explained using Figure 2.

[0036] Figure 2 is a diagram illustrating the operation of the coupler when an attachment is mounted on the work machine shown in Figure 1. As shown in Figure 2(A), the hydraulic excavator 100 moves to a position where the attachment 30 can be mounted. The position where the hydraulic excavator 100 can mount the attachment 30 is the position where the gripping portion 8b of the coupler 8 reaches the first pin 31 of the attachment 30 due to the operation of the work machine 2, and where the coupler 8 is directly facing the attachment 30. The position where the coupler 8 is directly facing the attachment 30 will be described later using Figure 10.

[0037] In this state, the lock pin 8c of the coupler 8 is in a locked position, for example, protruding from the coupler body 8a. The circle 60 shown by the dashed line in the figure is the target position of the arm top pin 15 when the attachment 30 is being mounted.

[0038] As shown in Figure 2(B), the arm 7 and boom 6 are driven, causing the arm top pin 15 to move toward the target position 60, thereby moving the coupler 8 toward the attachment 30. As the coupler 8 moves toward the target position 60, the lock pin 8c is retracted into the coupler body 8a, entering an unlocked state.

[0039] As shown in Figure 2(C), the above movement of the coupler 8 causes the arm top pin 15 to reach the target position 60. In this state, the first pin 31 of the attachment 30 is inserted into the gripping portion 8b of the coupler 8.

[0040] As shown in Figure 2(D), the coupler 8 rotates relative to the arm 7 while the first pin 31 remains inserted within the gripping portion 8b. This moves the coupler 8 to the mounting position of the attachment 30. At this mounting position, the mounting surface (bottom surface) 8d of the coupler 8 is aligned with a portion of the surface of the attachment 30. Up to this point, the attachment 30 remains stationary on the ground without moving.

[0041] As shown in Figure 2(E), the coupler 8 moves to lift the attachment 30 off the ground. At this time, the gripping portion 8b of the coupler 8 holds the first pin 31 of the attachment 30, and the mounting surface 8d of the coupler 8 supports a portion of the surface of the attachment 30.

[0042] As shown in Figure 2(F), the locking pin 8c is positioned in a locked state, protruding from the coupler body 8a. This causes the second pin 32 of the attachment 30 to be sandwiched and locked between the locking pin 8c and the coupler body 8a. As a result, the attachment 30 is locked to the coupler 8 and attached to the coupler 8.

[0043] In this embodiment, the attachment operation of the attachment 30 is performed as described above. The attachment operation of the attachment 30 is performed by an operator sitting in the operator's cab 4, or by remote operation by an operator at a remote location away from the hydraulic excavator 100.

[0044] Although the above describes a method of attaching an attachment 30 in which the attachment 30 is scooped up by the coupler 8 and then locked, this disclosure is also applicable to an attachment 30 in which the attachment 30 is locked to the coupler 8 without scooping up the attachment 30.

[0045] <Display system and display method>

[0046] Next, the display system and display method in this embodiment will be described.

[0047] Figure 3 is a functional block diagram showing the controller configuration in the display system of the work machine shown in Figure 1. Figure 4 is a flowchart showing the display method in the work machine shown in Figure 1.

[0048] In the following explanation, an image of attachment 30 (an image captured by a camera such as a monocular camera or depth camera) will be described as an example of sensor data related to attachment 30. However, the image of attachment 30 in the following explanation may also be information similar to an image (for example, point cloud data from LiDAR), as long as it is sensor data related to attachment 30.

[0049] As shown in Figure 3, the controller 50 acquires an image of the attachment 30 obtained by the external sensor 20. Based on the image of the attachment 30 detected by the external sensor 20, the controller 50 outputs a guidance image to the monitor 40 that includes the relative position of the hydraulic excavator 100 and the attachment 30. The guidance image includes a display that allows recognition of the relative position between the gripping portion 8b of the coupler 8 and the first pin 31 of the attachment 30 that is attached to and detached from the gripping portion 8b.

[0050] The controller 50 acquires information regarding the attitude of the work machine 2 detected by the attitude sensor 21. The controller 50 acquires information regarding the position and orientation of the slewing body 3 calculated by the GNSS receiver based on satellite positioning signals received by the GNSS antennas 23a and 23b. The controller 50 also acquires information regarding the slewing angle of the slewing body 3 relative to the traveling body 5 detected by the slewing angle sensor 22. The monitor 40 is composed of a graphic display, such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The monitor 40 may be attached to the hydraulic excavator 100, or it may be attached to a remote controller for remotely operating the hydraulic excavator 100.

[0051] As shown in Figures 3 and 4, the controller 50 includes an acquisition unit 50a, a selection unit 50b, an image generation unit 50c, an output control unit 50d, and a memory 50e. The acquisition unit 50a acquires an image of the attachment 30 detected by the external sensor 20 (Step S1: Figure 4). The controller 50 detects the attachment 30 based on the acquired image of the attachment 30 (Step S2: Figure 4). The detection of the attachment 30 by the controller 50 may also be performed by the operator touching the image area of ​​the attachment 30 on the monitor 40.

[0052] The selection unit 50b selects a candidate attachment based on the image of the attachment 30 acquired from the acquisition unit 50a (Step S3: Figure 4). When selecting a candidate attachment, the selection unit 50b refers to a list pre-stored in memory 50e. The list stored in memory 50e contains information related to the shapes of multiple candidate attachments. Memory 50e may also store, for example, 3D CAD (Computer-Aided Design) data of the candidate attachment as information related to the shape of the candidate attachment. From among the multiple candidate attachments included in the list, the selection unit 50b selects a candidate attachment that matches the image of the attachment 30. Specifically, the selection unit 50b selects a candidate attachment that matches the image of the attachment 30 based on a comparison between the image of the attachment 30 acquired from the acquisition unit 50a and the information related to the shape of the candidate attachment. If the selection unit 50b determines that there is an attachment among the multiple candidate attachments included in the list that is the same as the attachment 30 acquired from the acquisition unit 50a, it uses the information related to the shape of that identical candidate attachment. In this case, if the image of attachment 30 and the information related to the shape of the candidate attachment do not match, an error may be displayed on the monitor 40. If they do not match, the selection unit 50b may select the candidate attachment with the highest similarity in shape. The controller 50 may display multiple candidate attachments with high similarity in shape and allow the operator to select one.

[0053] In some cases, the attachment 30 may be replaced with one not included in the above list. In this case, the attachment 30 is photographed by an imaging device mounted on the hydraulic excavator 100 while the excavator 100 is being moved. This allows the three-dimensional shape of the attachment 30 to be acquired, and 3D CAD data may be created and stored in the list. Alternatively, the attachment 30 may be photographed by the operator using a handheld camera (for example, a camera mounted on a mobile information terminal) while changing the viewpoint, thereby creating 3D CAD data of the attachment 30 and storing it in the list. Furthermore, the operator may physically set the position of the first pin 31.

[0054] Memory 50e stores dimensional data related to the hydraulic excavator 100. This dimensional data may include, for example, the distance between the GNSS antennas 23a and 23b and the boom foot pin 13, the distance between the boom foot pin 13 and the boom top pin 14, the distance between the boom top pin 14 and the arm top pin 15, and data on the shapes of the boom 6, arm 7, and coupler 8 on the hydraulic excavator 100. The dimensional data stored in memory 50e may also include the position of the external sensor 20 and the installation angle of the external sensor 20 on the hydraulic excavator 100.

[0055] The controller 50 acquires information such as the type, shape, and dimensions of the attachment 30 based on information related to the shape of the selected candidate attachment, and estimates the relative position and orientation of the attachment 30 to the hydraulic excavator 100. The controller 50 also estimates the position and orientation of at least the coupler 8 on the hydraulic excavator 100 based on information acquired from the attitude sensor 21, information acquired from the position and orientation sensor 24, and dimensional data stored in the memory 50e. Furthermore, the controller 50 can improve the positional accuracy of the attachment 30 relative to the hydraulic excavator 100 by referring to the position of the external sensor 20 on the hydraulic excavator 100.

[0056] The image generation unit 50c acquires information related to the shape of the candidate attachment selected by the selection unit 50b and generates a guidance image based on the selected candidate attachment (Step S4: Figure 4). Specifically, the image generation unit 50c generates a guidance image that visualizes the position and orientation of the attachment 30 and coupler 8, respectively, as estimated by the controller 50. The guidance image includes an image showing the relative position between the attachment 30 and the hydraulic excavator 100. The relative position may also include the relative angle between the gripping part 8b and the first pin 31c. Specific examples of guidance images will be described later.

[0057] The output control unit 50d outputs the guidance image acquired from the image generation unit 50c to the monitor 40 (Step S5: Figure 4). As a result, the guidance image is displayed on the monitor 40 (Step S6: Figure 4). The guidance image is an image that allows recognition of the relative position between the gripping portion 8b of the coupler 8 and the first pin 31c of the attachment 30 that is attached to and detached from the gripping portion 8b, as described above.

[0058] The controller 50 includes a processor, main memory, and storage. The processor is, for example, a CPU (Central Processing Unit). The main memory includes, for example, non-volatile memory such as ROM (Read Only Memory) and volatile memory such as RAM (Random Access Memory).

[0059] The controller 50 reads the program stored in storage, loads it into main memory, and executes predetermined processing according to the program. The program may also be distributed to the controller 50 via the network.

[0060] The controller 50 may be mounted on the hydraulic excavator 100, or it may be located separately outside the hydraulic excavator 100. If the controller 50 is located separately outside the hydraulic excavator 100, the controller 50 may be wirelessly connected to the external sensor 20, attitude sensor 21, slewing angle sensor 22, etc. The controller 50 may be stored in a server located away from the hydraulic excavator 100.

[0061] <Example of guidance image>

[0062] Next, examples of guidance images displayed on monitor 40 will be explained using Figures 2, 5 to 12.

[0063] The guidance image may be a two-dimensional image representing the coupler 8 and attachment 30 in a side view, as shown in Figure 2. Alternatively, the guidance image may be a three-dimensional image representing the coupler 8 and attachment 30 in an oblique view, including a side view, as shown in Figure 5.

[0064] As shown in Figure 5, the guidance image only needs to display at least the gripping portion 8b and the first pin 31 of the coupler 8. However, the guidance image may also display the coupler 8, attachment 30, arm 7, link 17, etc., and may also display the entire hydraulic excavator 100. Furthermore, the components displayed in the guidance image may be displayed as semi-transparent skeleton images with the background transparent. For example, the part of the attachment 30 other than the first pin 31 may be displayed as a skeleton image. Similarly, the parts of the hydraulic excavator 100 other than the coupler 8 may be displayed as skeleton images.

[0065] The guidance images may also include symbols such as arrows to assist the operator, as shown in Figures 5 and 6. Operational assistance arrows indicate, for example, the direction in which to move the coupler 8 to the target position when attaching or detaching an attachment.

[0066] The above target position may be the position of the coupler 8 in the inserted state, where the first pin 31 is inserted into the gripping portion 8b of the coupler 8, or it may be an intermediate position in the transition from the current position to the inserted state. In Figure 6, a circuit breaker is shown as an example of the attachment 30.

[0067] By recognizing symbols such as arrows in the guidance image, the operator can determine the next direction of movement for the coupler 8, making it possible to perform the attachment and detachment operation of the attachment 30 more efficiently.

[0068] Furthermore, the guidance image may also show the target position of arm 7 and coupler 8 in addition to the current position of arm 7 and coupler 8, as shown in Figure 7. In Figure 7, the current position of arm 7 and coupler 8 is shown by a solid line, and the target position of arm 7 and coupler 8 is shown by a dashed line.

[0069] In the guidance image, it is preferable that the arm 7 and coupler 8 at the target position are displayed in a different format than the arm 7 and coupler 8 at the current position so that they can be easily distinguished from the arm 7 and coupler 8 at the current position. The arm 7 and coupler 8 at the target position may be displayed, for example, as a skeleton image with a transparent background.

[0070] The target position described above may be the inserted state in which the first pin 31 is inserted into the gripping portion 8b of the coupler 8, or it may be an intermediate state in the process of transitioning from the current position to the inserted state. Note that in Figure 7, the attachment 30 is shown in a simplified form. In Figure 7, the arm 7 and coupler 8 at the current position and the arm 7 and coupler 8 at the target position are shown, but either the arm 7 or the coupler 8 may be shown separately.

[0071] By recognizing the coupler 8 at the target position using the guidance image, the operator can understand the next transition state of the coupler 8, making it possible to perform the attachment and detachment operation of the attachment 30 more efficiently.

[0072] The guidance image may also include at least one of the lines and dimensions for operation assistance, as shown in Figure 8. The lines for operation assistance may include, for example, multiple straight lines L1a, L1b, L2a, L2b, L3a, L3b.

[0073] Lines L1a and L1b are, for example, a horizontal and a vertical line passing through the center of the arm top pin 15 in a side view of the hydraulic excavator 100. Lines L2a and L2b are, for example, a horizontal and a vertical line passing through the center of the gripping section 8b in a side view of the hydraulic excavator 100. The center of the gripping section 8b is the center of the inscribed or circumscribed circle of the gripping section 8b in a side view. Lines L3a and L3b are, for example, a horizontal and a vertical line passing through the center of the first pin 31 in a side view of the hydraulic excavator 100.

[0074] Lines L1a, L2a, and L3a are all parallel to each other. Lines L1b, L2b, and L3b are all parallel to each other. Lines L1a and L1b are perpendicular to each other, lines L2a and L2b are perpendicular to each other, and lines L3a and L3b are perpendicular to each other.

[0075] The lines used for operational support can be any lines that the operator uses as a reference point when attaching or detaching the attachment 30, and are not limited to the horizontal or vertical lines mentioned above.

[0076] By recognizing the lines used for operational assistance in the guidance image, the operator can determine the positional discrepancies between the arm top pin 15, the gripping part 8b, the first pin 31, etc., thereby enabling more efficient attachment and detachment of the attachment 30.

[0077] The dimensions for operation assistance include multiple dimensions X and Y. Dimension X is, for example, the horizontal distance between the center of the arm top pin 15 and the center of the first pin 31 in a side view of the hydraulic excavator 100. Dimension Y is, for example, the vertical distance between the center of the gripping portion 8b and the center of the first pin 31 in a side view of the hydraulic excavator 100.

[0078] By recognizing the dimensions for operational assistance in the guidance image, the operator can accurately determine the amount of misalignment between the arm top pin 15, gripping part 8b, first pin 31, etc., thereby enabling more efficient attachment and detachment of the attachment 30.

[0079] The guidance image includes the relative position of the gripping portion 8b and the first pin 31c, as described above. The relative position of the gripping portion 8b and the first pin 31c includes the relative angle between the gripping portion 8b and the first pin 31c. The relative angle is, for example, the angle θ1 between a straight line L2 extending along the axial direction of the first pin 31 in a top view and a straight line L1 extending along the axial direction of the gripping portion 8b, as shown in Figure 9. The axial direction of the gripping portion 8b refers to the axial direction of the first pin 31 when the first pin 31 is inserted into the gripping portion 8b. The relative angle may also be the angle θ2 between a straight line L4 perpendicular to the first pin 31 and a straight line L3 extending from the base end to the tip end of the boom 6. In this guidance image, the entire arm 7 or its tip and the coupler 8 may be hidden, or they may be shown as a skeleton.

[0080] By recognizing angles θ1 and θ2 in the guidance image, the operator can determine the difference in angle between the gripping portion 8b and the first pin 31 in a top view, thereby enabling more efficient attachment and detachment of the attachment 30.

[0081] The guidance image includes instructions for operating the slewing body 3 and the traveling body 5 so that the relative angle between the gripping portion 8b and the first pin 31 in a top view is reduced, as shown in Figure 10. These instructions include instructions for traveling the traveling body 5 (white arrows in Figure 10) and instructions for rotating the slewing body 3 (solid black arrows in Figure 10) so that the hydraulic excavator 100 is in the state shown by the dashed line in Figure 10. Specifically, the guidance image displays instructions for traveling the traveling body 5 and rotating the slewing body 3 so that in a top view, the straight line L3 extending from the base to the tip of the boom 6 coincides with the straight line L4 perpendicular to the first pin 31 (so that the angle θ2 is zero). The guidance image may also include instructions for traveling the traveling body 5 in a direction parallel to the straight line L2 extending in the axial direction of the first pin 31. The guidance image may also include instructions for traveling the traveling body 5 in a direction different from (not parallel to) the straight line L2.

[0082] In the top view, the state in which line L3 coincides with line L4 is the state in which the coupler 8 is directly facing the attachment 30. The state of the hydraulic excavator 100 shown by the dashed line in Figure 10 may also be shown as a skeleton.

[0083] The guidance image includes an indication that the traveling body 5 should be operated so that its extension direction L5 and the axial direction L2 of the first pin 31 are perpendicular or parallel to each other, as an instruction to orient the coupler 8 directly toward the attachment 30. The above indication is, for example, the dashed arrow and the dashed line L5A in Figure 10. The dashed arrow indicates the direction of change of the extension direction L5. The dashed line L5A is a line that extends parallel to the axial direction L2 of the first pin 31 in a top view, and indicates the target extension direction of the traveling body 5. For the sake of explanation, the indication of the direction of change is shown as a dashed arrow, but it is not limited to this, and any indication that shows the direction of change is acceptable. Similarly, the indication of the target extension direction is not limited to the dashed line 5LA, but any indication that shows the target extension direction of the traveling body 5 is acceptable.

[0084] Upon seeing the dashed arrow on the monitor 40, the operator changes the extension direction L5 of the vehicle 5 to extension direction L5A, for example, by operating only one of the pair of tracks 5Cr. In the above description, an indicator showing an instruction to make the extension direction L5 of the vehicle 5 parallel to the axial direction L2 of the first pin 31 was explained, but an indicator showing an instruction to make the extension direction L5 of the vehicle 5 perpendicular to the axial direction L2 of the first pin 31 may also be used.

[0085] By recognizing the above operation instructions in the guidance image, the operator can understand the operation of the traveling body 5 and the rotating body 3 to align the coupler 8 directly with the attachment 30, thereby enabling more efficient attachment and detachment of the attachment 30.

[0086] Furthermore, the remote controller for remote operation may display the captured image from the camera and the guidance image simultaneously, or it may switch between them.

[0087] <Effects>

[0088] Next, the effects of this embodiment will be described.

[0089] When attaching or detaching the attachment 30 shown in Figure 2, it is necessary to move the work machine 2 and the slewing body 3, but it is difficult to grasp the positional relationship of the attachment 30 in the depth direction. In addition, the gripping part 8b of the coupler 8 may be hidden from view by the attachment 30. For this reason, it is difficult to align the gripping part 8b of the coupler 8 with the first pin 31 of the attachment 30 when attaching or detaching the attachment 30. In particular, when attaching or detaching by remote control, the operator must perform the operation using only camera information, making the operation even more difficult. For this reason, there was a request for a way to efficiently attach the attachment 30.

[0090] In this embodiment, as shown in Figure 5 and other figures, the guidance image displayed on the monitor 40 includes a display that allows the operator to recognize the relative position between the gripping portion 8b of the coupler 8 and the first pin 31 of the attachment 30. This allows the operator to attach and detach the attachment 30 to the coupler 8 while confirming the relative position between the gripping portion 8b and the first pin 31 using the guidance image on the monitor 40. Therefore, it is possible to further improve the efficiency of attaching and detaching the attachment 30 not only when the operator is on board the hydraulic excavator 100 to perform the attachment and detachment operation, but also when the operator performs the attachment and detachment operation remotely.

[0091] Furthermore, in this embodiment, as shown in Figure 3, the controller 50 selects a candidate attachment corresponding to the image of the attachment from among a list of candidate attachments stored in the memory 50e, and outputs a guidance image to the monitor 40 based on the selected candidate attachment. This allows the controller 50 to accurately grasp the shape, dimensions, etc. of the attachment 30 from the image of the attachment 30 detected by the external sensor 20. As a result, the precise relative position between the gripping part 8b and the first pin 31 can be displayed on the monitor 40.

[0092] Furthermore, in this embodiment, as shown in Figure 7, the guidance image includes a display indicating the target position of the arm 7 when attaching or detaching the attachment 30. Guidance is performed with the position of the arm top pin 15 as the target position. Therefore, by aligning the current position of the arm 7 with the target position of the arm 7 in the guidance image, it is possible to align the position of the arm top pin 15 with the target position.

[0093] Furthermore, in this embodiment, as shown in Figure 7, the guidance image includes a display (solid arrow in the figure) indicating a rotation instruction for the coupler 8. The rotation instruction for the coupler 8 is an instruction to rotate the coupler 8 around the arm top pin 15. As described above, after aligning the position of the arm top pin 15 to the target position, the first pin 31 can be inserted into the gripping portion 8b by rotating the coupler 8 according to the rotation instruction for the coupler 8. By operating based on the target position of the arm 7 and the rotation instruction for the coupler 8 displayed in the guidance image in this way, it becomes possible to further improve the efficiency of attaching and detaching the attachment 30.

[0094] Furthermore, in this embodiment, as shown in Figure 9, the controller 50 obtains the relative angle between the hydraulic excavator 100 and the attachment 30 from an image of the attachment 30 and outputs a guidance image to the monitor 40 that includes instructions to operate the slewing body 3 so that the relative angle becomes smaller. As a result, the operator can easily reduce the relative angle between the hydraulic excavator 100 and the attachment 30 by following the instructions in the guidance image, thereby enabling further efficiency in attaching and detaching the attachment 30.

[0095] Furthermore, in this embodiment, as shown in Figure 9, the guidance image includes an instruction to move the traveling body 5. This makes it possible to move the hydraulic excavator 100 to a position where the coupler 8 faces the attachment 30 directly when the rotating body 3 is rotated.

[0096] <Note>

[0097] The above description includes the following features.

[0098] (Note 1) A sensor that acquires sensor data related to a detachable attachment to a coupler, Monitor and, The system includes a controller that outputs a guidance image to the monitor based on the sensor data, The guidance image is a display system that includes a display that allows recognition of the relative position between the gripping portion of the coupler and the pins of the attachment that are attached to and detached from the gripping portion.

[0099] (Note 2) The display system as described in Appendix 1, wherein the controller selects a candidate attachment corresponding to the sensor data from among a plurality of candidate attachments included in a pre-stored list, and outputs the guidance image to the monitor based on the selected candidate attachment.

[0100] (Note 3) The controller selects the candidate attachment based on information related to the shape of the candidate attachment, as described in Appendix 2.

[0101] (Note 4) The aforementioned coupler is attached to the work machine, including the arm. The guidance image includes a display indicating the target position of the arm when attaching or detaching the attachment, as described in any one of the appendices 1 to 3.

[0102] (Note 5) The guidance image includes a display indicating a rotation instruction for the coupler, as described in any one of the displays described in Appendix 1 to Appendix 4.

[0103] (Note 6) The coupler is attached to the work mechanism of a work machine having a rotating body. The aforementioned relative position includes the relative angle between the work machine and the attachment. The display system according to any one of Appendix 1 to Appendix 5, wherein the controller obtains the relative angle from the sensor data and outputs the guidance image to the monitor, which includes an instruction to operate the rotating body so that the relative angle becomes smaller.

[0104] (Note 7) The coupler is attached to the work implement of a work machine having a traveling body. The guidance image is a display system according to any one of the appendices 1 to 5, which includes instructions to drive the vehicle.

[0105] (Note 8) The aforementioned work machine has a rotating body to which the work machine is attached, The display system according to Appendix 7, wherein the guidance image includes displays indicating an instruction to move the vehicle and an instruction to rotate the rotating body, as instructions to face the coupler directly toward the attachment.

[0106] (Note 9) The display system according to Appendix 7, wherein the guidance image includes a display indicating an instruction to operate the traveling body such that the extending direction of the traveling body and the axial direction of the pin are perpendicular or parallel to each other, as an instruction to face the coupler directly toward the attachment.

[0107] (Note 10) The display system according to any one of the appendices 1 to 9, wherein the sensor data includes an image of the attachment.

[0108] (Note 11) A display method that displays images on a monitor to assist in attaching an attachment to a coupler, The steps include acquiring sensor data related to the attachment, The system includes the step of outputting a guidance image to the monitor based on the acquired sensor data, The guidance image is a display method that includes a display that allows recognition of the relative position between the gripping portion of the coupler and the pin of the attachment that is attached to and detached from the gripping portion.

[0109] (Note 12) The process further includes the step of selecting a candidate attachment that corresponds to the sensor data from among a list of candidate attachments included in a pre-stored list. The display method described in Appendix 11, wherein the guidance image is output to the monitor based on the selected candidate attachment.

[0110] (Note 13) The aforementioned coupler is attached to the work machine, including the arm. The display method according to Appendix 11 or Appendix 12, wherein the guidance image includes a display indicating the target position of the arm when attaching or detaching the attachment.

[0111] (Note 14) The coupler is attached to the work mechanism of a work machine having a rotating body. The aforementioned relative position includes the relative angle between the work machine and the attachment. The step further includes obtaining the relative angle from the sensor data, The display method according to Appendix 11 or Appendix 12, wherein the guidance image, which includes an instruction to operate the rotating body so that the relative angle becomes smaller, is output to the monitor.

[0112] (Note 15) The coupler is attached to the work implement of a work machine having a traveling body. The guidance image is displayed using the method described in Appendix 11 or Appendix 12, which includes instructions to drive the vehicle.

[0113] (Note 16) The display method described in any one of appendices 11 to 15, wherein the sensor data includes an image of the attachment.

[0114] (Note 17) Steps include acquiring sensor data related to the attachment, The steps include outputting a guidance image to a monitor based on the acquired sensor data, which includes a display that allows recognition of the relative position between the gripping portion of the coupler and the pins of the attachment that are attached to and detached from the gripping portion, A non-volatile storage medium that stores programs to be executed by the controller's processor.

[0115] (Note 18) The system stores a program that causes the processor of the controller to perform the step of selecting a candidate attachment that corresponds to the sensor data from among a list of candidate attachments that are stored in advance. The guidance image is output to the monitor based on the selected candidate attachment, and is a non-volatile storage medium as described in Appendix 17.

[0116] (Note 19) The aforementioned coupler is attached to the work machine, including the arm. The non-volatile storage medium according to Appendix 17 or Appendix 18, wherein the guidance image includes a display indicating the target position of the arm when attaching or detaching the attachment.

[0117] (Note 20) The coupler is attached to the work mechanism of a work machine having a rotating body. The aforementioned relative position includes the relative angle between the work machine and the attachment. The program is stored in which the processor of the controller performs the step of obtaining the relative angle from the sensor data. The guidance image includes instructions to move the rotating body so that the relative angle becomes smaller, in the non-volatile storage medium as described in Appendix 17 or Appendix 18.

[0118] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0119] 1 Main body, 2 Work equipment, 3 Slewing body, 4 Cab, 4S Driver's seat, 5 Underground body, 5Cr Tracks, 5M Travel motor, 6 Boom, 7 Arm, 8 Coupler, 8a Coupler body, 8b Gripping part, 8c Locking pin, 8d Mounting surface, 10 Boom cylinder, 11 Arm cylinder, 12 Attachment cylinder, 13 Boom foot pin, 14 Boom top pin, 15 Arm top pin, 16 Link pin, 17 Link, 20 External sensor, 21 Attitude sensor, 22 Slewing angle sensor, 23a, 23b GNSS antenna, 24 Position and direction sensor, 30 Attachment, 40 Monitor, 50 Controller, 50a Acquisition unit, 50b Selection unit, 50c Image generation unit, 50d Output control unit, 50e Memory, 60 Target position, 100 Hydraulic excavator.

Claims

1. A sensor that acquires sensor data related to a detachable attachment to a coupler, Monitor and, The system includes a controller that outputs a guidance image to the monitor based on the sensor data, The guidance image is a display system that includes a display that allows recognition of the relative position between the gripping portion of the coupler and the pins of the attachment that are attached to and detached from the gripping portion.

2. The display system according to claim 1, wherein the controller selects a candidate attachment corresponding to the sensor data from among a plurality of candidate attachments included in a pre-stored list, and outputs the guidance image to the monitor based on the selected candidate attachment.

3. The display system according to claim 2, wherein the controller selects the candidate attachment based on information relating to the shape of the candidate attachment.

4. The aforementioned coupler is attached to the work machine, including the arm. The display system according to claim 1, wherein the guidance image includes a display indicating the target position of the arm when attaching or detaching the attachment.

5. The display system according to claim 1, wherein the guidance image includes a display indicating a rotation instruction for the coupler.

6. The coupler is attached to the work mechanism of a work machine having a rotating body. The aforementioned relative position includes the relative angle between the work machine and the attachment. The display system according to claim 1, wherein the controller obtains the relative angle from the sensor data and outputs the guidance image to the monitor, which includes an instruction to operate the rotating body so that the relative angle becomes smaller.

7. The coupler is attached to the work implement of a work machine having a traveling body. The display system according to claim 1, wherein the guidance image includes instructions to drive the vehicle.

8. The aforementioned work machine has a rotating body to which the work machine is attached, The display system according to claim 7, wherein the guidance image includes displays indicating an instruction to move the traveling body and an instruction to rotate the rotating body, as instructions to face the coupler directly toward the attachment.

9. The display system according to claim 7, wherein the guidance image includes a display indicating an instruction to operate the traveling body such that the extending direction of the traveling body and the axial direction of the pin are perpendicular or parallel to each other, as an instruction to face the coupler directly toward the attachment.

10. The display system according to claim 1, wherein the sensor data includes an image of the attachment.

11. A display method that displays images on a monitor to assist in attaching an attachment to a coupler, The steps include acquiring sensor data related to the attachment, The system includes the step of outputting a guidance image to the monitor based on the acquired sensor data, The guidance image is a display method that includes a display that allows recognition of the relative position between the gripping portion of the coupler and the pin of the attachment that is attached to and detached from the gripping portion.

12. The process further includes the step of selecting a candidate attachment that corresponds to the sensor data from among a list of candidate attachments included in a pre-stored list. The display method according to claim 11, wherein the guidance image is output to the monitor based on the selected candidate attachment.

13. The aforementioned coupler is attached to the work machine, including the arm. The display method according to claim 11, wherein the guidance image includes a display indicating the target position of the arm when attaching or detaching the attachment.

14. The coupler is attached to the work mechanism of a work machine having a rotating body. The aforementioned relative position includes the relative angle between the work machine and the attachment. The step further includes obtaining the relative angle from the sensor data, The display method according to claim 11, wherein the guidance image, which includes an instruction to operate the rotating body so that the relative angle becomes smaller, is output to the monitor.

15. The coupler is attached to the work implement of a work machine having a traveling body. The display method according to claim 11, wherein the guidance image includes an instruction to drive the vehicle.

16. The display method according to claim 11, wherein the sensor data includes an image of the attachment.

17. Steps include acquiring sensor data related to the attachment, The steps include outputting a guidance image to a monitor based on the acquired sensor data, which includes a display that allows recognition of the relative position between the gripping portion of the coupler and the pins of the attachment that are attached to and detached from the gripping portion, A non-volatile storage medium that stores programs to be executed by the controller's processor.

18. The system stores a program that causes the processor of the controller to perform the step of selecting a candidate attachment that corresponds to the sensor data from among a list of candidate attachments that are stored in advance. The non-volatile storage medium according to claim 17, wherein the guidance image is output to the monitor based on the selected candidate attachment.

19. The aforementioned coupler is attached to the work machine, including the arm. The non-volatile storage medium according to claim 17, wherein the guidance image includes a display indicating the target position of the arm when attaching or detaching the attachment.

20. The coupler is attached to the work mechanism of a work machine having a rotating body. The aforementioned relative position includes the relative angle between the work machine and the attachment. The program is stored in which the processor of the controller performs the step of obtaining the relative angle from the sensor data. The non-volatile storage medium according to claim 17, wherein the guidance image includes an instruction to move the rotating body so that the relative angle becomes smaller.

Images