Work Support Device

Abstract

To provide images useful for operators who perform various tasks using work machines from an air vehicle.SOLUTION: The present invention discloses a work support device that controls the positional relationship between a work machine and an air vehicle equipped with an image pickup device. The work machine has a lower traveling body, an upper swivel body mounted on the lower traveling body, a work device rotatably provided on the upper swivel body, and a display device for displaying an image captured by the image pickup device. The work support device is capable of communicating with the air vehicle, and has a mode switching unit for switching the operation mode related to the air vehicle between a plurality of modes. The plurality of modes include a first operation mode for acquiring an image capable of monitoring the surroundings of the work machine and a second operation mode for acquiring an image capable of supporting work by the work device.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a work assistance device. [Background technology]

[0002] There is known a technique for capturing images of a space that cannot be captured by an imaging device attached to an upper rotating body of a shovel, using an imaging device attached to an autonomous flying vehicle (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. WO2017 / 131194 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technologies, the target flight position of the aircraft is determined with the objective of eliminating blind spots for the operator of the excavator (i.e., for periphery monitoring), and is not determined from the perspective of providing a visual angle for supporting the operator in their work. When using an aircraft to monitor the periphery of a work machine, an image that includes the entire work machine and its surroundings is useful to the operator, and therefore a position directly above or behind the work machine is appropriate as the target flight position of the aircraft, whereas when supporting work such as demolition work, an image that includes the tip of the work implement and the work object is useful to the operator, and therefore a position to the side of the work machine is appropriate as the target flight position of the aircraft.

[0005] Therefore, in one aspect, the present invention aims to provide, from an air vehicle, images that are useful to operators who perform various tasks using a work machine. [Means for solving the problem]

[0006] According to one aspect, there is provided a work support device that controls a positional relationship between a work machine and an aircraft equipped with an imaging device, the work support device comprising: The work machine includes: a lower running body; an upper rotating body mounted on the lower traveling body; a working device rotatably provided on the upper rotating body; a display device that displays an image captured by the imaging device, the work assistance device is capable of communicating with the aircraft and has a mode switching unit that switches an operation mode related to the aircraft between a plurality of modes; A work assistance device is provided, wherein the plurality of modes include a first operation mode for acquiring the image capable of monitoring the periphery of the work machine, and a second operation mode for acquiring the image capable of assisting work by the work device. [Effects of the Invention]

[0007] According to one aspect, the present invention makes it possible to provide, from an air vehicle, images that are useful to operators who perform various tasks using a work machine. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is an explanatory diagram relating to the configuration of a work machine and an unmanned aerial vehicle. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration related to a control system of a work machine. [Figure 2A] FIG. 2 is a schematic explanatory diagram of an operating system of the work machine. [Figure 3] FIG. 2 is a diagram illustrating functions realized by various control devices. [Figure 4] FIG. 10 is an explanatory diagram of a method for calculating the orientation of a lower traveling body. [Figure 4A] FIG. 2 is a schematic explanatory diagram of a first operation mode and a second operation mode. [Figure 5] FIG. 10 is an explanatory diagram of a method for setting the latitude and longitude of a target flight position in the lower tracking mode. [Figure 6A]4 is a schematic flowchart showing an example of processing executed by a flight control device in relation to setting a target flight position. [Figure 6B] 10 is a schematic flowchart illustrating an example of a mode switching process. [Figure 6C] 10 is a schematic flowchart showing an example of a target flight position / attitude setting process. [Figure 6D] 10 is a schematic flowchart showing an example of a target flight position / attitude calculation process for a first operation mode. [Figure 6E] 10 is a schematic flowchart showing an example of a target flight position / attitude calculation process for a second operation mode. [Figure 7] FIG. 10 is an explanatory diagram of a mode transition process. DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment will be described in detail below with reference to the accompanying drawings.

[0010] Fig. 1 is an explanatory diagram relating to the configuration of a work machine 1 and an unmanned airplane 40 according to one embodiment. In addition to the work machine 1 and the unmanned airplane 40, Fig. 1 also shows a flight control device 50 (an example of a work support device) and a remote control device 52.

[0011] The work machine 1 carries out predetermined work in cooperation with the unmanned aerial vehicle 40. The work machine 1 is a construction machine equipped with a crusher 145 suitable for demolition work, for example, and is equipped with a crawler-type undercarriage 110, an upper rotating body 120 that is rotatably mounted on the undercarriage 110 via a rotating mechanism 130, and a work mechanism 140. A cab (operator's compartment) 122 is provided on the front left side of the upper rotating body 120. The work mechanism 140 is provided in the front center of the upper rotating body 120, and the crusher 145 is provided at the tip of the work mechanism 140. The work machine 1 may also be equipped with a basement from which the unmanned aerial vehicle 40 takes off and arrives.

[0012] The working mechanism 140 includes a boom 141 that is mounted on the upper rotating body 120 so that it can be raised and lowered, an arm 143 that is rotatably connected to the tip of the boom 141, and a crusher 145 that is attached to the tip of the arm 143. The working mechanism 140 is equipped with a boom cylinder 142, an arm cylinder 144, and a bucket cylinder 146 that are formed by extendable and retractable hydraulic cylinders. Instead of the crusher 145, another tip attachment such as a bucket may be attached to the tip of the arm 143.

[0013] The boom cylinder 142 is interposed between the boom 141 and the upper rotating body 120 so as to extend and retract by receiving a supply of hydraulic oil to rotate the boom 141 in the hoisting direction. The arm cylinder 144 is interposed between the arm 143 and the boom 141 so as to extend and retract by receiving a supply of hydraulic oil to rotate the arm 143 about a horizontal axis relative to the boom 141. The bucket cylinder 146 is interposed between the crusher 145 and the arm 143 so as to extend and retract by receiving a supply of hydraulic oil to rotate the crusher 145 about a horizontal axis relative to the arm 143. The crusher cylinder 147 is provided on the crusher 145 so as to extend and retract by receiving a supply of hydraulic oil to open and close the crusher 145.

[0014] Unmanned airplane 40 is a rotorcraft equipped with a plurality of blades (for example, 4, 6, or 8), a battery that supplies power to an electric motor (actuator) for rotating the plurality of blades, etc. Instead of or in addition to such a battery, unmanned airplane 40 may be connected to a power supply line from the ground.

[0015] The unmanned airplane 40 includes a control device 400 and an imaging device 410 .

[0016] The control device 400 realizes various flight states (forward, backward, ascending, descending, hovering, etc.) of the unmanned airplane 40 in accordance with control information (commands) from the flight control device 50 (described later) and operation information from the remote control device 52. The control device 400 also transmits an image (forward environment image) acquired by the imaging device 410 to the work machine 1.

[0017] The imaging device 410 includes a camera. The type of camera is arbitrary, and may be, for example, a wide-angle camera. The imaging device 410 may be removably attached to the unmanned airplane 40, or may be firmly fixed to the unmanned airplane 40. The imaging device 410 acquires an image of the forward environment in front of the body of the unmanned airplane 40 using an imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The imaging device 410 may, for example, acquire an image of the forward environment in real time and supply the image to the control device 400 in a stream format at a predetermined frame rate.

[0018] The imaging device 410 preferably includes a gimbal (not shown), which functions to keep the optical axis of the imaging device 410 in a fixed direction (for example, a predetermined direction in a horizontal plane) even if the attitude of the unmanned aerial vehicle 40 changes.

[0019] The flight control device 50 executes various controls of the unmanned airplane 40. In one embodiment, the flight control device 50 is realized by a server (server computer). In this case, the flight control device 50 is connected to the work machine 1 and the unmanned airplane 40 via a network (not shown). In this case, the network may include a wireless communication network, the Internet, a Virtual Private Network (VPN), a Wide Area Network (WAN), a wired network, or any combination thereof. In another embodiment, the flight control device 50 may be realized by the control device 10 (see FIG. 2 ) of the work machine 1 (i.e., the flight control device 50 may be provided in the work machine 1). In another embodiment, the flight control device 50 may be realized by the control device 400 of the unmanned airplane 40. Alternatively, in another embodiment, the functions of the flight control device 50 may be realized by a combination of any two or all of the server, the control device 10, and the control device 400 working together. Details of the flight control device 50 will be described later.

[0020] The remote operation device 52 may take the form of, for example, a remote controller, and may be operated by a user (for example, the operator of the work machine 1 or a worker other than the operator). If the user is the operator of the work machine 1, the remote operation device 52 may be brought into the cab 122. The remote operation device 52 is capable of wireless communication with the unmanned airplane 40, and transmits an operation signal to the unmanned airplane 40 in response to an operation by the user. In this case, when the unmanned airplane 40 receives operation information from the remote operation device 52, the control device 400 of the unmanned airplane 40 realizes the movement of the unmanned airplane 40 (forward, backward, up and down, etc.) in response to the operation information. In a modified example, the remote operation device 52 may be omitted. The remote operation device 52 may also be realized by a smartphone or the like.

[0021] FIG. 2 is a diagram showing an example of the hardware configuration of the control system of the work machine 1. As shown in FIG.

[0022] As shown in FIG. 2, the work machine 1 includes an electrical system device 8 and a control device 10.

[0023] The electrical system 8 includes electronically controllable devices and various sensors mounted on the work machine 1. The electrical system 8 may include, for example, an image output device 80 (an example of a display device), a buzzer, an audio output device (not shown), a hydraulic generating device (see FIG. 2A) that operates the swing mechanism 130, the working mechanism 140, the lower traveling structure 110, etc., and various sensors 82 that detect the operating states of various operating members. The electrical system 8 (particularly the various sensors 82) is basically provided in the upper rotating structure 120. This is because the lower traveling structure 110 is more likely to be exposed to the external environment than the upper rotating structure 120.

[0024] The hydraulic pressure generating device may include a hydraulic pump (see main pump 914 in FIG. 2A) driven by an engine and / or an electric motor. When a hydraulic pump driven by an electric motor is used, the hydraulic pressure generating device may further include an inverter for driving the electric motor.

[0025] The various sensors 82 may include a gyro sensor, various angle sensors, an acceleration sensor (tilt sensor), a hydraulic sensor (see pressure sensor 929 in FIG. 2A) that detects hydraulic pressure at a predetermined location in a hydraulic line (see high-pressure hydraulic line 916 in FIG. 2A) applied by a hydraulic pressure generating device, and the like.

[0026] In this embodiment, the sensors 82 particularly include a GPS (Global Positioning System) compass 821 (an example of a satellite radio wave receiver). The GPS compass 821 includes two antennas (not shown) mounted on the upper rotating body 120. The two antennas are positioned apart from each other and each receive radio waves from a satellite. The satellite signals received by each of the two antennas are converted into position information (positioning information) by performing positioning processing using, for example, an interferometric positioning method. Then, the direction is calculated based on the position information of each antenna. In this way, the GPS compass 821 calculates the direction (azimuth) based on the relative positional relationship between the two antennas. Note that instead of the GPS compass 821, a similar compass that uses a GNSS (Global Navigation Satellite System) other than GPS may be used.

[0027] The image output device 80 (see FIG. 1) is provided inside the cab 122 so as to be visible to the operator of the work machine 1. The image output device 80 is optional, and may be, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. In a modified example, the image output device 80 may be a portable device (for example, a tablet terminal) that can be brought into the cab 122 by the operator of the work machine 1.

[0028] The control device 10 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, an auxiliary storage device 14, a drive device 15, and a communication interface 17, all connected by a bus 19, as well as a wired transceiver unit 25 and a wireless transceiver unit 26 connected to the communication interface 17.

[0029] The auxiliary storage device 14 is, for example, a hard disk drive (HDD) or a solid state drive (SSD), and is a storage device that stores data related to application software and the like.

[0030] The wired transceiver 25 includes a transceiver capable of communicating using a wired network. The electrical equipment 8 is connected to the wired transceiver 25. However, some or all of the electrical equipment 8 may be connected to the bus 19 or to the wireless transceiver 26.

[0031] The wireless transceiver 26 is a transceiver capable of communicating using a wireless network. The wireless network may include a wireless communication network for mobile phones, the Internet, VPN, WAN, etc. The wireless transceiver 26 may also include a near field communication (NFC) unit, a Bluetooth (registered trademark) communication unit, a Wi-Fi (Wireless-Fidelity) transceiver, an infrared transceiver, etc. For example, the wireless transceiver 26 can communicate with a flight control device 50 in the form of a server.

[0032] The control device 10 may be connectable to a recording medium 16. The recording medium 16 stores a predetermined program. The program stored in the recording medium 16 is installed in the auxiliary storage device 14 of the control device 10 via the drive device 15. The installed predetermined program can be executed by the CPU 11 of the control device 10. For example, the recording medium 16 may be a recording medium that records information optically, electrically, or magnetically, such as a CD (Compact Disc)-ROM, a flexible disk, or a magneto-optical disk, or a semiconductor memory that records information electrically, such as a ROM or a flash memory. The recording medium 16 does not include a carrier wave.

[0033] Here, the control system of the work machine 1 has been described with reference to Figure 2, but the control system of the unmanned airplane 40 may be substantially similar, except for the configuration related to the electrical equipment device 8. For example, in the case of the control system of the unmanned airplane 40, the hardware configuration of the control device 400 may be similar to that of the control device 10. Furthermore, the electrical equipment device corresponding to the electrical equipment device 8 includes an imaging device 410 (see Figure 1) and various sensors.

[0034] The hardware configuration of the flight control device 50 may also be substantially the same as the hardware configuration of the control device 10 shown in FIG.

[0035] Fig. 2A is a schematic explanatory diagram relating to the operating system of the work machine 1. In Fig. 2A, high-pressure hydraulic lines are indicated by solid lines, and pilot lines are indicated by dashed lines.

[0036] The main pump 914 is a pump that generates hydraulic pressure to be supplied to the control valve 917. This hydraulic pressure is supplied via the control valve 917 to drive the hydraulic motors 110A, 110B, the boom cylinder 142, the arm cylinder 144, and the cylinder 146.

[0037] The pilot pump 915 is a pump that generates the pilot pressure required for the hydraulic operating system.

[0038] The control valve 917 and a pressure sensor 929 are connected to the operating device 926 via hydraulic lines 927 and 928. The pressure sensor 929 is connected to the control device 10 described above, which controls the drive of the electrical system of the work machine 1.

[0039] Hydraulic motors 110A (for the right) and 110B (for the left) for the undercarriage 110, a boom cylinder 142, an arm cylinder 144, and a cylinder 146 are connected to the control valve 917 via high-pressure hydraulic lines 916.

[0040] The control valve 917 controls the hydraulic pressure supplied to each of the hydraulic motors 110A, 110B, boom cylinder 142, arm cylinder 144, and cylinder 146 for the undercarriage 110, which are connected via a high-pressure hydraulic line 916, in accordance with the operator's operational input.

[0041] The operating device 926 is an operating device for operating the lower traveling structure 110, the boom 141, the arm 143, and the crusher 145. The operating device 926 includes levers 926A and 926B and a lever 926C. The lever 926A is a lever for operating the arm 143, and the lever 926B is a lever for operating the boom 141 and the crusher 145. The lever 926C is a pair of levers for operating the lower traveling structure 110. Specifically, the lever 926C includes a left lever for driving and rotating the left crawler, and a right lever for driving and rotating the right crawler. The levers 926C may be in the form of a pair of pedals provided under the driver's seat.

[0042] The operating device 926 converts the hydraulic pressure (primary hydraulic pressure) supplied through the pilot line 925 into a hydraulic pressure (secondary hydraulic pressure) corresponding to the amount of operation by the driver and outputs the converted hydraulic pressure. The secondary hydraulic pressure output from the operating device 926 is supplied to the control valve 917 through a hydraulic line 927 and is detected by a pressure sensor 929.

[0043] When levers 926A and 926B and lever 926C are operated, control valve 917 is driven through hydraulic line 927, which controls the hydraulic pressure in hydraulic motors 110A and 110B, boom cylinder 142, arm cylinder 144, and cylinder 146, thereby driving lower running body 110, boom 141, arm 143, and crusher 145.

[0044] The hydraulic line 927 supplies the hydraulic pressure required to drive the hydraulic motors 110A and 110B, the boom cylinder 142, the arm cylinder 144, and the cylinder 146 to the control valve 917.

[0045] The pressure sensor 929 detects a change in the hydraulic pressure in the hydraulic line 928 due to the operation of the lever 926C. The pressure sensor 929 outputs an electric signal representing the hydraulic pressure in the hydraulic line 928. This electric signal includes a signal representing the operation mode of the lever 926C (the amount and direction of operation of each of the left and right levers), and is input to the control device 10. The operation mode of the lever 926C may be detected magnetically or optically.

[0046] Next, the flight control device 50 will be described in detail with reference to FIG. 3 and subsequent figures, along with the control device 10 and the control device 400.

[0047] FIG. 3 is a diagram illustrating functions realized by the control device 10, the control device 400, and the flight control device 50. FIG. 3 mainly illustrates functions related to hovering maintenance control, which will be described later. Therefore, the control device 10 and the like may be provided with other functions other than those illustrated, as appropriate. FIG. 4 is an explanatory diagram illustrating a method for calculating the orientation of the undercarriage 110, and is a diagram schematically illustrating the upper rotating body 120 and the lower running body 110 as viewed from above. In FIG. 4, the left side shows a state in which the upper rotating body 120 is not rotating (hereinafter referred to as the "neutral state"), and the right side shows a state in which the upper rotating body 120 has rotated clockwise by an angle α (hereinafter referred to as the "rotating state"). FIG. 4A is a schematic explanatory diagram of a first operating mode and a second operating mode.

[0048] 3, the control device 10 includes a position information acquisition unit 150, an attitude information acquisition unit 151, an orientation information acquisition unit 152, a communication processing unit 153, an image output processing unit 154, an operation information acquisition unit 155, and an actuator control unit 156. Each functional unit such as the position information acquisition unit 150 can be realized by the CPU 11 shown in FIG. 2 executing a program in a storage device such as the ROM 13 shown in FIG. 2.

[0049] The position information acquisition unit 150 acquires position information of the work machine 1 from the GPS compass 821 of the sensors 82. The position information of the work machine 1 is expressed in terms of latitude, longitude, and altitude. The GPS sensor includes a GPS receiver, and calculates the latitude, longitude, and altitude by interferometric positioning or the like based on radio waves from satellites.

[0050] The attitude information acquisition unit 151 acquires attitude information of the work machine 1 based on various sensors from the sensors 82 that acquire parameters related to the attitude of the work machine 1. In this case, the various sensors that acquire parameters related to the attitude may be, for example, a boom angle sensor, an arm angle sensor, a bucket angle sensor, a machine body inclination sensor, etc. The boom angle sensor is a sensor that acquires the boom angle, and includes, for example, a rotation angle sensor that detects the rotation angle of the boom foot pin, a stroke sensor that detects the stroke amount of the boom cylinder 142, and a tilt (acceleration) sensor that detects the tilt angle of the boom 141. The same applies to the arm angle sensor and the bucket angle sensor. The machine body inclination sensor is a sensor that acquires the machine body inclination angle, and detects, for example, the inclination angle of the upper rotating body 120 with respect to the horizontal plane.

[0051] The orientation information acquisition unit 152 acquires information relating to the orientation (orientation around the vertical axis) of the work machine 1. In this embodiment, the orientation information acquisition unit 152 includes an upper rotating structure orientation calculation unit 1520 and a lower traveling structure orientation calculation unit 1521.

[0052] The upper rotating structure orientation calculation unit 1520 calculates the orientation of the upper rotating structure 120 based on the orientation calculation result obtained from the GPS compass 821 of the sensors 82. As described above, the GPS compass 821 is provided on the upper rotating structure 120 and therefore rotates together with the upper rotating structure 120. Therefore, the orientation (azimuth) detected by the GPS compass 821 correlates with the orientation of the upper rotating structure 120. In this embodiment, as an example, the GPS compass 821 calculates the forward orientation along the central axis L0 (see FIG. 4 ) of the upper rotating structure 120 with the boom 141 as the reference, as the orientation of the upper rotating structure 120. In other words, the central axis L0 is the direction of the baseline. The calculation function of the upper rotating structure orientation calculation unit 1520 may be realized by the GPS compass 821.

[0053] The lower traveling structure orientation calculation unit 1521 calculates the orientation of the lower traveling structure 110 based on swing angle information from a rotation angle sensor 822 among the sensors 82. The rotation angle sensor 822 detects the swing angle around the swing axis of the swing mechanism 130 (the relative swing angle around the swing axis between the upper rotating structure 120 and the lower traveling structure 110). The swing angle detected by the rotation angle sensor 822 is the rotation angle from a predetermined reference position (zero point). In this embodiment, as an example, the predetermined reference position is a position where the central axis L0 of the upper rotating structure 120 and the central axis L1 of the lower traveling structure 110 overlap. The central axis L1 of the lower traveling structure 110 is perpendicular to the rotation axes of the left and right crawlers. For example, in the state on the right side of FIG. 4 , the central axis L0 of the upper rotating structure 120 forms an angle α greater than 0 degrees with respect to the central axis L1 of the lower traveling structure 110. In this case, the rotation angle sensor 822 detects the angle α. Hereinafter, the rotation angle around the rotation axis of the rotation mechanism 130 refers to the angle α that the central axis L0 forms with respect to the central axis L1.

[0054] The lower running structure orientation calculation unit 1521 calculates the orientation of the lower running structure 110 by correcting the orientation of the upper rotating structure 120 by an amount corresponding to the rotation angle around the rotation axis of the rotation mechanism 130. For example, in the state on the right side of Fig. 4, the central axis L0 of the upper rotating structure 120 forms an angle α with the central axis L1 of the lower running structure 110. In this case, when the orientation (azimuth) of the upper rotating structure 120 is due north, the orientation of the lower running structure 110 is calculated (corrected) to the west by an amount corresponding to the angle α.

[0055] Thus, in this embodiment, the orientation information acquired by the orientation information acquisition unit 152 includes information representing the orientation of the upper rotating body 120 calculated by the upper rotating body orientation calculation unit 1520 (hereinafter also referred to as "upper rotating body orientation information"), and information representing the orientation of the lower running body 110 calculated by the lower running body orientation calculation unit 1521 (hereinafter also referred to as "lower running body orientation information").

[0056] The communication processing unit 153 transmits various pieces of information acquired by the position information acquisition unit 150, the attitude information acquisition unit 151, and the orientation information acquisition unit 152 to the flight control device 50. For example, the communication processing unit 153 may transmit the latest information to the flight control device 50 at predetermined intervals in response to a request from the flight control device 50.

[0057] Furthermore, the communication processing unit 153 receives image data from the unmanned airplane 40. The image data is data of an image of the forward environment captured by the imaging device 410.

[0058] The image output processing unit 154 outputs a forward environment image onto the image output device 80 based on the image data acquired by the communication processing unit 153. This enables the operator of the work machine 1 to understand, for example, the situation of the work site that cannot be seen with a direct view, from the forward environment image on the image output device 80.

[0059] The operation information acquisition unit 155 acquires various pieces of operation information (operation input) input via the operation device 926.

[0060] The actuator control unit 156 generates various command values ​​based on various pieces of operation information input via the operation device 926. For example, the actuator control unit 156 may control the discharge pressure, discharge amount, etc. from the main pump 914 based on various pieces of operation information input via the operation device 926.

[0061] 3, the control device 400 includes an aircraft information acquisition unit 401, a target flight state setting unit 402, an aircraft control unit 403, and a communication processing unit 404. Each functional unit such as the aircraft information acquisition unit 401 can be realized by a CPU such as CPU 11 shown in FIG. 2 executing a program in a storage device such as ROM 13 shown in FIG. 2.

[0062] The aircraft information acquisition unit 401 acquires aircraft information that indicates various conditions related to the aircraft of the unmanned airplane 40. The aircraft information may include position information of the unmanned airplane 40, attitude information of the unmanned airplane 40, and the like. The position information of the unmanned airplane 40 may be expressed, for example, by latitude, longitude, and altitude. Note that such position information of the unmanned airplane 40 can be acquired from a GPS sensor. The attitude information of the unmanned airplane 40 may include information about the rotation of the unmanned airplane 40 around each of the yaw axis, roll axis, and pitch axis. Note that such attitude information of the unmanned airplane 40 can be acquired from a sensor such as an inertial measurement unit (IMU) mounted on the unmanned airplane 40.

[0063] The target flight state setting unit 402 sets a target flight state for the unmanned airplane 40 based on control information (commands) from the flight control device 50. The target flight state includes a target flight position and a target flight attitude. In this embodiment, the control information includes a target flight position and a target flight attitude as described below, and in this case, the target flight state setting unit 402 may use the target flight position and the target flight attitude as they are. However, in a modified example, as described above, the target flight state setting unit 402 may calculate at least one of the target flight position and the target flight attitude instead of the flight control device 50.

[0064] Aircraft control unit 403 controls various actuators (not shown) of unmanned airplane 40 so as to achieve the target flight state set by target flight state setting unit 402. The various actuators of unmanned airplane 40 include electric motors for rotating multiple blades.

[0065] The communication processing unit 404 transmits the aircraft information and the like acquired by the aircraft information acquisition unit 401 to the flight control device 50. For example, the communication processing unit 404 may transmit the latest aircraft information to the flight control device 50 at predetermined intervals in response to a request from the flight control device 50.

[0066] Furthermore, the communication processing unit 404 transmits data of the forward environment image captured by the imaging device 410 to the work machine 1. For example, the communication processing unit 404 may transmit data of the forward environment image to the flight control device 50 at predetermined intervals in response to a request from the flight control device 50.

[0067] 3, the flight control device 50 includes an information acquisition unit 510, a mode switching unit 511, a relative positional relationship determination unit 512, and a flight control unit 514. Each functional unit such as the information acquisition unit 510 can be realized by a CPU such as CPU 11 shown in FIG. 2 executing a program in a storage device such as ROM 13 shown in FIG. 2.

[0068] The information acquisition unit 510 acquires various pieces of information necessary for various controls of the flight control unit 514. In this embodiment, as an example, the information acquisition unit 510 acquires position information of the work machine 1, attitude information and orientation information related to the work mechanism 140, and aircraft information of the unmanned airplane 40. The attitude information is, for example, attitude information that can be used when deriving the position of the crusher 145, and represents, for example, the attitude of the arm 143, boom 141, etc. The information acquisition unit 510 also acquires operation information of the work machine 1 (particularly operation information related to the lever 926C) as appropriate. The position information, attitude information, and orientation information of the work machine 1 can be acquired by communication from the communication processing unit 153 of the control device 10 of the work machine 1. The aircraft information of the unmanned airplane 40 can be acquired from the communication processing unit 404 of the control device 400 of the unmanned airplane 40.

[0069] The mode switching unit 511 switches between a plurality of operating modes for the unmanned airplane 40. The plurality of modes include a first operating mode that acquires a forward environmental image that can monitor the periphery of the work machine 1, and a second operating mode that acquires a forward environmental image that can support work by the crusher 145 (an example of a work device). The first operating mode also includes an operating mode in which a target flight position, etc. is set based on the orientation of the lower running structure 110 (hereinafter also referred to as a "lower following mode"), and an operating mode in which a target flight position, etc. is set based on the orientation of the upper rotating structure 120 (hereinafter also referred to as an "upper following mode").

[0070] The mode switching unit 511 may switch the operation mode in response to a mode switching input that may be input by the user. In this case, the mode switching input may be any input such as an input by operating a button, a voice input, or a gesture input. When a mode switching input is received, the control device 10 transmits a mode switching request to the flight control device 50. The mode switching request may include a destination operation mode (for example, a first operation mode or a second operation mode). In this case, the mode switching unit 511 may switch the operation mode in response to the mode switching request.

[0071] Instead of or in addition to responding to the above-described mode switching input, the mode switching unit 511 may automatically switch the operation mode when a predetermined switching condition unrelated to the above-described mode switching input is satisfied. For example, the mode switching unit 511 may switch the operation mode based on various operation information (operation signals) generated by the operation device 926. Specifically, when an operation signal for traveling the lower traveling structure 110 is input, the mode switching unit 511 may switch to the first operation mode. This is because the forward environment image obtained in the first operation mode is useful when the work machine 1 is traveling. Furthermore, when an operation signal for the crusher 145 (tip attachment) is input, the mode switching unit 511 may switch to the second operation mode. This is because the forward environment image obtained in the second operation mode is useful when operating the crusher 145. Note that other operations include the operation of rotating the upper rotating structure 120, the operation of the boom 141, and the operation of the arm 143, and the situations in which these three operations are performed may be suitable for either the first operation mode or the second operation mode. Therefore, the operation mode may be maintained even when operation signals related to these three operations are input. In a configuration in which an automatic mode in which the operation mode is automatically switched based on such predetermined switching conditions and a manual mode in which the operation mode is switched based on a mode switching input by the user as described above can be formed, the user may be able to select between the automatic mode and the manual mode.

[0072] The relative positional relationship determination unit 512 ascertains (determines) the relative positional relationship of the unmanned airplane 40 with respect to the work machine 1, based on the positional information of the work machine 1 and the positional information of the unmanned airplane 40 (part of the aircraft information) acquired by the information acquisition unit 510. The relative positional relationship of the unmanned airplane 40 with respect to the work machine 1 may be a positional relationship that takes altitude into account (i.e., a three-dimensional positional relationship), or may be a positional relationship that does not take altitude into account. An example of a positional relationship that does not take altitude into account may be the positional relationship when projected onto a predetermined reference plane (e.g., the ground surface).

[0073] Furthermore, the relative positional relationship of the unmanned airplane 40 to the work machine 1 may be the relative positional relationship of the unmanned airplane 40 to a predetermined part of the work machine 1. In this case, the predetermined part may be a part of the undercarriage 110 of the work machine 1 (for example, a part near the center of gravity), a part of the upper rotating body 120, or a part of the rotating mechanism 130 (for example, the crusher 145). Furthermore, the predetermined part may be automatically changed depending on the operating mode, etc. Note that, hereinafter, the relative positional relationship of the unmanned airplane 40 to the work machine 1 will also be simply referred to as the "relative positional relationship."

[0074] The flight control unit 514 generates control information (commands) to be transmitted to the unmanned airplane 40 based on the various information acquired by the information acquisition unit 510. As described above, the control information is information for causing the target flight state setting unit 402 of the unmanned airplane 40 to set a target flight state.

[0075] When the hovering start condition is met, the flight control unit 514 generates control information so that the unmanned airplane 40 starts hovering. The hovering start condition may be met, for example, when the position of the unmanned airplane 40 reaches the target flight position. Whether the position of the unmanned airplane 40 has reached the target flight position can be determined based on the aircraft information acquired by the information acquisition unit 510, for example.

[0076] Flight control unit 514 may cause control device 400 to start hovering, for example, by not changing the control information (and therefore not changing the target flight position of unmanned airplane 40). Alternatively, flight control unit 514 may cause control device 400 to start hovering by sending a command to unmanned airplane 40 to start hovering (for example, control information instructing a mode such as a hovering mode).

[0077] In this embodiment, even if the relative positional relationship changes from a reference relative positional relationship while the unmanned airplane 40 is hovering, the flight control unit 514 may execute hovering maintenance control to maintain the hovering of the unmanned airplane 40 if the amount of change from the reference relative positional relationship is equal to or less than a predetermined threshold value Th1. In this case, the reference relative positional relationship may be the relative positional relationship at the time when the unmanned airplane 40 reaches the target flight position or before or after that, for example, the relative positional relationship when hovering starts. Alternatively, the reference relative positional relationship may be set by the user. In this case, for example, the reference relative positional relationship may be the relative positional relationship at the time when a predetermined input is made by the user. Whether the unmanned airplane 40 is hovering can be determined, for example, based on aircraft information acquired by the information acquisition unit 510.

[0078] Hereinafter, in contrast to this type of hovering maintenance control, control that dynamically changes the target flight position of the unmanned airplane 40 in response to changes in the relative positional relationship will also be referred to as "position tracking control."

[0079] In this embodiment, the flight control unit 514 includes a target flight position calculation unit 5141 and a target flight attitude calculation unit 5142. Based on the target flight position calculated by the target flight position calculation unit 5141 and the target flight attitude calculated by the target flight attitude calculation unit 5142, the flight control unit 514 generates control information including these.

[0080] In this embodiment, the target flight position is calculated differently depending on the operating mode.

[0081] Specifically, in the lower follow-up mode of the first operating mode, the target flight position (i.e., the target flight position for the lower follow-up mode) is set to a position behind the undercarriage 110. The target flight position may be expressed, for example, by latitude, longitude, and altitude. Figure 4A schematically shows the position (P2) of the unmanned airplane 40 in the first operating mode and the angle of view R2 of the imaging device 410 at that time.

[0082] In this case, the latitude and longitude associated with the target flight position are, for example, a position on the longitudinal axis of the undercarriage 110 of the work machine 1, and are set a predetermined distance D1 rearward from the rear of the undercarriage 110 of the work machine 1. The position on the longitudinal axis of the undercarriage 110 corresponds to the position on the central axis L1 of the undercarriage 110 described above when viewed from above. Therefore, the longitudinal axis of the undercarriage 110 can be identified from the above-mentioned undercarriage orientation information among the above-mentioned orientation information. The predetermined distance D1 may be, for example, approximately 20 m. The predetermined distance D1 may also be variable by the user. The latitude and longitude associated with the target flight position may also be set so as to be offset by a predetermined distance D2 in the lateral direction from the longitudinal axis of the undercarriage 110 of the work machine 1. The predetermined distance D2 may be, for example, approximately 20 m, so that the target flight position is approximately 45 degrees rearward from the work machine 1. In this case, the predetermined distance D2 may also be variable by the user.

[0083] Furthermore, in the upper tracking mode of the first operation mode, the latitude and longitude of the target flight position (i.e., the target flight position for the upper tracking mode) change in accordance with the rotation of the upper rotating body 120, and is set to a rear position on the central axis L0 of the upper rotating body 120 (rear of the rotation axis of the rotation mechanism 130). Alternatively, in the upper tracking mode, the target flight position may be set on the rotation axis of the rotation mechanism 130 (i.e., directly above the rotation mechanism 130).

[0084] The target flight position may be any position that allows the imaging device 410 to obtain an image of the forward environment that includes the entire work machine 1, and may be adapted according to the angle of view of the imaging device 410. In a modified example, the target flight position may be set on the rotation axis of the swing mechanism 130 of the work machine 1 (i.e., directly above the swing mechanism 130) or further forward, regardless of the lower follow mode or upper follow mode.

[0085] In the second operation mode, the latitude and longitude associated with the target flight position (i.e., the target flight position for the second operation mode) is a flight position where the tip of the crusher 145 is within the angle of view of the image capture device 410, and may be set, for example, to the side or in front of the work machine 1 when viewed from above. FIG. 4A schematically shows the position of the unmanned airplane 40 in the second operation mode (target flight position P1) and the angle of view R1 of the image capture device 410 at that time. Here, the position of the tip of the crusher 145 changes in accordance with the rotation of the upper rotating body 120 in addition to the movement of the arm 143, etc. Therefore, the target flight position in the second operation mode may be set in accordance with the movement of these various movable members. For example, the target flight position calculation unit 5141 may set the latitude and longitude of the target flight position at a predetermined distance D3 (not shown) in the horizontal direction perpendicular to the central axis L0 of the upper rotating body 120 in a top view, and at a predetermined distance D4 (not shown) forward of the rotation axis of the rotation mechanism 130. In this case as well, the predetermined distances D3 and D4 may be variable by the user.

[0086] Furthermore, the altitude associated with the target flight position may be constant (for example, within a range of 30 m to 40 m). However, the altitude associated with the target flight position may also be variable by the user, or may be changed automatically depending on the operation mode, etc. For example, in the second operation mode, the altitude may be changed based on the state of the boom 141 (for example, the boom hoisting angle) in such a way that the more the boom 141 is hoisted, the higher the altitude becomes.

[0087] The target flight attitude may be expressed, for example, by parameters relating to rotation around the yaw axis, roll axis, and pitch axis. The target flight attitude may be set, for example, so that the longitudinal axis of the unmanned airplane 40's body is parallel to the longitudinal axis of the undercarriage 110 of the work machine 1 and so that the longitudinal axis of the unmanned airplane 40's body is located in a horizontal plane. Similarly, the parameters relating to the target flight attitude may be variable by the user or may be changed automatically depending on the operation mode, etc. Alternatively, the target flight attitude may be set, for example, so that the longitudinal axis of the unmanned airplane 40's body is parallel to the extension direction of the arm 143 and so that the longitudinal axis of the unmanned airplane 40's body is located in a horizontal plane.

[0088] For example, the target flight attitude in the upper tracking mode of the first operation mode (i.e., the target flight attitude for the upper tracking mode) may be set so that the optical axis of the image capture device 410 is parallel to the central axis L0 of the upper rotating body 120 when viewed from above around the yaw axis. Also, the target flight attitude in the upper tracking mode may be set so that the optical axis of the image capture device 410 passes through the center of the upper rotating body 120 (the center along the central axis L0) when viewed from the side around the pitch axis.

[0089] Furthermore, the target flight attitude in the lower tracking mode of the first operating mode (i.e., the target flight attitude for the lower tracking mode) may be set so that the optical axis of the imaging device 410 is parallel to the central axis L1 of the lower running body 110 when viewed from above around the yaw axis. Furthermore, the target flight attitude in the lower tracking mode may be set so that the optical axis of the imaging device 410 passes near the center of the lower running body 110 (the center along the central axis L1) when viewed from the side around the pitch axis.

[0090] Furthermore, in the second operating mode, the target flight attitude may be set so that the crusher 145 fits within the angle of view of the image capture device 410. In this case, the target flight attitude may be set so that the crusher 145 fits approximately in the center of the angle of view of the image capture device 410. Furthermore, the target flight attitude may be set so that the crusher 145 and the work object (for example, in the case of demolition work, the object to be demolitioned) fit within the angle of view of the image capture device 410. For example, simply, the target flight attitude may be set so that the tip of the crusher 145 fits within the angle of view of the image capture device 410. In this case, the position of the tip of the crusher 145 may be calculated, and the target flight attitude of the unmanned airplane 40 may be calculated so that the optical axis of the image capture device 410 passes through the calculated position.

[0091] In addition, the target flight attitude around the roll axis may basically be set so that the body of the unmanned airplane 40 is approximately horizontal in any operation mode.

[0092] 5 is an explanatory diagram of an example of a method for setting the latitude and longitude associated with the target flight position in the bottom follow mode. In the bottom follow mode, the latitude and longitude associated with the target flight position are set to behind the undercarriage 110 of the work machine 1, as described above. Therefore, whether the central axis L0 of the upper rotating body 120 and the central axis L1 of the undercarriage 110 coincide with each other, as in the neutral state shown on the left side of FIG. 5, or whether the central axis L0 of the upper rotating body 120 and the central axis L1 of the undercarriage 110 do not coincide with each other, as in the turning state shown on the right side of FIG. 5, the latitude and longitude associated with the target flight position will similarly correspond to behind the undercarriage 110.

[0093] In this way, in the case of lower follow-up mode in this embodiment, as described above, the latitude and longitude for the target flight position are set behind the undercarriage 110 of the work machine 1, so that a forward environmental image from behind on the central axis L1 of the undercarriage 110 (a forward environmental image from the imaging device 410 of the unmanned airplane 40) can be stably provided from the unmanned airplane 40. In this case, by appropriately setting the target flight attitude as described above, a useful forward environmental image capturing the work machine 1 and its surroundings (i.e., a forward environmental image that is highly useful from the perspective of the surroundings monitoring function) can be obtained.

[0094] In the case of the bottom following mode, according to this embodiment, as described above, the latitude and longitude associated with the target flight position are set in accordance with the orientation of the undercarriage 110 of the work machine 1, so when the work machine 1 is made to travel in a turning state, the forward environmental image provided by the imaging device 410 of the unmanned airplane 40 will follow the traveling motion of the work machine 1. Therefore, in the case of the bottom following mode, even when the undercarriage 110 is traveling while the upper rotating body 120 is turning, the usefulness (usefulness to the user) of the forward environmental image provided by the imaging device 410 of the unmanned airplane 40 can be effectively increased. From this perspective, when the work machine 1 is traveling, the bottom following mode is implemented, and when the work machine 1 is not traveling, another operating mode (for example, the top following mode or the second operating mode) may be implemented.

[0095] Incidentally, when using an unmanned airplane 40 to monitor the periphery of a work machine 1, the target flight position of the unmanned airplane 40 is preferably directly above or behind the work machine 1, whereas when assisting with work such as demolition work, the target flight position of the unmanned airplane 40 is preferably to the side of the work machine.

[0096] In this regard, according to this embodiment, as described above, the operating mode of the unmanned airplane 40 is appropriately switched between the first operating mode and the second operating mode, thereby providing a suitable forward environmental image for both surrounding monitoring and work support.

[0097] Next, an example of the operation of this embodiment will be described with reference to FIG. 6A and subsequent figures.

[0098] 6A is a schematic flowchart showing an example of processing executed by the flight control device 50 in relation to setting a target flight position. The processing shown in FIG. 6A may be executed repeatedly, for example, at predetermined intervals. Note that in the processing flow diagrams (flowcharts) from FIG. 6A onwards, the processing order of each step may be changed as long as the relationship between input and output of each step is not impaired.

[0099] In step S6, the flight control device 50 executes a mode switching process to set (switch) the operation mode of the unmanned airplane 40. An example of the mode switching process will be described later with reference to Fig. 6B.

[0100] In step S7, the flight control device 50 executes a target flight position / attitude setting process to set a target flight position and a target flight attitude of the unmanned airplane 40. An example of the target flight position / attitude setting process will be described later with reference to FIG. 6C.

[0101] In step S8, the flight control device 50 generates control information to indicate the target flight position and target flight attitude obtained in step S7 (step S32 or step S34 described below), and transmits the generated control information to the unmanned airplane 40.

[0102] Fig. 6B is a schematic flowchart showing an example of the mode switching process, and Fig. 7 is an explanatory diagram of the mode transition process.

[0103] In step S10, the flight control device 50 determines whether the mode transition in progress flag is "0." The mode transition in progress flag is "1" only while the operating mode is transitioning (during the mode transition process described below), and its initial value is "0." If the determination result is "YES," the process proceeds to step S11; otherwise (if the mode transition in progress flag is "1"), the process proceeds to step S15.

[0104] In step S11, the flight control device 50 determines whether the mode switching condition is met. The mode switching condition is met, for example, when the above-mentioned mode switching input is in a waiting state for processing. The waiting state for processing of the mode switching input refers to a state in which the mode switching input has occurred but the operation mode has not yet been switched. The mode switching condition may be determined automatically. For example, the first operation mode may be implemented while the work machine 1 is moving to the work position, and the mode switching condition for the transition from the first operation mode to the second operation mode may be met when the work machine 1 arrives at the work position and the crusher 145 moves relative to the work object. Alternatively, the mode switching condition for the transition from the first operation mode to the second operation mode may be met when the work machine 1 arrives at the work position and the upper rotating body 120 rotates. If the determination result is "YES," the process proceeds to step S12; otherwise, the processing for the current cycle ends.

[0105] In step S12, the flight control device 50 determines whether immediate mode switching is possible. For example, if the unmanned airplane 40 and the work machine 1 (e.g., the boom 141 of the work machine 1) come into contact with or come close to each other during the movement of the unmanned airplane 40 that occurs in conjunction with a change in operating mode (movement associated with a change in the target flight position), it may be determined that immediate mode switching is not possible. If the determination result is "YES," the process proceeds to step S13; otherwise, the process proceeds to step S15.

[0106] In step S13, the flight control device 50 switches the operation mode in accordance with the mode switching input awaiting processing. For example, if the mode switching input is an input requesting switching from the first operation mode to the second operation mode, the flight control device 50 switches the operation mode from the first operation mode to the second operation mode.

[0107] In step S14, the flight control device 50 sets the mode switching input that is waiting to be processed to a processed state.

[0108] In step S15, the flight control device 50 sets the content of the mode transition process and sets the mode transition in progress flag to “1.” The mode transition process is a process for moving the unmanned airplane 40 in a manner that prevents contact or proximity between the unmanned airplane 40 and the work machine 1. In this embodiment, as an example, in the case of a mode switching input between the bottom following mode and the second operation mode of the first operation mode, the mode transition process includes a process for interposing (passing through) the top following mode between the bottom following mode and the second operation mode. Note that FIG. 7 schematically shows the movement of the unmanned airplane 40 when switching from the bottom following mode to the second operation mode. In FIG. 7, the unmanned airplane 40 at the target flight position P21 for the bottom following mode first moves to the target flight position P22 for the top following mode (see arrow R70) through the mode transition process, and then moves to the target flight position P1 for the second operation mode (see arrow R71). This prevents inconveniences (such as the boom 141 coming into contact with or coming close to the unmanned aircraft 40) that may occur when transitioning directly from the lower following mode to the second operating mode (i.e., moving from the target flight position P21 to the target flight position P1 in the shortest distance) as shown by the dotted arrow R73 in Figure 7.

[0109] In other embodiments, the mode transition process may involve the unmanned airplane 40 ascending. In this case, the height of the unmanned airplane 40 may be determined based on the highest point of the boom 141 at that time, or based on the maximum height within the movable range of the boom 141. For example, the unmanned airplane 40 at the target flight position P21 for the bottom following mode first ascends to a predetermined height through the mode transition process, and then directly transitions from the bottom following mode to the second operating mode as indicated by the dotted arrow R73 in FIG. 7 (i.e., moves from the target flight position P21 to the target flight position P1 over the shortest distance). Alternatively, the unmanned airplane 40 at the target flight position P21 for the bottom following mode may first ascend to a predetermined height through the mode transition process, then move to the target flight position P22 for the top following mode (see arrow R70), and then move to the target flight position P1 for the second operating mode (see arrow R71).

[0110] In step S16, the flight control device 50 executes mode transition processing in accordance with the content determined in step S15.

[0111] In step S17, the flight control device 50 determines whether the mode transition process is complete. If the determination result is "YES", the process proceeds to step S18; otherwise, the process for the current cycle ends.

[0112] In step S18, the flight control device 50 sets the mode switching input that is waiting to be processed to a processed state.

[0113] In step S19, the flight control device 50 resets the mode transition flag to "0."

[0114] In this way, according to the processing shown in FIG. 6B, the operation mode is switched via the mode transition processing, so that the operation mode can be switched in a safe manner.

[0115] In a modified example, the boom 141 and the like may be automatically moved to a retracted position (for example, the lowest position) before switching the operating mode in accordance with the mode switching input. In this case, for example, the unmanned airplane 40 at target flight position P21 for the lower part following mode may first rise to a predetermined height corresponding to the highest position of the work machine 1, and then directly transition from the lower part following mode to the second operating mode as indicated by the dotted arrow R73 in Figure 7 (i.e., it may move from target flight position P21 to target flight position P1 over the shortest distance).

[0116] Alternatively, while switching of the operating mode is being performed in accordance with the mode switching input, operation of the working mechanism 140, such as the boom 141, may be temporarily prohibited. In this case, for example, unmanned airplane 40 at target flight position P21 for the lower part following mode may first rise to a predetermined height corresponding to the current height of the boom 141, and then directly transition from the lower part following mode to the second operating mode as indicated by dotted arrow R73 in Figure 7 (i.e., it may move from target flight position P21 to target flight position P1 over the shortest distance).

[0117] FIG. 6C is a schematic flowchart showing an example of a target flight position / attitude setting process.

[0118] In step S30, the flight control device 50 determines whether the operation mode is the first operation mode. If the determination result is "YES", the process proceeds to step S32, and otherwise the process proceeds to step S34.

[0119] In step S32, the flight control device 50 executes a target flight position / attitude calculation process for the first operation mode. An example of the target flight position / attitude calculation process for the first operation mode will be described later with reference to FIG. 6D.

[0120] In step S34, the flight control device 50 executes a target flight position / attitude calculation process for the second operation mode. An example of the target flight position / attitude calculation process for the second operation mode will be described later with reference to FIG. 6E.

[0121] FIG. 6D is a schematic flowchart showing an example of a target flight position / attitude calculation process for the first operation mode.

[0122] In step S40, the flight control device 50 acquires position information of the work machine 1 and upper rotating body orientation information of the upper rotating body 120 from the control device 10. The position information of the work machine 1 and the upper rotating body orientation information of the upper rotating body 120 are acquired by the position information acquisition unit 150 and orientation information acquisition unit 152 of the control device 10, respectively, as described above, and transmitted to the flight control device 50.

[0123] In step S42, the flight control device 50 acquires attitude information of the work machine 1 from the control device 10. The attitude information is information that affects the position of the crusher 145 (attitude information related to the tip attachment), and is acquired by the attitude information acquisition unit 151 of the control device 10 and transmitted to the flight control device 50 as described above.

[0124] In step S44, the flight control device 50 calculates the position of the tip of the crusher 145 (tip attachment position) based on the position information and upper rotating body orientation information obtained in step S40 and the attitude information obtained in step S42. The position of the tip of the crusher 145 is determined relative to the upper rotating body 120. Specifically, the position of the tip of the crusher 145 with respect to the upper rotating body 120 is uniquely determined according to the attitudes of the boom 141 and the arm 143, respectively. Note that in a modified example, instead of the position of the tip of the crusher 145, the position of another part of the crusher 145 or the position of the tip of the arm 143 may be calculated. This is because these positions can be treated as being substantially equivalent to the position of the tip of the crusher 145.

[0125] In step S46, the flight control device 50 calculates a target flight position (an example of a predetermined position) and a target flight attitude based on the position of the tip of the crusher 145 (tip attachment position) calculated in step S44 so that the tip is located in or near the center of the angle of view of the image capture device 410. In this case, the target flight position may be set around the tip of the crusher 145 as described above, and the target flight attitude may be set so that the crusher 145 and the work object (for example, an object to be disassembled) are within the angle of view of the image capture device 410 when the unmanned airplane 40 is located at the target flight position as described above.

[0126] 6D, the target flight position and the target flight attitude for the first operation mode can be calculated appropriately. As a result, a forward environment image including the tip of the crusher 145 (and the work object) can be acquired as a forward environment image that can support the work by the crusher 145.

[0127] FIG. 6E is a schematic flowchart showing an example of a target flight position / attitude calculation process for the second operation mode.

[0128] In step S50, the flight control device 50 acquires position information of the work machine 1 from the control device 10. The position information of the work machine 1 and the upper rotating body orientation information of the upper rotating body 120 are acquired by the position information acquisition unit 150 and orientation information acquisition unit 152 (upper rotating body orientation calculation unit 1520) of the control device 10, respectively, as described above, and transmitted to the flight control device 50.

[0129] In step S52, the flight control device 50 determines whether the operation mode is the lower follow mode of the first operation mode. If the determination result is "YES", the process proceeds to step S54, and otherwise (i.e., if the operation mode is the upper follow mode), the process proceeds to step S58.

[0130] In step S54, the flight control device 50 acquires undercarriage orientation information of the work machine 1 from the control device 10. The undercarriage orientation information of the work machine 1 is acquired by the orientation information acquisition unit 152 (undercarriage orientation calculation unit 1521) of the control device 10 as described above, and is transmitted to the flight control device 50.

[0131] In step S56, the flight control device 50 calculates the target flight position and target flight attitude for the undercarriage following mode based on the position information of the work machine 1 obtained in step S50 and the undercarriage orientation information obtained in step S54. The target flight position for the undercarriage following mode is as described above. For example, the target flight position for the undercarriage following mode may be calculated as a position on the central axis L1 (see FIG. 4) of the undercarriage 110 in a top view and behind the undercarriage 110. In this case, the flight control device 50 may set a predetermined target flight attitude as the target flight attitude for the undercarriage following mode.

[0132] In step S58, the flight control device 50 acquires the upper rotating body orientation information of the work machine 1 from the control device 10. The upper rotating body orientation information of the upper rotating body 120 of the work machine 1 is acquired by the orientation information acquisition unit 152 (upper rotating body orientation calculation unit 1520) of the control device 10 as described above, and is transmitted to the flight control device 50.

[0133] In step S60, the flight control device 50 calculates a target flight position and a target flight attitude for the upper following mode based on the position information of the work machine 1 obtained in step S50 and the upper rotating body orientation information obtained in step S58. The target flight position for the upper following mode is as described above. For example, the target flight position for the upper following mode may be calculated as a position on the central axis L0 of the upper following mode (see FIG. 4) in a top view and behind the upper rotating body 120 (or a position near the rotation axis of the rotation mechanism 130). In this case, the flight control device 50 may set a predetermined target flight attitude as the target flight attitude for the upper following mode.

[0134] 6E, the target flight position and target flight attitude for the second operation mode can be calculated appropriately. As a result, a forward environment image including the tip of the work machine 1 and the work object can be acquired as an image that can assist work by the work machine 1.

[0135] Although each embodiment has been described in detail above, it is not limited to the specific embodiment, and various modifications and changes are possible within the scope of the claims. It is also possible to combine all or a plurality of components of the above-described embodiments. [Explanation of symbols]

[0136] 1. Work machinery 8 Electrical equipment 10 Control device 40 Unmanned Aircraft 50 Flight control unit 52 Remote Control Device 80 Image output device 82 Sensors 821 GPS Compass 822 Rotation angle sensor 110 Undercarriage 120 Upper rotating body 122 Cab (operator's compartment) 130 Swivel mechanism 140 Working mechanism 141 Boom 142 Boom cylinder 143 Arm 144 Arm Cylinder 145 Crusher 146 Bucket cylinder 147 Crusher cylinder 150 Location information acquisition section 151 Attitude information acquisition unit 152 Orientation information acquisition unit 1520 Upper rotating body direction calculation unit 1521 Lower body direction calculation unit 153 Communication processing unit 154 Image output processing unit 400 control device 401 Aircraft Information Acquisition Unit 402 Target flight state setting unit 403 Aircraft Control Unit 404 Communication processing unit 410 Imaging device 510 Information Acquisition Department 511 Mode switching unit 512 Relative position relationship determination unit 514 Flight Control Unit 5141 Target flight position calculation section 5142 Target flight attitude calculation section

Claims

1. A work support device that controls the positional relationship between a work machine and an aircraft equipped with an imaging device, The work machine includes: a lower running body; an upper rotating body mounted on the lower traveling body; a working mechanism provided on the upper rotating body and having a tip attachment at its tip, The working mechanism includes: a boom attached to the upper rotating body so as to be able to be raised and lowered; an arm rotatably connected to the tip of the boom; the tip attachment rotatably attached to the tip of the arm, The work assistance device includes a flight control unit that generates control information for instructing a target flight attitude of the aircraft, When an operation signal for the tip attachment is input, the flight control unit calculates the position of the tip attachment and calculates the target flight attitude of the aircraft so that the optical axis of the imaging device passes through the calculated position.

2. The work assistance device includes a mode switching unit that switches an operation mode related to the flying object between a plurality of modes, The work support device according to claim 1, wherein, when an operation signal for the end attachment is input, the mode switching unit switches to a second operation mode in which an image including the end attachment capable of supporting work by the end attachment and a work object is acquired.

Images