Work support system, and work machine

Abstract

To photograph a work machine by flying an aircraft continuously for a long period of time without complicating a work.SOLUTION: A work support system comprises: a work machine; an aircraft flying above the work machine; and a cable connected to the work machine and the aircraft and equipped with a power line for supplying power from the work machine to the aircraft. The aircraft has first photographing means for photographing an area around the work machine and communication means for transmitting images photographed by the first photographing means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a work assistance system and a work machine. [Background technology]

[0002] For example, Patent Document 1 discloses a technique for photographing a work machine by flying an aircraft above the work machine. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7176874 Summary of the Invention [Problem to be solved by the invention]

[0004] The aircraft flies using an on-board battery, but the battery capacity is limited, so the aircraft cannot fly continuously for a long period of time. In the technology disclosed in Patent Document 1, a ground device equipped with a power generating device is placed near the work machine, and the ground device and the aircraft are connected via a connection cable, and power is supplied from the ground device to the aircraft.

[0005] However, in the technology disclosed in Patent Document 1, the ground equipment must be placed near the work machine to supply power to the flying object, which means that the ground equipment must be moved every time the work machine moves, making the work complicated.

[0006] It is therefore desirable to provide a work support system and a work machine that can fly an aircraft continuously for a long period of time to photograph the work machine without complicating the work. [Means for solving the problem]

[0007] The work assistance system of the present disclosure comprises: A work machine; a flying object that flies above the work machine; a cable connected to the work machine and the flying object and including a power line for supplying power from the work machine to the flying object; The flying object is a first photographing means for photographing the surroundings of the work machine; and a communication means for transmitting the image captured by the first imaging means.

[0008] Further, the work machine of the present disclosure is A work machine photographed by an aircraft flying above, It is connected to the aircraft via a power line and supplies power to the aircraft via the power line. Effect of the Invention

[0009] According to the work assistance system of the present disclosure, the aircraft can be flown continuously for a long period of time to photograph a work machine without complicating the work. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing an overall configuration of an embodiment of a work support system according to the present disclosure; [Diagram 2] 1 is a side view of one embodiment of a work machine. [Diagram 3] FIG. 1 is a top view of one embodiment of a work machine. [Figure 4] FIG. 2 is a diagram for explaining one embodiment of an aircraft port provided on a work machine. [Diagram 5] 2 is a block diagram showing specific examples of the configurations and functions of the work machine, the flying object, and the remote operation support device shown in FIG. 1. [Figure 6A] FIG. 1 is a diagram showing an example of a form in which the surroundings of a work machine are photographed by a camera of an aircraft. [Figure 6B] FIG. 1 is a diagram showing an example of a form in which the surroundings of a work machine are photographed by a camera of an aircraft. [Figure 6C]FIG. 1 is a diagram showing an example of a form in which the surroundings of a work machine are photographed by a camera of an aircraft. [Figure 7] 11 is a diagram showing an example of an image captured by an aircraft and transmitted to a remote operation support device. FIG. [Figure 8] FIG. 2 is a diagram for explaining tracking flight of an aircraft. [Figure 9] 10 is a flowchart for explaining an example of a landing operation of an aircraft on a work machine. [Figure 10] 11 is a diagram showing an example of an image captured by an aircraft and transmitted to a remote operation support device. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] System Overview First, an overview of the work assistance system according to the present disclosure will be described.

[0013] FIG. 1 is a diagram showing an overall configuration of an embodiment of a work support system according to the present disclosure.

[0014] 1, the work support system in this embodiment has a work machine 100, an aircraft 400, and a remote operation support device 200. The work machine 100 and the aircraft 400 are connected via a cable 500. The work machine 100, the aircraft 400, and the remote operation support device 200 are each configured to be connectable to each other via a communication line NW.

[0015] In the work support system configured as described above, power is supplied from the work machine 100 to the flying object 400 via the cable 500 , and the flying object 400 flies above the work machine 100 .

[0016] <Overview of the work machine> Next, an overview of one embodiment of the work machine 100 will be described.

[0017] FIG. 2 is a side view of one embodiment of work machine 100, and FIG. 3 is a top view of one embodiment of work machine 100.

[0018] 2 and 3, the work machine 100 according to this embodiment is a shovel including a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a rotating mechanism 2, an attachment AT for performing various tasks, and a cabin 10. Hereinafter, the front of the work machine 100 (upper rotating body 3) corresponds to the direction in which the attachment AT extends from the upper rotating body 3 when the work machine 100 is viewed in a plan view (top view) from directly above along the rotation axis of the upper rotating body 3. In addition, the left and right sides of the work machine 100 (upper rotating body 3) correspond to the left and right sides, respectively, as viewed from an operator seated in a driver's seat in the cabin 10.

[0019] Although the work machine 100 will be described below taking a shovel as an example, the work machine 100 is not limited to a shovel. For example, the work machine 100 may be an asphalt finisher, a forklift, a wheel loader, or the like.

[0020] The lower traveling structure 1 includes, for example, a pair of left and right crawlers 1C. Specifically, the crawlers 1C include a left crawler 1CL and a right crawler 1CR. The lower traveling structure 1 causes the work machine 100 to travel by driving each of the left crawler 1CL and the right crawler 1CR by a hydraulic motor or an electric motor.

[0021] The upper rotating body 3 is an example of a rotating section. The upper rotating body 3 rotates with respect to the lower traveling body 1 as a result of the rotating mechanism 2 being driven by a hydraulic motor or an electric motor.

[0022] The attachment AT (an example of a work attachment) includes a boom 4 , an arm 5 and a bucket 6 .

[0023] A boom 4 is attached to the front center of the upper rotating body 3 so as to be able to tilt up and down, an arm 5 is attached to the tip of the boom 4 so as to be able to rotate up and down, and a bucket 6 is attached to the tip of the arm 5 so as to be able to rotate up and down.

[0024] The bucket 6 is an example of an end attachment. The bucket 6 is used, for example, for excavation work. In addition, instead of the bucket 6, another end attachment may be attached to the tip of the arm 5 depending on the work content, etc. The other end attachment may be, for example, another type of bucket, such as a large bucket, a slope bucket, or a dredging bucket. The other end attachment may be a type of end attachment other than a bucket, such as an agitator, a breaker, or a grapple.

[0025] The boom 4, the arm 5 and the bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8 and a bucket cylinder 9, which serve as hydraulic actuators, respectively.

[0026] The work machine 100 may be configured such that some of the operating elements (hereinafter referred to as driven elements) such as the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6 are electrically driven. In other words, the work machine 100 may be a hybrid excavator, an electric excavator, or the like in which some of the driven elements are driven by electric actuators.

[0027] The cabin 10 is a control room in which an operator rides, and is mounted on the front left side of the upper rotating body 3. Note that when the work machine 100 is remotely operated or operates by fully automatic driving, the cabin 10 may be omitted.

[0028] Furthermore, the work machine 100 may be equipped with a communication device T1, for example, and may be capable of communicating with an external device via a predetermined communication line.

[0029] The work machine 100 operates actuators (for example, hydraulic actuators) in response to operation by an operator inside the cabin 10 to drive driven elements such as the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6.

[0030] An engine 11 is mounted on the rear of the upper rotating body 3. The engine 11 is a prime mover and the main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses diesel as fuel. The mounting position of the engine 11 is not limited to the rear of the upper rotating body 3. An engine hood 11a is attached on the engine 11 to cover it.

[0031] 3, two markers MA1 and MA2 are attached to the upper rotating body 3. The markers MA1 and MA2 are an example of a following support object, and are configured to be recognizable by the flying body 400. The number of markers is not limited to two, as long as there are more than one.

[0032] The work machine 100 also has a controller 300. The function of the controller 300 will be described later.

[0033] An air vehicle 400 is connected to the work machine 100 configured as described above via a cable 500. A connection port 121 is provided on the upper rotating body 3 of the work machine 100. One end of the cable 500 is connected to the connection port 121, and the other end is connected to the air vehicle 400. The air vehicle 400 flies above the work machine 100, and is landed on the work machine 100 when not flying.

[0034] Fig. 4 is a diagram for explaining one embodiment of an air vehicle port of the work machine 100. Note that in Fig. 4, the cable 500 connected to the air vehicle 400 is omitted from the illustration.

[0035] As shown in Fig. 4, the aircraft port 70 includes a recess 71 and a fixing mechanism 72. The recess 71 accommodates a portion of the aircraft 400. A side surface 71A of the recess 71 matches a side surface of the inverted truncated cone that widens upward. Here, the configuration in which "side surface 71A matches a side surface of the inverted truncated cone" includes not only a configuration in which the side surface 71A is in close contact with the side surface of the inverted truncated cone, but also a configuration in which the side surface 71A has multiple protrusions, and the inverted truncated cone is supported by the side surface 71A as the side surface of the inverted cone comes into contact with the tips of the multiple protrusions.

[0036] The fixing mechanism 72 fixes the flying object 400 accommodated in the recess 71. For example, the fixing mechanism 72 includes a fixing member 72A and a driving device 72B. The driving device 72B moves the fixing member 72A to sandwich the main body of the flying object 400 from both sides, thereby fixing the flying object 400.

[0037] The flying object 400 has a main body 411 and a number of rotors 412. The main body 411 has a side surface 413 that matches the side surface 71A of the recess 71. When the flying object 400 is accommodated in the recess 71, the side surface 413 of the flying object 400 comes into contact with the side surface 71A of the recess 71. Since the side surface 71A of the recess 71 widens upward, the positional deviation of the flying object 400 when landing is automatically eliminated. Furthermore, since the side surface of the truncated cone is infinitely rotationally symmetrical about its central axis, the flying object 400 can enter the flying object port 70 at any azimuth angle.

[0038] The main body 411 of the flying object 400 has a side surface 414 that is inclined in the opposite direction to the side surface 413 and is located above the side surface 413 that matches the side surface 71A of the recess 71. The side surface 414 matches the side surface of a truncated cone that tapers upward.

[0039] The fixing member 72A has a contact surface that contacts the side surface 414. This contact surface faces diagonally downward. The fixing members 72A are arranged to face each other across the recess 71. With the main body 411 of the flying object 400 housed in the recess 71, the fixing members 72A move in a direction toward each other. As a result, the main body 411 of the flying object 400 is pressed downward and fixed to the flying object port 70.

[0040] The aircraft port 70 configured as described above may be provided in, for example, the upper rotating body 3 of the work machine 100. The upper rotating body 3 is provided with the connection port 121 as described above. Therefore, the aircraft port 70 may be provided in the vicinity of the connection port 121 of the upper rotating body 3.

[0041] System Functions Next, the functions of the work machine 100, the flying object 400, and the remote operation support device 200 in the work support system will be described.

[0042] FIG. 5 is a block diagram showing a specific example of the configurations and functions of the work machine 100, the flying object 400, and the remote operation support device 200 shown in FIG.

[0043] As shown in FIG. 5, the flying object 400 has cameras 401a and 401b, an autonomous flying device 402, communication devices 403a and 403b, a power source 404, a power receiving unit 405, and a control device 420.

[0044] Camera 401a is an example of a first imaging means, and captures images of the periphery of work machine 100. For example, when flying object 400 is controlled to fly directly above work machine 100, camera 401a is attached to main body 411 of flying object 400 (see FIG. 4) with the lens facing directly downward.

[0045] Camera 401b is an example of a second imaging means, and captures an image of the work area of ​​the work machine 100. For example, when the flying object 400 is controlled to fly directly above the work machine 100, the lens is attached to the main body 411 of the flying object 400 (see FIG. 4) so ​​that it faces obliquely ahead of the work machine 100.

[0046] The communication devices 403a and 403b are examples of communication means.

[0047] The communication device 403a transmits and receives information via a signal line 520 for exchanging information with the work machine 100. For example, images captured by the cameras 401a, 401b are transmitted to the work machine 100. A command to land the flying object 400 at the flying object port 70 may be received via the signal line 520. A signal indicating that the flying object is in flight may be transmitted via the signal line. The signal line 520 may be built into the cable 500 together with the power line 510. The communication device 403a may also exchange information with the work machine 100 wirelessly without using the signal line 520.

[0048] The communication device 403b is for transmitting and receiving information to and from the remote operation support apparatus 200 via the communication line NW. For example, images captured by the cameras 401a and 401b are transmitted to the remote operation support apparatus 200.

[0049] The power source 404 is composed of a rechargeable battery such as a battery, and when kept charged, serves as a power source for the flying object 400 when power is not supplied from the work machine 100. Note that the power source 404 may be omitted.

[0050] The power receiving unit 405 is connected to a power supply line 510, and receives power supplied from the work machine 100 via the power supply line 510. For example, a power supply circuit may be used.

[0051] The autonomous flight device 402 includes a plurality of rotors 412 (see FIG. 4) and a motor for rotating the rotors 412. The motor rotates the rotors 412 to fly the flying object 400.

[0052] The control device 420 is a device for controlling the flying object 400. In this embodiment, the control device 420 is configured as a computer equipped with a RAM, a ROM, etc. The CPU of the control device 420 reads out programs corresponding to various functions from the ROM, loads them into the RAM, and executes them to realize the functions corresponding to each of the programs. The control device 420 includes, as functional units, a marker recognition unit 421, an autonomous flight control unit 422, and a communication control unit 423. The functions of the marker recognition unit 421, the autonomous flight control unit 422, and the communication control unit 423 are realized, for example, by loading a program installed in an auxiliary storage device into a memory device and executing it on the CPU.

[0053] The marker recognition unit 421 is an example of a first detection means, and detects the markers MA1 and MA2 from an image captured by the camera 401a. For example, the marker recognition unit 421 detects the markers MA1 and MA2 by recognizing features such as shapes and colors of the markers MA1 and MA2 in advance and detecting the recognized features from the image captured by the camera 401a.

[0054] The autonomous flight control unit 422 is an example of an autonomous flight control means. The autonomous flight control unit 422 controls the flight of the flying object 400 based on the positions of the markers MA1, MA2 detected by the marker recognition unit 421. Specifically, the autonomous flight control unit 422 controls the autonomous flight device 402 so that the flying object 400 is at a predetermined relative position with respect to the work machine 100 based on the positions of the markers MA1, MA2 detected by the marker recognition unit 421. The autonomous flight device 402 rotates the rotor 412 by a motor under the control of the autonomous flight control unit 422, causing the flying object 400 to fly.

[0055] The communication control unit 423 controls the communication in the communication devices 403a and 403b.

[0056] The above-mentioned cameras 401 a , 401 b , autonomous flight device 402 , communication devices 403 a , 403 b and control device 420 are driven by power supplied from the work machine 100 and received by a power receiving unit 405 or a power source 404 .

[0057] As shown in FIG. 5, the work machine 100 has a power source 120, a connection port 121, a traveling unit 141, a drive unit 142, communication devices 110a and 110b, a display device 143, and a controller 300.

[0058] The power source 120 is configured with an alternator or the like, and serves as a power source that supplies power to the flying object 400. The power source 120 may also drive the work machine 100 itself.

[0059] The connection port 121 is for connecting the cable 500. If an attachment plug is provided at one end of the power line 510 of the cable 500, a socket type may be used for the portion of the connection port 121 where the power line 510 is connected.

[0060] The power supply wire 510 may also include a winding unit 122 for winding the power supply wire 510. The winding unit 122 is configured to be able to wind the power supply wire 510 in a roll, and may be biased in the winding direction.

[0061] The travelling section 141 allows the working machine 100 to travel, and is made up of the crawler 1C, a motor for driving the crawler 1C and the like.

[0062] The drive unit 142 drives the attachment AT when the work machine 100 is performing work, and rotates the upper rotating body 3 via the rotating mechanism 2, and is composed of the attachment AT, a cylinder that drives the attachment AT, etc.

[0063] The communication device 110a is connected to the flying object 400 via a signal line 520 and a connection port 121, and exchanges information with the flying object 400. The communication device 110a receives images captured by the cameras 401a, 401b of the flying object 400 via the signal line 520. The communication device 110a may also exchange information with the flying object 400 wirelessly, without using the signal line 520.

[0064] The communication device 110b is connected to the remote operation support device 200 via the communication line NW, and exchanges information with the remote operation support device 200. The communication device 110b receives information related to the remote operation of the work machine 100 by the operator, which is input to the remote operation support device 200.

[0065] The display device 143 displays an image received by the communication device 110a. The display device 143 may also display information received by the communication device 110b from the remote operation support device 200. The display device 143 also functions as an example of an output means, and when a signal indicating that the flying object 400 is flying is received from the flying object 400 via the signal line 520, may output information indicating that.

[0066] The controller 300 has, as functional units, an operation control unit 301, an image receiving unit 302, an image transmitting unit 303, an object detection unit 304, and a safety control unit 305.

[0067] The operation control unit 301 controls the traveling unit 141 and the drive unit 142 based on information received by the communication device 110b from the remote operation support device 200. In addition, the operation control unit 301 may control the traveling unit 141 and the drive unit 142 based on an operation of a lever or the like provided on the work machine 100.

[0068] The image receiving unit 302 receives images captured by the cameras 401a and 401b of the flying object 400 via the communication device 110a and displays them on the display device 143.

[0069] The image transmission unit 303 may transmit the image received by the communication device 110a to the remote operation support device 200 via the communication device 110b. When the images captured by the cameras 401a and 401b of the flying object 400 are directly transmitted from the flying object 400 to the remote operation support device 200, the image transmission unit 303 does not need to transmit the image received by the communication device 110a to the remote operation support device 200. The image transmission unit 303 may process the image received by the communication device 110a before transmitting it. For example, the detection result by the object detection unit 304, the control contents by the safety control unit 305, and the like may be added before transmission.

[0070] The object detection unit 304 is an example of a second detection means. The object detection unit 304 detects a predetermined object (hereinafter referred to as a monitored object) from an image received by the image receiving unit 302. The image received by the image receiving unit 302 is an image captured by the camera 401a of the flying object 400, transmitted via the signal line 520 or wirelessly, and received by the communication device 110a. The monitored object may include a person such as a worker working around the work machine 100 or a supervisor at the work site. The monitored object may also include any object (obstacle) other than a person at the work site. The obstacle other than a person at the work site may include, for example, a specific topography such as a hole, a ditch, or a pile of earth and sand, a road cone, a fence, a utility pole, a temporarily stored material, a temporary office at the work site, or other fixed (non-moving) obstacle. The obstacle other than a person at the work site may also include, for example, a movable obstacle such as another work machine or a work vehicle.

[0071] The safety control unit 305 is an example of a safety control means, and performs control related to the functional safety of the work machine 100. Specifically, when the object detection unit 304 detects a monitoring target, the safety control unit 305 activates a safety function.

[0072] As shown in FIG. 5, the remote operation support device 200 includes a control device 210, a communication device 220, an output device 230, and an input device 240.

[0073] The control device 210 performs various controls related to the remote operation support device 200. The functions of the control device 210 are realized by any hardware or a combination of any hardware and software. The control device 210 is mainly configured with a computer including, for example, a CPU, a memory device such as a RAM, a non-volatile auxiliary storage device such as a ROM, and various input / output interface devices.

[0074] The control device 210 has, as functional units, a remote operation support unit 2101, a display control unit 2102, a notification control unit 2103, a communication status monitoring unit 2104, and an operation status monitoring unit 2105. The functions of the remote operation support unit 2101, the display control unit 2102, the notification control unit 2103, the communication status monitoring unit 2104, and the operation status monitoring unit 2105 are realized, for example, by loading a program installed in an auxiliary storage device into a memory device and executing the program on a CPU.

[0075] The communication device 220 (an example of a communication device for an external device) is connected to the communication line NW and communicates with the outside of the remote operation support device 200 (for example, the flying object 400 or the work machine 100).

[0076] The output device 230 outputs various information to a user of the remote operation support device 200. The output device 230 includes a display device 230A and a sound output device 230B.

[0077] The display device 230A outputs various information related to the remote operation support device 200 in a visual manner to a user of the remote operation support device 200 under the control of the control device 210. Specifically, the display device 230A displays an information image related to the remote operation support device 200. The display device 230A includes, for example, a liquid crystal display or an organic EL display.

[0078] The output device 230 may include a lighting device that visually outputs various information to the user of the remote operation support device 200. The lighting device includes, for example, a warning lamp.

[0079] The sound output device 230B includes a sound output device that outputs various information by an auditory means to the user of the remote operation support apparatus 200. The sound output device includes, for example, a buzzer, a speaker, and the like.

[0080] The input device 240 accepts input from a user of the remote operation support device 200, and outputs a signal representing the content of the input (for example, operation input, voice input, gesture input, etc.). The signal representing the content of the input is taken into the control device 210. The user of the remote operation support device 200 includes an administrator of the remote operation support device 200, an operator of the work machine 100, a monitor of the work machine 100, etc.

[0081] The input device 240 includes, for example, a remote control device 240A.

[0082] The remote control device 240A is used for remote control of the work machine 100 by an operator or for remote control of the work machine 100 by an observer intervening in the automatic driving function. The remote control device 240A receives inputs from the operator or observer for remote control of a plurality of actuators (driven elements), and outputs a signal indicating the content of the input to the control device 210.

[0083] The remote control device 240A may be used to control the flight of the flying object 400. The remote control device 240A receives input for remotely controlling the flying object 400 from an operator or observer, and outputs a signal representing the content of the input to the control device 210.

[0084] The remote operation support unit 2101 performs control related to the remote operation of the work machine 100. Specifically, the remote operation support unit 2101 may take in an input signal related to the remote operation of the work machine 100 received by the remote operation device 240A, and transmit a remote operation signal indicating the content of the remote operation of the work machine 100 to the work machine 100 using the communication device 220.

[0085] The remote operation support unit 2101 may perform control related to the remote operation of the flying object 400. Specifically, the remote operation support unit 2101 may take in an input signal related to the remote operation of the flying object 400 received by the remote control device 240A, and use the communication device 220 to transmit a remote operation signal indicating the content of the remote operation of the flying object 400 to the flying object 400.

[0086] The display control unit 2102 controls the display device 230A to display various information images on the display device 230A.

[0087] The display control unit 2102, for example, causes an image of the surroundings of the work machine 100 transmitted from the flying object 400 to be displayed on the display device 230A.

[0088] Specifically, the display control unit 2102 causes the display device 230A to display an image of the surroundings and the work area of ​​the work machine 100 corresponding to the imaging range of the cameras 401a, 401b of the flying object 400. This allows the user (operator) of the remote operation support device 200 to remotely operate the work machine 100 while checking the movement of the attachment AT by visually checking the image of the surroundings and the work area corresponding to the imaging range of the cameras 401a, 401b of the flying object 400. In addition, the user (monitor) of the remote operation support device 200 can monitor the work status of the work machine 100 while checking the movement of the attachment AT by visually checking the image of the surroundings and the work area corresponding to the imaging range of the cameras 401a, 401b of the flying object 400.

[0089] The display control unit 2102 may generate a processed image as a peripheral image based on the image transmitted from the flying object 400, and display the processed image on the display device 230A.

[0090] When a command signal indicating activation of the notification function (remote notification function) is received via the communication device 220, the notification control unit 2103 notifies an operator or a supervisor via the output device 230 that a monitored object has been detected within the notification range around the work machine 100. This command signal may be transmitted from the work machine 100 by activation of a safety function by the safety control unit 305 of the work machine 100, for example.

[0091] The notification control unit 2103 controls the sound output device 230B, for example, to auditorily notify the operator or supervisor that a monitored object has been detected within the notification range around the work machine 100. At this time, the notification control unit 2103 may vary the pitch, sound pressure, tone color, the sound cycle when the sound is blown periodically, the content of the sound, and the like of the output sound according to various conditions.

[0092] Furthermore, the notification control unit 2103 notifies the operator or the supervisor that a monitored object has been detected within the notification range around the work machine 100, for example, by a visual method. Specifically, the notification control unit 2103 may cause the display device 230A to display an image indicating that a monitored object has been detected, by controlling the display device 230A via the display control unit 2102. Furthermore, the notification control unit 2103 may cause the display device 230A to highlight the monitored object shown in the peripheral image displayed on the display device 230A or the position on the peripheral image corresponding to the detected monitored object, via the display control unit 2102. More specifically, the notification control unit 2103 may cause the display device 230A to display a frame surrounding the detected monitored object superimposed on the peripheral image displayed on the display device 230A, or to display a marker superimposed on a position on the peripheral image corresponding to the actual position of the detected monitored object, via the display control unit 2102. In this way, the display device 230A can realize a visual notification function for the operator or the supervisor. Furthermore, the notification control unit 2103 may use a lighting device such as a warning light to notify an operator or a supervisor that a monitored object has been detected within the notification range.

[0093] The communication status monitoring unit 2104 monitors the communication status between the work machine 100 and the remote operation support device 200.

[0094] Specifically, the communication status monitoring unit 2104 monitors the communication status between the work machine 100 and the remote operation support device 200 based on the presence or absence of a predetermined signal received from the work machine 100, the time required from transmission to reception of the predetermined signal, and the like. The communication status between the work machine 100 and the remote operation support device 200 includes, for example, the occurrence of communication interruption between the work machine 100 and the remote operation support device 200. In addition, the communication status between the work machine 100 and the remote operation support device 200 includes, for example, the occurrence of communication delay (the magnitude of communication delay) when a signal is transmitted from the work machine 100 to the remote operation support device 200. The predetermined signal is, for example, a response signal from the work machine 100 to a signal requesting a response that is periodically transmitted to the work machine 100 by the communication status monitoring unit 2104 via the communication device 220. In addition, the predetermined signal may be, for example, a signal corresponding to data (for example, a peripheral image) periodically transmitted from the work machine 100. This allows the communication status monitoring unit 2104 to monitor whether a communication interruption has occurred based on whether a specific signal has been received. Also, the communication status monitoring unit 2104 can measure the magnitude of a communication delay based on data (time stamp) indicating the transmission timing included in the specific signal and time data when the specific signal is actually received via the communication device 220.

[0095] The operation state monitoring unit 2105 monitors the operation state of the actuator (driven element) of the work machine 100 based on the input state received by the remote operation device 240A.

[0096] System Operation <Shooting mode using the flying object 400> In the work support system configured as described above, the flying object 400 flies so as to always be stationed above the work machine 100, and photographs of the surroundings of the work machine 100 are taken by the camera 401a of the flying object 400.

[0097] 6A to 6C are diagrams showing an example of a form in which the camera 401a of the flying object 400 captures an image of the surroundings of the work machine 100. FIG.

[0098] As shown in FIG. 6A, flying body 400 flies so as to stay constantly above the work machine 100. Camera 401a is attached to flying body 400. Camera 401a is attached to flying body 400, for example, with its lens facing directly downward. As a result, camera 401a photographs the surroundings of the work machine 100. Camera 401b may also be attached to flying body 400. Camera 401b is attached to flying body 400, for example, with its lens facing obliquely forward of the work machine 100. As a result, camera 401b photographs the work area to be taken by the work machine 100.

[0099] The flying object 400 is connected to the work machine 100 via a cable 500. The cable 500 includes a power line 510 for supplying power from the work machine 100 to the flying object 400. One end of the cable 500 is connected to the flying object 400, and the other end is connected to a connection port 121 provided on the work machine 100.

[0100] In this way, the work machine 100 and the flying object 400 are connected via the cable 500, and power is supplied from the work machine 100 to the flying object 400 via the power line 510 provided on the cable 500. The power supplied to the flying object 400 is received by the power receiving unit 405, enabling the flying object 400 to fly. Therefore, even if the work machine 100 moves, the flying object 400 can move together with the work machine 100, and the flying object 400 can be flown continuously for a long time to photograph the work machine 100 without complicating the work. In addition, there is no need to frequently charge or replace the battery installed in the flying object 400.

[0101] Furthermore, in contrast to the case where multiple cameras were attached to the work machine 100, images of the area surrounding the work machine 100 can be taken with a single flying object 400, allowing the number of cameras to be reduced. Furthermore, the camera 401a attached to the flying object 400 is not affected by vibrations caused by the work machine 100, unlike the camera attached to the work machine 100, so for example, an operator of the remote operation support device 200 can perform monitoring and operation while viewing unblurred captured images.

[0102] Here, the cable 500 can be inserted and removed from the connection port having the connection port 121. For example, if an attachment plug is provided at one end of the power line 510, a wall outlet type may be used for the part of the connection port 121 to which the power line 510 is connected. Therefore, when the flying object 400 is not in use, the cable 500 can be removed from the connection port 121 to disconnect the flying object 400 from the work machine 100.

[0103] The flying object 400 flies so as to stay above the work machine 100. The flying object 400 flies in an area at a height of, for example, 2 m, depending on the angle of view when the images of the cameras 401a and 401b are captured. Here, if the cable 500 is bent when the flying object 400 flies, when the periphery or work area of ​​the work machine 100 is captured by the cameras 401a and 401b, the cable 500 may get in the way and cause a blind spot. Therefore, by setting the length of the cable 500 to approximately the height at which the flying object 400 flies, as shown in FIG. 6A, the cable 500 is less likely to bend when the flying object 400 flies. As a result, when the periphery or work area of ​​the work machine 100 is captured by the cameras 401a and 401b, the cable 500 gets in the way of the capture and causes a blind spot to be less likely to be generated. Furthermore, if the length of cable 500 is made long enough to allow some leeway compared to the flying height of aircraft 400, it is possible to change the shooting range or move aircraft 400 horizontally to capture blind spots.

[0104] Furthermore, if the winding part 122 for winding the cable 500 is biased in the winding direction, even if the length of the cable 500 is longer than the flight height of the flying object 400, the cable 500 is pulled out from the winding part 122 by the flight height of the flying object 400. This makes it less likely that the cable 500 will get in the way of the image capture and cause blind spots when the cameras 401a, 401b capture images of the surroundings or work area of ​​the work machine 100.

[0105] 6B, the work machine 100 may be provided with a pole 80 extending upward. A power line 510 may be attached to the pole 80, or the power line 510 may be built into the pole 80. A connection port 121 may be provided at the tip of the pole 80, and a cable 500 may be connected to the connection port 121.

[0106] 6C, the pole 180 provided on the work machine 100 may have a length that matches the flight height of the flying object 400. In this case, too, a connection port 121 may be provided at the tip of the pole 180, and the cable 500 may be connected thereto. In such a configuration, when the surroundings or work area of ​​the work machine 100 are photographed with the cameras 401a, 401b, the cable 500 is even less likely to interfere with the photographing. Furthermore, by installing the pole 180 in an area away from the flight area of ​​the flying object 400 in the horizontal direction, the pole 180 is also less likely to interfere with the photographing.

[0107] The image captured by the camera 401a of the flying object 400 is transmitted, for example, to the remote operation support device 200. In this case, the image may be directly transmitted from the communication device 403b of the flying object 400 to the remote operation support device 200 via the communication line NW. Alternatively, the image may be once transmitted from the communication device 403a of the flying object 400 to the work machine 100 via the signal line 520, and then transmitted from the work machine 100 to the remote operation support device 200 via the communication line NW.

[0108] FIG. 7 is a diagram showing an example of an image captured by the flying object 400 and transmitted to the remote operation support device 200. As shown in FIG.

[0109] As shown in FIG. 7, a captured image 701 of the work machine 100 and its surrounding area 601 captured by the flying object 400 and transmitted to the remote operation support device 200 is displayed on the display device 230A of the remote operation support device 200. This allows the operator operating the remote operation support device 200 to monitor the surrounding area of ​​the work machine 100. In particular, in remote operation using a plurality of cameras attached to the work machine 100, the field of view is narrow and there is no sense of realism, so it is very effective to display the image captured using the flying object 400 on the remote operation support device 200. In addition, since a system using the remote operation support device 200 is generally expensive overall, the increase in cost to the extent of adding the flying object 400 is not much of a problem. The image displayed on the display device 230A of the remote operation support device 200 can be enlarged using a zoom function by operating the remote operation support device 200.

[0110] <Following flight of Aircraft 400> FIG. 8 is a diagram for explaining the tracking flight of the flying object 400. As shown in FIG.

[0111] Two markers MA1 and MA2 are attached to the upper rotating body 3. Meanwhile, the marker recognition unit 421 of the flying object 400 detects the markers MA1 and MA2 from an image captured by the camera 401a.

[0112] Therefore, the flying object 400 can be made to follow the movement of the work machine 100 based on the X-direction deviation amount X1 of the marker MA1, the X-direction deviation amount X2 of the marker MA2, and the Y-direction deviation amount Y1 of the markers MA1 and MA2 relative to the center point O of the image captured by the camera 401a.

[0113] Specifically, the values ​​of X1, X2, and Y1 when the flying object 400 is flying at a height and position that is a predetermined relative position to the work machine 100 are recognized in advance, and the rotor 412 is rotated by the motor under the control of the autonomous flight control unit 422 so that these X1, X2, and Y1 remain constant at those values, causing the flying object 400 to fly.

[0114] This makes it possible to have the flying body 400 follow the movement of the work machine 100 and fly at a constant position and height relative to the work machine 100 without having to wirelessly control the flight of the flying body 400 using a radio-controlled vehicle or the like. For example, if the flying body 400 is made to fly so as to always be stationed directly above the work machine 100, the flying body 400 will continue to fly so as to always be stationed directly above the work machine 100 even if the work machine 100 moves.

[0115] Here, the markers MA1 and MA2 are attached to the upper rotating body 3. Therefore, when the upper rotating body 3 rotates, if the flight of the flying body 400 is controlled as described above based on the positions of the markers MA1 and MA2 detected from the image captured by the camera 401a, the flying body 400 will also rotate following the rotation of the upper rotating body 3. As a result, when the upper rotating body 3 is provided with the connection port 121 and the cable 500 is connected to the upper rotating body 3, the cable 500 will not be twisted and deformed even if the upper rotating body 3 rotates. Also, by configuring the cable 500 in a spiral shape, it is possible to make it less likely to twist.

[0116] In this embodiment, the markers MA1 and MA2 are attached to the upper rotating body 3, and the markers MA1 and MA2 are used as the following support object. However, as long as the following support object can be recognized by the flying object 400, it may be, for example, an object that sways with a certain degree of regularity. Also, a part of the upper rotating body 3, such as the engine hood 11a, may be used as the following support object. In this case, the control device 420 of the flying object 400 recognizes the shape of the object used as the following support object, such as the engine hood 11a. Also, the control device 420 of the flying object 400 recognizes in advance the size and orientation of the following support object when the flying object 400 is flying at a height and position that is a predetermined relative position with respect to the work machine 100 in the image captured by the camera 401a. Then, when the following support object is detected from the image captured by the camera 401a, the autonomous flight control unit 422 controls the flight of the flying object 400 so that the size and orientation of the following support object in the image are recognized in advance. In this way, using a part of the upper rotating body 3, such as the engine hood 11a, as a tracking assistance object is effective in that it does not become difficult to detect due to dirt adhering thereto, unlike the markers MA1 and MA2.

[0117] <Safety control processing> The object detection unit 304 detects a monitored object from an image captured by the camera 401a of the flying object 400. The image captured by the camera 401a of the flying object 400 is transmitted to the work machine 100 via the signal line 520 or wirelessly, and is received by the image receiving unit 302 via the communication device 110a. For example, the object detection unit 304 may detect a monitored object in a monitoring area in which the distance D from a predetermined part of the work machine 100 is within a threshold value Dth1 in the image received by the image receiving unit 302. In this case, the threshold value Dth1 may be constant regardless of the direction viewed from the predetermined part of the work machine 100, or may change depending on the direction viewed from the predetermined part of the work machine 100. For example, the object detection unit 304 of the work machine 100 detects a monitored object in a monitoring area in which the horizontal distance D from the upper rotating body 3 is within a threshold value Dth1 (for example, 5 meters).

[0118] The object detection unit 304 recognizes surveillance objects in images captured by the aircraft 400 by arbitrarily applying, for example, various known image processing techniques and machine learning-based classifiers including artificial intelligence (AI).

[0119] The safety control unit 305 activates a safety function when a monitoring target is detected by the object detection unit 304. As the safety function, for example, a notification function is activated to output an alarm or the like to at least one of the inside of the cabin 10, the outside of the cabin 10, and an operator or manager operating the remote operation support device 200, and to notify the detection of the monitoring target. This makes it possible to alert the operator inside the cabin 10, the workers around the work machine 100, and the operator or supervisor operating the remote operation support device 200 that a monitoring target exists within a predetermined range around the work machine 100.

[0120] Also, as a safety function, for example, an operation limiting function may be activated that limits the operation of the work machine 100 in response to an operation command corresponding to the operation, remote operation, or automatic driving function of the work machine 100. This forcibly limits the operation of the work machine 100, and reduces the possibility of the work machine 100 or the suspended load approaching or coming into contact with surrounding objects. The operation limiting function may include an operation deceleration function that slows down the operation speed of the work machine 100 in response to an operation command corresponding to the operation, remote operation, or automatic driving function of the work machine 100, compared to normal. The operation limiting function may also include an operation stop function that stops the operation of the work machine 100 and maintains the stopped state, regardless of the operation command corresponding to the operation, remote operation, or automatic driving function of the work machine 100. By stopping the operation of the work machine 100, it is possible to prevent a contact accident before it occurs and to safely stop the work machine 100.

[0121] Further, the safety control unit 305 may control the sound output device 230B of the remote operation support device 200 to output a warning sound, for example.

[0122] Furthermore, the safety control unit 305 may display an image indicating that a monitored object has been detected, for example, by controlling the display device 230A of the remote operation support device 200. Furthermore, the safety control unit 305 may emphasize the monitored object shown in the peripheral image displayed on the display device 230A or the position on the peripheral image corresponding to the detected monitored object. More specifically, the safety control unit 305 may superimpose and display a frame surrounding the detected monitored object on the peripheral image displayed on the display device 230A, or may superimpose and display a marker on a position on the peripheral image corresponding to the actual position of the detected monitored object.

[0123] As a result, an operator or a supervisor who remotely operates or monitors the work machine 100 via the remote operation support device 200 can know that a monitored object has entered the notification range around the work machine 100.

[0124] <Landing Operation of the Aircraft 400 on the Work Machine 100> FIG. 9 is a flowchart for explaining an example of the landing operation of the flying object 400 on the work machine 100.

[0125] When the supply of power from the work machine 100 via the power line 510 is cut off, the flying object 400 will not be able to fly for a long period of time, even though the flying object 400 is equipped with the power source 404.

[0126] Therefore, when power is supplied from the work machine 100 via the power line 510 and the aircraft 400 is flying following the work machine 100 (step S1), if the supply of power from the work machine 100 is stopped (step S2), the autonomous flight control unit 422 controls the aircraft 400 to land on the aircraft port 70 as shown in Figure 4 (step S3).

[0127] The autonomous flight control unit 422 has a function of monitoring the presence or absence of power supply from the work machine 100. In other words, the autonomous flight control unit 422 monitors whether the power supplied from the work machine 100 is being received by the power receiving unit 405.

[0128] Then, when the supply of power from the work machine 100 is stopped, the autonomous flight control unit 422 drives the autonomous flight device 402 with the power source 404, and lands the air vehicle 400 on the air vehicle port 70. At this time, a landing marker may be attached to the air vehicle port 70, and the air vehicle 400 may detect the marker from an image captured by the camera 401a, and land the air vehicle 400 on the air vehicle port 70 in the same manner as in the case of following the work machine 100 by detecting the markers MA1 and MA2.

[0129] As a result, if power is no longer supplied from the work machine 100 to the flying object 400, the flying object 400 can land on the work machine 100 before it becomes impossible for the flying object 400 to fly. At this time, by providing the work machine 100 with an flying object port 70 on which the flying object 400 lands, the flying object 400 can land stably on the work machine 100.

[0130] In addition, since the cable 500 can be inserted and removed from the connection port 121, the cable 500 can be removed from the connection port after the aircraft 400 has landed on the aircraft port 70.

[0131] Furthermore, if the winding portion 122 for winding the cable 500 is biased in the winding direction, when the aircraft 400 lands on the aircraft port 70, the cable 500 will not be wound up and get in the way.

[0132] Furthermore, when the autonomous flight control unit 422 receives a command to land the flying object 400 on the flying object port 70 from the work machine 100 via the signal line 520, the flying object 400 may be landed on the flying object port 70. Furthermore, when the autonomous flight control unit 422 receives a command to land the flying object 400 on the flying object port 70 from the remote operation support device 200 via the communication line NW, the flying object 400 may be landed on the flying object port 70.

[0133] This allows the aircraft 400 to land on the aircraft port 70 when there is no need to photograph the area around the work machine 100 or before the engine 11 of the work machine 100 is shut down.

[0134] Furthermore, particularly when the flying object 400 does not have a power source 404, if the supply of power from the work machine 100 is stopped, the flying object 400 may not be able to fly and may crash. Therefore, when the work machine 100 receives a signal indicating that the flying object 400 is in flight via the signal line 520, the display device 143 of the work machine 100 may display that the flying object 400 is in flight. Also, when the remote operation support device 200 receives a signal indicating that the flying object 400 is in flight via the communication line NW, the display device 230A of the remote operation support device 200 may display that the flying object 400 is in flight.

[0135] This allows the operator of the work machine 100 or the remote operation support device 200 to recognize that the flying object 400 is in flight. This makes it possible to avoid a situation in which power is no longer supplied from the work machine 100 to the flying object 400 while the flying object 400 is flying, for example by stopping the engine 11 of the work machine 100. Note that the work machine 100 may be configured to continue the supply of power when the flying object 400 is in flight, even if the engine 11 is stopped.

[0136] <Photographing the work area> In addition to camera 401a for photographing the area surrounding the work machine 100, camera 401b for photographing the area worked by the work machine 100 is attached to the air vehicle 400. Camera 401b is attached to the air vehicle 400 with its lens facing obliquely forward of the work machine 100. This allows the air vehicle 400 to photograph the area worked by the work machine 100.

[0137] The image captured by the camera 401b of the flying object 400 is transmitted, for example, to the remote operation support device 200. In this case, the image may be directly transmitted from the communication device 403b of the flying object 400 to the remote operation support device 200 via the communication line NW. Alternatively, the image may be once transmitted from the communication device 403a of the flying object 400 to the work machine 100 via the signal line 520, and then transmitted from the work machine 100 to the remote operation support device 200 via the communication line NW.

[0138] FIG. 10 is a diagram showing an example of an image captured by the flying object 400 and transmitted to the remote operation support device 200. As shown in FIG.

[0139] As shown in Fig. 10, a captured image 702 of the work machine 100 and its working area 602, which is captured by the flying object 400 and transmitted to the remote operation support device 200, is displayed on the display device 230A of the remote operation support device 200. This allows the operator operating the remote operation support device 200 to remotely control the work machine 100 while viewing the image displayed on the display device 230A. At this time, the camera 401b attached to the flying object 400 is not affected by vibrations caused by the work machine 100 compared to the camera attached to the work machine 100, so the operator can remotely control the work machine 100 while viewing a blur-free captured image. The image displayed on the display device 230A of the remote operation support device 200 can be enlarged using a zoom function by operating the remote operation support device 200.

[0140] In the above embodiment, one flying object 400 flies above one work machine 100, but multiple flying objects 400 may fly above one work machine 100. In this case, the multiple flying objects 400 may photograph different areas. Furthermore, power can be supplied from the work machine 100 to the flying objects 400 by connecting one of the multiple flying objects 400 to the work machine 100 via a cable 500. Power can be supplied from the work machine 100 to one of the flying objects 400 via a power line 510 included in the cable 500, and power can be supplied from that flying object 400 to the other flying objects 400, for example, serially. [Explanation of symbols]

[0141] 1 Undercarriage 1C Crawler 3. Upper rotating body 4. Boom 5 Arm 6 Bucket 7 Boom cylinder 8 Arm Cylinder 9 Bucket Cylinder 10 Cabin 11 Engine 11a Engine hood 70 Aircraft Port 71 Recess 71A Side 72 Fixing mechanism 72A Fixing member 72B Drive Unit 80,180 Paul 100 Work Machines 110a,110b,220,403a,403b Communication device 120,404 Power supply 121 Connection port 122 Winding section 141 Running part 142 Drive unit 143,230A display device 200 Remote operation support device 210,420 Control device 230 Output Device 230B Sound output device 240 Input Device 240A Remote Control Device 300 Controller 301 Operation control section 302 Image receiving unit 303 Image transmission unit 304 Object detection unit 305 Safety Control Section 400 Flying Objects 401a, 401b Camera 402 Autonomous Flight Device 405 Power Receiving Section 411 Main unit 412 Rotor 413,414 Side 421 Marker Recognition Unit 422 Autonomous Flight Control Unit 423 Communication Control Unit 500 Cable 510 Power line 520 Signal Line 601 Surrounding Area 602 Work area 701,702 Images 2101 Remote Operation Support Department 2102 Display control unit 2103 Notification control unit 2104 Communication Status Monitoring Department 2105 Operation status monitoring unit AT attachment (work attachment) MA1,MA2 markers Network communication line O center point

Claims

1. A work machine; a flying object that flies above the work machine; a cable connected to the work machine and the flying object and including a power line for supplying power from the work machine to the flying object; The flying object is a first photographing means for photographing the surroundings of the work machine; A work support system comprising: a communication means for transmitting the image captured by the first imaging means.

2. The flying object is An autonomous flight control means for controlling the flight of the flying object; A first detection means for detecting a tracking support object from an image captured by the first imaging means, 2. The work support system according to claim 1, wherein the autonomous flight control means controls the flight of the aircraft so that the aircraft is in a predetermined relative position with respect to the work machine based on the tracking support object detected by the first detection means.

3. The work machine has a rotatable rotating part, The cable is connected to the pivot portion, The following assistance object is at least a part of the turning section, or is provided on the turning section, 3. The work support system according to claim 2, wherein the autonomous flight control means causes the flying object to turn in accordance with the turning of the turning section, based on the following support object detected by the first detection means.

4. The work support system according to claim 2 , wherein the work machine has an air vehicle port on which the air vehicle takes off and lands.

5. the cable includes a signal line for transmitting information between the work machine and the aircraft, The work support system of claim 4, wherein the autonomous flight control means lands the aircraft at the aircraft port when it receives a command from the work machine via the signal line to land the aircraft at the aircraft port.

6. The flying object is a power source for flying the aircraft when the supply of power from the work machine via the power line is stopped, The work support system according to claim 4 , wherein the autonomous flight control means lands the aircraft on the aircraft port when the supply of power from the work machine via the power line is stopped.

7. the cable includes a signal line for transmitting information between the work machine and the aircraft, 2. The work support system according to claim 1, wherein the work machine has an output means for outputting information indicating that the flying object is in flight when a signal indicating that the flying object is in flight is received via the signal line.

8. The work support system according to claim 1 , wherein the cable is configured to be pluggable into and removable from a connection port of the work machine.

9. 2. The work support system according to claim 1, wherein the work machine has a winding section that winds up the cable, and the winding section is biased in a direction in which it winds up the cable.

10. a second detection means for detecting a predetermined object from an image captured by the first imaging means; The work support system according to claim 1 , further comprising: a safety control means for activating a safety function when the predetermined object is detected by the second detection means.

11. The work support system according to claim 10 , wherein the safety control means stops operation of the work machine when the predetermined object is detected by the second detection means.

12. the aircraft has a second photographing means for photographing an area where the work machine is to work, The work support system according to claim 1 , wherein the communication means transmits the images captured by the first and second imaging means.

13. A work machine photographed by an aircraft flying above, A work machine that is connected to the flying object via a cable having a power line and supplies power to the flying object via the power line.

Images