Robot control systems, flying devices, robot control programs

Abstract

It performs tasks autonomously, regardless of environmental conditions, depending on the state of the object being worked on. [Solution] Even in high places or harsh environments where it is dangerous for workers to perform tasks, the drone 14 can be moved close to the target object using its flight function without the need for safety measures, and detailed work can be performed by AI control. When the flight device 12 arrives at the work site and before the arms 48L and 48R perform work, the electromagnet section 104 of the leg section 102 is magnetically attached to the magnetic material at the work site. This allows the flight to be interrupted while the arms 48L and 48R are working, enabling stable work and saving power because the power consumption during flight is extremely low compared to the power consumption of the electromagnet section 104.

Description

Technical Field

[0001] The present invention relates to a control system for a robot, a flying device, and a control program for a robot.

Background Art

[0002] It has been proposed to substitute a robot in a high place or a harsh environment where it is difficult for an operator to approach the work target.

[0003] In Patent Document 1, a structure inspection system has been proposed that can inspect a structure located on the lower surface of a structure, even on an inspection surface having a narrow space or a step, without installing a scaffold or using an inspection vehicle for high-altitude work. The structure inspection system of Patent Document 1 includes a flying robot and a remote control device, and the flying robot can inspect even on an inspection surface having a narrow space or a step, located on the lower surface of a structure, without installing a scaffold or using an inspection vehicle for high-altitude work.

[0004] Further, Patent Document 2 proposes a mobile robot that moves along a guide member, the mobile robot including a contact portion that restricts the moving direction of the mobile robot along the guide member by contacting the guide member, a propulsion portion that generates a propulsion force in the direction along the guide member, and a support portion that supports an electric wire.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, while conventional robots can operate in high places and harsh environments, their work is limited to single tasks. That is, the robots perform tasks based on predetermined sequence control. Therefore, they cannot handle tasks involving multiple steps, such as repair work. Furthermore, they are unsuitable for delicate tasks requiring autonomous control functions, such as on-site decision-making regarding changes to work processes.

[0007] Considering the above facts, the present invention aims to provide a robot control system, flight device, and robot control program that can autonomously perform tasks regardless of environmental conditions and depending on the state of the work object. [Means for solving the problem]

[0008] A robot control system according to Embodiment 1 of the present invention is a robot control system that causes a robot attached to a drone to face an object and perform work on the object, and comprises: a holding unit that maintains the state in which the robot is facing the object when the flight of the drone is stopped; a sensor that identifies the position of at least the object to be worked on; a determination unit that determines the type of work to be done on the object; and a control unit that controls the operation of attaching a tool corresponding to the type of work determined by the determination unit to the robot, and controls the work according to the type of work based on the detection result of the sensor.

[0009] According to one aspect of the present invention, flight control by the drone is used to position the robot facing the object, determine the type of work to be done on the object, control the operation to attach a tool corresponding to the determined type of work to the robot, and control the work according to the type of work based on the detection result of a sensor that identifies the position of the object to be worked on. The holding unit maintains the state in which the robot is facing the object when the drone's flight is stopped.

[0010] This allows the system to perform tasks autonomously, regardless of environmental conditions, depending on the state of the object being worked on.

[0011] A second aspect of the present invention is characterized in that the holding portion comprises an electromagnet portion that can be magnetically attached to the object or a magnetic support surface provided near the object, and a leg portion that faces the support surface toward the magnetic attachment portion.

[0012] A third aspect of the present invention is characterized in that the robot is a humanoid robot.

[0013] A fourth aspect of the present invention is characterized in that the determination unit makes a determination based on information from a sensor unit mounted on the tool, which includes a camera that captures an image of the object and identifies the type of the object, and a motion processing unit that identifies the position of the object.

[0014] A camera identifies the object it captures based on the image information it obtains. In other words, it plays a role in acquiring information to identify the type of object (shape, size, hardness, etc.).

[0015] The motion processing unit (MoPU) outputs vector information of the movement along predetermined coordinate axes of a point indicating the location of an object, along with motion information, as position information. In other words, the motion information output from the MoPU contains only information indicating the movement (direction of movement and speed of movement) along the coordinate axes (x axis, y axis, z axis) of the object's center point (or center of gravity). This means that the trajectory of the gripping part as it approaches the object can be guided with high precision.

[0016] A flying device according to aspect 5 of the present invention is a flying device that is detachable from the wrist portion of an arm that moves in three dimensions, and which attaches a tool according to the type of work to be done on an object and performs work on the object, comprising: a drone capable of flying to a predetermined position; a robot unit attached to the drone and equipped with a sensing unit that identifies at least the position of the work object and the tool, which autonomously performs work by attaching a tool according to the type of work to be done on the object while facing the object; an electromagnet unit that can be magnetically attached to the object or a magnetic support surface provided near the object, and a leg unit that faces the electromagnet unit toward the support surface, which maintains the state of the robot facing the object when the flight of the drone is stopped.

[0017] The robot control program according to aspect 6 of the present invention is characterized in that, in the invention described in any one of aspects 1 to 3, a computer is operated as the determination unit and the control unit described above.

[0018] It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. Furthermore, subcombinations of these features may also constitute an invention. [Effects of the Invention]

[0019] As explained above, the present invention has the effect of enabling autonomous work to be performed regardless of environmental conditions (for example, at high altitudes or in harsh environments), depending on the state of the work object. [Brief explanation of the drawing]

[0020] [Figure 1] This is a perspective view of a flying device equipped with the robot according to this embodiment, while it is performing work related to wind power generation equipment. [Figure 2] (A) is a front view of the humanoid robot according to this embodiment, and (B) is a perspective view of the humanoid robot according to this embodiment. [Figure 3]It is a plan view showing a state where a robot tool is attached to the humanoid robot according to this embodiment. [Figure 4] It is a diagram schematically showing an example of the functional configuration of the humanoid robot according to this embodiment. [Figure 5] It is a flowchart showing the procedure of gripping control when gripping a load by a gripping part in conjunction with the overall operation of the humanoid robot. [Figure 6] (A) is a control flowchart showing the movement processing subroutine to the destination in step 151 of FIG. 5, and (B) is a control flowchart showing the details of the robot tool application processing subroutine in step 160 of FIG. 5. [Figure 7] It is a perspective view of the robot part attached to the flying device according to Modification 1, where (A) is a two-arm structure and (B) is a single-arm structure. [Figure 8] It is a perspective view of the flying device provided with the holding part according to Modification 2, where (A) shows a fixed type of leg part and (B) shows a telescopic type of leg part. [Figure 9] It is a perspective view of the flying device provided with the holding part according to Modification 3, where (A) is a structure provided with a holding part for a plane and a holding part for a wall surface, (B) is an application example of the holding part for a plane, and (C) is an application example of the holding part for a wall surface. [Figure 10] It is a diagram schematically showing an example of the hardware configuration of a computer functioning as a Central Brain. ​​​​​​​​​​​

[0023] The drone 12 is a type of autonomous flying object, consisting of a main body 14A and propellers 14B attached to the four corners of the main body 14A. For example, it can fly autonomously while constantly knowing its position using GPS (Global Positioning System). In some cases, it may be flown by remote control by an operator, either partially or entirely.

[0024] Furthermore, the drone is not limited to Drone 14; any multi-rotor aircraft with multiple propellers that can stop (hover) at the desired location is also acceptable.

[0025] The main body 14A houses drive and control system devices for performing flight control. A camera 14C is mounted on the lower part of the casing covering the main body 14A, and a holding plate 14D is provided to suspend and hold the humanoid robot 10 via its legs. The camera 14C monitors the flight trajectory of the drone 14.

[0026] In this embodiment, the humanoid robot 10, suspended from the drone 14, is brought closer to the work target (in this embodiment, the rotor blades 22A of the wind power generation equipment 20) by the flight of the drone 14. Then, for example, the humanoid robot 10 performs a task (for example, fault diagnosis, maintenance, repair, assembly, etc. of the rotor blades 22A) using the AI ​​(Artificial Intelligence) function provided by the humanoid robot 10.

[0027] Here, the drone 14 can obtain lift and thrust through the rotation of its propeller 14B, but in environments with relatively strong winds, its flight becomes unstable.

[0028] To overcome this flight instability, the humanoid robot 10 is equipped with a pair of wings 10A. The drone 14 works in conjunction with the humanoid robot 10 to generate thrust to move towards the target object (the rotor blades 22A of the wind power generation equipment 20) (especially during high-speed flight when flight is relatively unstable), by extending the pair of wings 10A to achieve stable flight (shown by arrows A and B in Figure 1, indicating the state during flight movement).

[0029] On the other hand, once the robot reaches the target object, thrust is no longer needed, and it can maintain its current position with lift. Also, since the wings 10A would get in the way of the work, the humanoid robot 10 retracts them (shown by arrow C in Figure 1, indicating the state upon arrival at the target object and during the work).

[0030] Figures 2(A) and (B) are a front view and a perspective view of the humanoid robot 10 mounted on the drone 14. As shown in Figures 2(A) and (B), the humanoid robot 10 comprises an upper body 24, a lower body 25, and arms 26L and 26R, and performs work on objects (in this embodiment, the rotor blades 22A of the wind power generation equipment 20) at high altitudes or in harsh environments, such as the wind power generation equipment 20 shown in Figure 1.

[0031] In other words, the operator only needs to wait on the ground and instruct the flying device 12 to perform tasks (including some operations), eliminating the need to go to high places or other elevated locations.

[0032] Furthermore, an electromagnet unit 25B is attached to the leg portion 25A, which constitutes part of the lower body portion 25 of the humanoid robot 10 used in this implementation. This allows the robot to magnetically attach to a magnetic material at the work site and temporarily suspend the flight of the drone 14 to perform work on the target object. The leg portion 25A and the electromagnet unit 25B constitute a holding portion.

[0033] The legs 25A of the humanoid robot 10 are provided with multiple joints, so that, for example, when facing the ground in the vertical direction of the flight device 12, they can be magnetically attached to a flat support surface, and when facing the object to be worked on, they can be magnetically attached to a wall support surface.

[0034] The arms 26L and 26R attached to the upper body 24 are rotatably mounted to the left and right of the upper body 24. Furthermore, robotic tools 28L and 28R (details described later) for performing predetermined tasks on the workpiece are attached to the ends (wrists) of the arms 26L and 26R. Note that the number of arms is not limited to two; there may be one or three or more.

[0035] The humanoid robot 1 is controlled by a control system 30 (see Figure 4) implemented within the humanoid robot 10.

[0036] (Structure of robot tools 28L and 28R) As shown in Figure 3(A), the robot tool 28L, which is attached to the tip (wrist) of the arm 26L, corresponding to the left hand in humans, has a structure similar to that of a human hand (Intelligent Hand System), and is mainly used to assist in tasks such as holding down the rotor blade 22A when the robot tool 28R is working.

[0037] Furthermore, as shown in Figure 3(A), the robot tool 28R, which is attached to the tip (wrist) of the arm 26R, which corresponds to the right hand in humans, is a tool that performs specific tasks.

[0038] The robotic tools 28L and 28R are connected to the arms 26L and 26R by a universal joint structure, and are each mounted to rotate three-dimensionally. More specifically, the wrists can rotate (twist) and move up and down (cocking), and may also extend and retract as needed.

[0039] Furthermore, at least the robot tool 28R attached to the arm 26R is detachable from the arm 26R and can be replaced with the robot tool 30EX, which will be described later.

[0040] Arm 26R is replaced with a tool for performing the instructed task. On the other hand, arm 26L is positioned as an auxiliary work tool that assists in performing the instructed task (for example, supporting the rotor blade 22A or holding the rotor blade 22A in place).

[0041] Note that the functions of the arms 26L and 26R may be reversed left and right. Also, although the robot tool 28L attached to the arm 26L is not detachable in this embodiment (specialized for human finger movements), it may be detachable, similar to the robot tool 28R.

[0042] (Robot Tool 28L) As shown in Figure 3, the robot tool 28L according to this embodiment has a palm portion as a base corresponding to a human palm, and five finger portions, each with multiple joints, are attached to the palm portion. In this embodiment, the robot tool 28L has five fingers, but it may also have a different number of fingers, such as three.

[0043] A palm sensor 32 is attached to the palm. The high-resolution camera constituting the palm sensor 32 in this embodiment identifies the attributes (shape, size, hardness, material, etc.) of the captured object based on the captured image information.

[0044] In other words, the high-resolution camera serves to acquire information to identify the type (shape, size, stiffness, etc.) of the rotor blade 22A.

[0045] On the other hand, the MoPU, which constitutes the palm sensor 32 of this embodiment together with the high-resolution camera, outputs motion information indicating the movement of the captured rotor blade 22A (in this case, the relative movement between it and the arm parts 26L and 26R) from the image of the rotor blade 22A captured at a frame rate of 1000 frames / second or more, for example at a frame rate of 1000 frames / second or more. Note that when detecting a moving rotor blade 22A, the frame rate may be increased, and when detecting a stationary object (a non-operating rotor blade 22A), the frame rate may be decreased.

[0046] The MoPU outputs motion information that is vector information of the movement of a point indicating the location of the rotor blade 22A along a predetermined coordinate axis. In other words, the motion information output from the MoPU does not contain any information necessary to identify the attributes of the captured rotor blade 22A, and only contains information indicating the movement (direction of movement and speed of movement) of the rotor blade 22A's center point (or center of gravity) along the coordinate axes (x axis, y axis, z axis).

[0047] In other words, the robot tool 20 can be guided with high precision along its trajectory as it approaches the rotor blade 22A.

[0048] Information output from the palm sensor 32, which includes a high-resolution camera and a MoPU, is supplied to the information processing device 34 (see Figure 4). The information processing device 34 functions as a determination unit and a control unit according to the present invention.

[0049] Although the palm sensor 32 (high-resolution camera and MoPU) is shown as being in the location closest to the work area, it is not mandatory to attach it to the palm. For example, the high-resolution camera and MoPU may be attached to the back of the hand or the wrist. Furthermore, in order to grasp the work from an overhead perspective, the high-resolution camera and MoPU may be attached to the head of the humanoid robot 10 as shown in Figure 3. In addition, the high-resolution camera and MoPU may be attached to multiple locations.

[0050] As shown in Figure 4, the information processing device 34 accurately determines the position of the rotor blade 22A using information from a high-resolution camera and a palm sensor 232 including a MoPU, calculates the degree of finger spread when the robot tool 28L grasps, the gripping strength (or suction force if it has a suction function), and other factors, and can accurately control minute movements of the arm 26L and the robot tool 28L.

[0051] (Robot Tool 28R and Robot Tool 30EX) In this embodiment, the types of work performed on the rotor blade 22A include inspection and repair, such as gripping heavy objects, tightening and loosening screws with a screwdriver, and wiping away dirt.

[0052] In this case, it is possible for the robot tool 28R to grip a tool for various tasks and face the rotor blade 22A, but the burden of maintaining and controlling the gripping state (such as controlling the relative position between the gripping part and the gripped tool) is significant.

[0053] Therefore, in this embodiment, instead of the robot tool 28R (see Figure 3(A)), which is the basic tool attached to the arm 26R, robot tools 30EX (three types in this embodiment: 30EXA, 30EXB, and 30EXC) are provided according to the type of work to be done on the rotor blade 22A, as shown in Figures 3(B) to (C). When necessary, the robot tool 28R is replaced with the robot tool 30EX (A to C) to perform work corresponding to a type of work other than gripping the rotor blade 22A.

[0054] (Storage example for Robot Tool 30EX) As shown in Figure 2, the humanoid robot 10 has a belt 35 attached to the lower part of its upper body 24 (so-called waist position), and holders (not shown) are attached to the belt 35 for detachably holding three robot tools 30EXA, 30EXB, and 30EXC, respectively.

[0055] In Figure 2, there are three robot tools 30EX (A-C), but the number of robot tools 30EX that can be attached in place of robot tool 28R may be one, two, or four or more types. The number to be attached should be determined according to the attributes of the object to be worked on (in this embodiment, the shape and material of the rotor blade 22A, etc.).

[0056] Figure 3 shows the detailed configuration of robot tools 30EXA, 30EXB, and 30EXC, which can be attached to the arm 5 in place of robot tool 28R and robot tool 20R, and their relationship to their applications.

[0057] As shown in Figure 3(A), the robot tool 28R has a structure similar to a fork used as an attachment for heavy machinery, for example. In this embodiment, the robot tool 28R has a two-finger structure, and the two fingers open and close due to the pressure supplied from the pressure source via piping. This increases the gripping strength of the rotor blade 22A, although it is less versatile than a gripping operation by a motor or the like (robot tool 28L).

[0058] As shown in Figure 3(B), the types of work performed by the robot tool 30EXA are tightening, loosening, and drilling screws, and in Figure 3(B), a screwdriver is attached as the tool to the chuck.

[0059] As shown in Figure 3(C), the robot tool 30EXB's job type is cleaning, and it is equipped with a cotton swab.

[0060] With the robot tool 30EXB, dirt adhering to the surface of an object (rotating blade 22A of the wind power generation equipment 20) held down by the robot tool 28L can be wiped away by moving a cotton swab back and forth while making contact with it.

[0061] As shown in Figure 3(D), the robot tool 30EXC's job is to pick up and remove debris, and it is equipped with tweezers as its tool.

[0062] In addition, although not mounted on the humanoid robot 1 in this embodiment, other robotic tools for different types of work, such as a spray gun for painting or a soldering iron, may be attached as robotic tools.

[0063] In this embodiment, the optimal robot tool 28 and robot tool 30EX (A-C) are selected and attached based on the type of work, and the processing is executed.

[0064] Figure 4 is a schematic diagram of an example of a control system 30 for a humanoid robot 10 according to this embodiment. The control system 30 includes a sensor 36 mounted on the humanoid robot, a palm sensor 32 including a high-resolution camera and MoPU, and an information processing device 34.

[0065] Sensor 36 sequentially acquires information that at least represents the distance and angle between the humanoid robot 10 and the object it is working on (the rotor blade 22A of the wind turbine 20) and its arms 26L and 26R, which are located around the humanoid robot 10. Sensor 36 can be a high-performance camera, solid-state LiDAR, multi-color laser coaxial displacement meter, or various other sensor groups. Other examples of sensors 36 include vibration meters, thermal cameras, hardness testers, radar, LiDAR, high-resolution, telephoto, ultra-wide-angle, 360-degree, high-performance cameras, vision recognition, minute sound, ultrasound, vibration, infrared, ultraviolet, electromagnetic waves, temperature, humidity, spot AI weather forecast, high-precision multi-channel GPS, low-altitude satellite information, or long-tail incident AI data.

[0066] In addition to the above information, sensor 36 also detects images, distance, vibration, heat, odor, color, sound, ultrasound, ultraviolet light, or infrared light. Sensor 36 performs these detections, for example, every nanosecond.

[0067] The palm sensor 32 (high-resolution camera and MoPU) is a sensor provided on the robot tool 28L of the arm 26L, and, separate from sensor 36, has a camera function for photographing the rotor blade 22A and a positioning function for determining the position of the rotor blade 22A.

[0068] Furthermore, when using one MoPU, it is possible to obtain vector information of the motion of a point indicating the location of the rotor blade 22A along each of the two coordinate axes (x-axis and y-axis) in a three-dimensional Cartesian coordinate system. Alternatively, using the principle of a stereo camera, two MoPUs may be used to output vector information of the motion of a point indicating the location of the rotor blade 22A along each of the three coordinate axes (x-axis, y-axis, and z-axis) in a three-dimensional Cartesian coordinate system. The z-axis is the axis along the depth direction (vehicle movement).

[0069] The information processing device 34 comprises an information acquisition unit 38, a control unit 40, and an information storage unit 42.

[0070] The information acquisition unit 38 acquires information about the rotor blades 22A detected by the sensor 36 and the palm sensor 32 (high-resolution camera and MoPU).

[0071] The control unit 40 uses the information acquired by the information acquisition unit 38 from the sensor 36 and AI (Artificial Intelligence) to control the movements of the arm sections 26L and 26R.

[0072] Furthermore, the control unit 40 uses the information acquired by the information acquisition unit 38 from the palm sensor 32 (high-resolution camera and MoPU) to determine in detail the type (shape, size, hardness, etc.) and position of the rotor blade 22A, and aligns it according to its external shape and position.

[0073] The operation of this embodiment will be described below. (Work control for rotor blade 22A) Figure 5 is a flowchart showing the work control procedure when the robot tool 20 performs work on the rotor blade 22A.

[0074] In step 150, it is determined whether or not there has been an instruction for the rotor blade 22A to work. If the determination is positive, the process proceeds to step 151, where the movement process to the destination is executed.

[0075] Figure 6(A) is a control flowchart showing the subroutine for moving to the destination in step 151.

[0076] In step 210, the magnetic attachment state is terminated and the system switches to flight by the drone 14, then the system proceeds to step 212, where the drone 14 flies to the vicinity of the destination.

[0077] In the next step 214, it is determined whether or not there is a support surface to which magnetization can be applied. If the determination is positive, the process proceeds to step 216. In step 216, the leg portion 25A is pointed towards the support surface, and the electromagnet portion 25B attached to the tip is energized to magnetically attach to the support surface, while the drone 14 is stopped, i.e., hovering is stopped. Then, the process returns to step 152 in Figure 5. This allows the drone to maintain its current position.

[0078] Furthermore, if the determination in step 214 is negative, the drone 14 maintains its current position by hovering and returns to step 152 in Figure 5.

[0079] In step 152, it is determined whether or not the object has been encountered. If the determination is positive, the process proceeds to step 154.

[0080] Furthermore, during flight (propulsion) until approaching the rotor blade 22A, the humanoid robot 10 can achieve stable flight by extending its wing section 10A. When it reaches a position facing the rotor blade 22A, the wing section 10A is retracted so as not to interfere with the task to be performed.

[0081] In step 154, the robot tool 28L is positioned opposite the rotor blade 22A, and information about the rotor blade 22A is detected using the palm sensor 32 (high-resolution camera and MoPU).

[0082] In the next step 156, the detection information from the palm sensor 32 is analyzed to determine the type (shape, size, hardness, material, etc.) and position of the rotor blade 22A in detail, and then the process proceeds to step 158.

[0083] In step 158, the task (job type) for the rotor blade 22A is selected. Then, the process moves to step 160, where the robot tool 28R, 30EXA, 30EXB, or 30EXC is selected according to the attributes of the rotor blade 22A (see Figure 6 for details), and the process moves to step 162.

[0084] Step 162 performs work on the rotor blade 22A.

[0085] In the next step 164, it is determined whether the work on the rotor blade 22A was successful. If the determination is positive, the necessary post-processing is performed, and the process proceeds to step 150 to await instructions for the next work on the rotor blade 22A.

[0086] Furthermore, if a negative result is obtained in step 164, the process proceeds to step 166 to perform error handling, and then returns to step 150.

[0087] The robotic tool 28L is equipped with a palm sensor 32 that includes a high-resolution camera and a MoPU, allowing for real-time monitoring of the work status. Even if the work is unsuccessful, quick response (troubleshooting) is possible.

[0088] Furthermore, since the palm sensor 32 (high-resolution camera and MoPU) is mounted on the palm side, it can capture the rotor blade 22A with high precision, enabling it to handle tasks involving minute movements.

[0089] (Details of the robot tool application process)

[0090] Figure 6 is a control flowchart showing the details of the robot tool application processing subroutine.

[0091] In step 200, the robot tool 28R, 30EXA, 30EXB, or 30EXC is selected based on the type of work performed on the rotor blade 22A.

[0092] In other words, the type of robot tool required, whether it be the 28R or 30EXA, 30EXB, or 30EXC, is determined by the type of work performed on the rotor blade 22A.

[0093] In this embodiment, three types of robot tools 30EXA, 30EXB, and 30EXC are attached to the belt 35. Adding robot tool 28R, there are a total of four types of robot tools to choose from: 28R, 30EXA, 30EXB, and 30EXC. However, the number of tools attached to the belt 35 may be increased, or a variety of robot tools may be selected and pre-attached to the belt 28 depending on the work site. Alternatively, different types of robot tools may be attached to each humanoid robot 10.

[0094] In the next step 202, the tool attached to the replacement arm 26R (usually the robot tool 28R, but sometimes another robot tool 30EXA, 30EXB, or 30EXC is already attached) is removed using the robot tool 20L attached to the arm 26L.

[0095] In the next step 204, the replacement arm 26R is moved to the position of the belt 28, and the robot tools 30EXA, 30EXB, and 30EXC are attached.

[0096] In the next step 206, the robot tool that was originally attached to the replacement arm 26R and is being held by the non-replacement arm 26L is stored in the holder on the belt 35, and this routine ends.

[0097] As described above, in this embodiment, a flight device 12 is used on which a humanoid robot 10 is mounted so as to be suspended from a drone 14, and the AI ​​control of the humanoid robot 10 is used to perform work control using robot tools 28L, 28R, 30EXA, 30EXB, and 30EXC.

[0098] This allows for precise work to be performed by AI control, even in dangerous locations or harsh environments where it would be risky for workers to perform tasks, simply by using the drone's flight capabilities to move it close to the target object (the rotor blades 22A of the wind turbine equipment 20), without the need for any safety measures to avoid danger.

[0099] Furthermore, when moving (flying) to the target object (the rotor blades 22A of the wind power generation equipment 20), relatively high-speed flight is involved, so the humanoid robot 10 can stabilize its flight by extending its wing section 10A. After arriving at the target object, the wing section 10A can be retracted and will not interfere with the work.

[0100] In this embodiment, the object in question is the rotor blade 22A of the wind power generation equipment 20, but it may also be an object installed in other high places or in areas or locations with poor environmental conditions, such as high-rise buildings, steel towers, and power lines.

[0101] (Variation 1) As shown in Figures 7(A) and (B), the humanoid robot 10 in this embodiment may be equipped with an arm unit 50 that is suspended from the drone 14 and includes a camera 46 attached to a bellows-shaped, deformable arm 44, and a pair of arm sections 48L and 48R corresponding to the arms 26L and 26R of this embodiment (see Figure 7(A)). The camera 46 has the same function as the palm sensor 26 in this embodiment, and the arm sections 48L and 48R of the arm unit 50 serve the same purpose as the arms 26L and 26R of the humanoid robot 10, so that it can perform the same tasks as a human robot 10.

[0102] Note that the arm sections 48L and 48R are not limited to a pair; there may be one (the arm 26 and robot tool 28 shown in Figure 7(B)) or three or more. Also, the camera 46 attached to the arm section 44 may be a palm sensor 32, as in this embodiment.

[0103] (Modification 2) The flight device 12 of this embodiment is based on the premise that the drone 14 will continue flying to carry out the work, but depending on the working time and working conditions at the work site, it may not be possible to continue flying. In particular, since the flight device 12 continues to consume power when flying, depending on the charge capacity of the onboard battery, it may be necessary to interrupt the work and replace the battery pack or the like.

[0104] Therefore, in the modified version 2 and the modified version 3 described later, the flight device 12 is equipped with magnets (electromagnets) on the legs that are separately attached to the modified version 1, and the flight device 12 is stabilized at the work site by magnetic attachment to a magnetic material by the electromagnet, so that work can be carried out even if the flight device 12 does not continue to fly (see Modified version 2 in Figures 8(A) and (B), and Modified version 3 in Figures 9(A) to (C)).

[0105] In Figures 8(A) and (B), and Figures 9(A) to (C), examples of a flight device 12 in which an arm unit 50 is attached to the drone 14 shown in Figure 7(B) are shown, but this is not the only example. A flight device 12 in which a humanoid robot 10 shown in Figure 1 is suspended from the drone 14, or a flight device equipped with a pair of arm sections 48L and 48R shown in Figure 7(A) may be fitted with legs that have electromagnets attached separately. In Figure 9(B), the contact surface is shown as a surface parallel to the ground, but it does not necessarily have to be parallel to the ground; it may be a slope or a ceiling surface.

[0106] Alternatively, a camera 46 may be mounted on a bellows-shaped, deformable arm portion 44 as shown in Figures 7(A) and (B), or a camera may be attached to at least one wrist of a pair of arm portions 48L and 48R.

[0107] As shown in Figure 8(A), the flight device 12 has legs 102 attached to the four corners of the main body 14A, which serve as support parts 100. The legs 102 extend toward the ground in the vertical direction of the flight device 12.

[0108] An electromagnet 104 is attached to the tip of the leg portion 102 in the extension direction, and by supplying and cutting off power from a battery pack 106 that is detachably attached to the rear side of the main body 14 during flight, it is possible to magnetically attach to and detach from magnetic materials (not shown) at the work site.

[0109] Therefore, before the flight device 12 arrives at the work site and the arms 48L and 48R begin work, the electromagnet section 104 of the leg section 102 is magnetically attached to the magnetic material at the work site. This allows flight to be interrupted while the arms 48L and 48R are working, enabling stable operation and saving power because the power consumption during flight is extremely low compared to the power consumption of the electromagnet section 104. Furthermore, when tightening bolts while flying in the air, the propeller flight control needs to withstand the torque of the bolt tightening, but the magnetic attachment to the support surface allows it to withstand the torque of the bolt tightening.

[0110] In the flight device 12 shown in Figure 8(A), the length of the legs 102 was uniform and fixed, making magnetic attachment difficult on sloped or uneven ground. Therefore, as shown in Figure 8(B), the legs 102A may be individually extendable and retractable.

[0111] (Variation 3) As shown in Figure 9(A), the flight device 12 according to Modification 3 has a structure based on Modification 2 shown in Figure 8(A) or Figure 8(B), and is provided with a flat surface holding part 101A and a wall surface holding part 101B as holding parts.

[0112] The plane-mounted support unit 101A has legs 110 that extend toward the ground in the vertical direction of the flight device 12, and a plane-mounted electromagnet unit 112 is attached to its tip.

[0113] Furthermore, the wall-mounted support unit 101B is equipped with legs 114 that extend toward the work object on the side to which the arms 48L and 48R of the flying device 12 are attached, and a wall-mounted electromagnet unit 116 is attached to the tip of the legs.

[0114] As shown in Figure 9(B), if there is a support surface 118 on the plane of the support that can be magnetically attached, the legs 110 of the flat surface holding part 101A are grounded to this support surface 118, and the flat surface electromagnet part 112 is selected and magnetically attached to the support surface 118.

[0115] As shown in Figure 9(C), if there is a support surface 120 on the wall surface of the support body that can be magnetically attached, the legs 114 of the wall-mounted holding part 101B are grounded to this support surface 120, and the wall-mounted electromagnet part 116 is selected and magnetically attached to the support surface 120.

[0116] According to Modification 3, the magnetic attachment position can be selected depending on the conditions at the work site, thus increasing the degree of freedom in the work while the magnetic attachment is in place.

[0117] Furthermore, by selecting and energizing either the flat surface electromagnet 112 of the flat surface holding unit 101A or the wall surface electromagnet 116 of the wall surface holding unit 101B, power consumption can be reduced compared to always energizing both. Also, when tightening bolts while flying in the air, the propeller's flight control needs to withstand the torque of the bolt tightening, and because it is magnetically attached to the support surface, it can withstand the torque of the bolt tightening.

[0118] Figure 10 schematically shows an example of the hardware configuration of a computer 1200 that functions as an information processing device 14. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to this embodiment, or to cause the computer 1200 to execute operations associated with the device according to this embodiment or such one or more "parts", and / or to cause the computer 1200 to execute a process or a stage of such process according to this embodiment. Such a program may be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0119] The computer 1200 according to this embodiment includes a CPU 1212, RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive and a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive and a solid-state drive, etc. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0120] The CPU 1212 operates according to programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 and stores it in a frame buffer provided in RAM 1214 or within itself, so that the image data is displayed on the display device 1218.

[0121] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0122] The ROM 1230 stores boot programs and / or hardware-dependent programs of the computer 1200, which are executed by the computer 1200 upon activation. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

[0123] The program is provided on a computer-readable storage medium such as a DVD-ROM or IC card. The program is read from the computer-readable storage medium and installed on a storage device 1224, RAM 1214, or ROM 1230, which are examples of computer-readable storage media, and executed by the CPU 1212. The information processing described within these programs is read by the computer 1200, resulting in coordination between the program and the various types of hardware resources described above. The apparatus or method may be configured to realize the operation or processing of information in accordance with the use of the computer 1200.

[0124] For example, when communication is performed between a computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into RAM 1214 and, based on the processing described in the communication program, instruct the communication interface 1222 to perform communication processing. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as RAM 1214, storage device 1224, DVD-ROM, or IC card, transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area provided on the recording medium.

[0125] Furthermore, the CPU 1212 may read all or necessary parts of a file or database stored on an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), or an IC card into the RAM 1214, and perform various types of processing on the data in the RAM 1214. The CPU 1212 may then write the processed data back to the external recording medium.

[0126] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 1212 may perform various types of processing on the data read from RAM 1214, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., as described throughout the present invention and specified by the program instruction sequence, and write the results back to RAM 1214. The CPU 1212 may also retrieve information in files, databases, etc., within the recording medium. For example, if multiple entries are stored in the recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 may search among the multiple entries for an entry that matches the specified condition for the attribute value of the first attribute, read the attribute value of the second attribute stored in that entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies the predetermined condition.

[0127] The program or software module described above may be stored on or near the computer 1200 in a computer-readable storage medium. Alternatively, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0128] In this embodiment, blocks in the flowchart and block diagram may represent a stage in a process in which an operation is performed or a "part" of a device that has the role of performing an operation. A particular stage and "part" may be implemented by a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include reconfigurable hardware circuits, such as field-programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), which include logical AND, logical OR, exclusive OR, negated AND, negated OR, and other logical operations, flip-flops, registers, and memory elements.

[0129] A computer-readable storage medium may include any tangible device capable of storing instructions to be executed by a suitable device, and as a result, a computer-readable storage medium having instructions stored therein will comprise a product that includes instructions that can be executed to create means for performing operations specified in a flowchart or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital multipurpose disc (DVD), Blu-ray® disc, memory stick, integrated circuit card, etc.

[0130] Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, Java®, C++, and traditional procedural programming languages ​​such as the C programming language or similar languages.

[0131] Computer-readable instructions may be provided to a general-purpose computer, a special-purpose computer, or a programmable circuit, either locally or via a wide area network (WAN) such as a local area network (LAN) or the internet, so that the computer-readable instructions may be executed by the processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, in order to generate means for performing operations specified in a flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

[0132] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0133] It should be noted that the execution order of operations, procedures, steps, and stages in the apparatus, systems, programs, and methods described in the claims, specifications, and drawings is not explicitly stated as "before" or "prior to," and that these can be implemented in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," and "next," for convenience, this does not mean that it is essential to perform the operations in that order.

[0134] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0135] It should be noted that the execution order of operations, procedures, steps, and stages in the apparatus, systems, programs, and methods described in the claims, specifications, and drawings is not explicitly stated as "before" or "prior to," and that these can be implemented in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," and "next," for convenience, this does not mean that it is essential to perform the operations in that order. [Explanation of Symbols]

[0136] 10 Humanoid robot, 10A Wing section, 12 Flight device, 14 Drone, 14A Main body, 14B Propeller, 14C Camera, 14D Holding plate, 20 Wind power generation equipment, 22A Rotary wing, 24 Upper body section, 25 Lower body section, 25A Leg section, 25B Electromagnet section, 26 (L, R) Arm section, 28 (L, R) Robot tool, 30EX (A~C) Robot tool, 32 Palm sensor, 34 Information processing device, 36 Sensor, 38 Information acquisition unit, 40 Control unit, 42 Information storage unit, 44 Arm section, 46 Camera, 48 (L, R) Arm section, 50 Arm unit, 100 Holding unit, 102 (102A) Leg section, 104 Electromagnet section, 106 Battery pack, 101A Planar holding unit, 101B Wall mounting section, 110 Leg section, 112 Planar electromagnet section, 114 Leg section, 116 Wall electromagnet section, 1200 Computer, 1210 Host controller, 1212 CPU, 1214 RAM, 1216 Graphics controller, 1218 Display device, 1220 Input / Output controller, 1222 Communication interface, 1224 Storage device, 1230 ROM, 1240 Input / Output chip

Claims

1. A robot control system that involves positioning a robot attached to a drone to confront an object and perform tasks on the object, With the drone's flight stopped, a holding unit maintains the robot's facing position toward the object, At least a sensor to identify the location of the object being worked on, A determination unit for determining the type of work to be performed on the aforementioned object, A control unit controls the operation of attaching a tool corresponding to the type of work determined by the determination unit to the robot, and controls the work according to the type of work based on the detection results of the sensor. A robot control system having

2. The robot control system according to claim 1, wherein the holding portion comprises an electromagnet portion that can be magnetically attached to the object or a magnetic support surface provided near the object, and a leg portion that faces the support surface toward the magnetic attachment portion.

3. The robot control system according to claim 1, wherein the robot is a humanoid robot.

4. The robot control system according to claim 1, wherein the determination unit makes a determination based on information from a sensor unit mounted on the tool, which includes a camera that captures an image of the object and identifies the type of the object, and a motion processing unit that identifies the position of the object.

5. A flying device that is detachable from the wrist portion of an arm that moves in three dimensions, and which is equipped with tools according to the type of work to be done on the object, and which performs work on the object, A drone capable of flying to a designated location, A robot unit is attached to the drone and includes a sensing unit that identifies at least the location of the work object and the tool, and which, while facing the work object, attaches the tool according to the type of work to be done on the work object and autonomously performs the work. The robot comprises an electromagnet portion that can be magnetically attached to the object or a magnetic support surface provided near the object, and a leg portion that faces the electromagnet portion toward the support surface, and a holding portion that maintains the robot facing the object when the drone's flight is stopped, A flying device having

6. A robot control program that causes a computer to operate as the determination unit and the control unit according to any one of claims 1 to 4.

Images