Flight control system

Abstract

A flight control system 1 comprises: an unmanned flying body 10 which is provided with a millimeter-wave radar; a control device 20 which controls the operation of the unmanned flying body; and a port 30 on the surface of which three or more reflectors are disposed. The control device 20 is provided with: a measurement unit 21 that measures the three-dimensional positions of the three or more reflectors by using the millimeter-wave radar; and an estimation unit 22 that estimates the central position of the port as viewed from the unmanned flying body, and performs said estimation by using the emission direction of the millimeter-wave radar, the measured positions of the three or more reflectors subjected to measurement, and the positions at which the three or more reflectors are disposed in relation to the central position of the port.

Description

Flight Control System

[0001] The present disclosure relates to flight control systems.

[0002] There are technologies for measuring the landing point of a drone using a camera, LiDAR (Light Detection and Ranging), and millimeter-wave radar (see Non-Patent Documents 1 to 8). Non-Patent Document 8 discloses a technology for estimating the slope of the landing surface of a drone using millimeter-wave radar.

[0003] Phong Ha Nguyen, 5 others, "LightDenseYOLO: A fast and Accurate Marker Tracker for Autonomous UAV Landing by Visible Light Camera Sensor on Drone", Sensors 2018, 18, 1703Jonghwi Kim, 2 others, "Lidar-guided Autonomous Landing of an Aerial Vehicle on a Ground Vehicle", 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), IEEE, 2017, June 28 - July 1, 2017Akash Deep Singh, 3 others, "RADHAR: Human Activity Recognition from Point Clouds Generated through a Millimeter-wave Radar", Proceedings of the 3rd ACM Workshop on Millimeter-wave Networks and Sensing Systems, October 25, 2019, p.51-p.56Girish Tiwari, 2 others, "mmFiT: Contactless Fitness Tracker Using mmWave Radar and Edge Computing Enabled Deep Learning", IEEE INTERNET OF THINGS JOURNAL, 2021Jaime Lien, 7 others, "Soli: Ubiquitous Gesture Sensing with Millimeter Wave Radar", ACM Transactions on Graphics, Vol.35, No.4, Article 142, July 2016Christopher Doer, 3 others, “Radar Based Autonomous Precision Takeoff and Landing System for VTOLs in GNSS Denied Environments”, 2020 International Conference on Unmanned Aircraft Systems (ICUAS), IEEE, 2020, September 1-4, 2020Tatsuya Iizuka, 5 others, “MilliSign: mmWave-Based Passive Signs for Guiding UAVs in Poor Visibility Conditions”, Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. 2023, October 2-6, 2023, p.752-p.766Tatsuya Iizuka and 5 others, “An inclination estimation method for UAV landing surfaces using millimeter wave radar”, in IGARSS 2023-2023 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2023, p.2049-p.2052.

[0004] However, in Non-Patent Document 8, a device that requires a power source (e.g., a GPS sensor) is used to measure the relative position between the drone and the landing surface, which increases the installation cost of the flight control system. Furthermore, in addition to processing the millimeter-wave radar signal, the system also performs integrated processing of various data obtained by various sensors, including the GPS sensor, which increases the development cost of the flight control system.

[0005] The present disclosure has been made in consideration of the above circumstances, and an object of the present disclosure is to provide a technology that can estimate the center position of a port at low cost.

[0006] One embodiment of the flight control system of the present disclosure is a flight control system comprising an unmanned aerial vehicle equipped with a millimeter-wave radar, a control device that controls the operation of the unmanned aerial vehicle, and a port having three or more reflectors arranged on its surface, wherein the control device comprises a measurement unit that measures the three-dimensional positions of the three or more reflectors using the millimeter-wave radar, and an estimation unit that estimates the center position of the port as seen from the unmanned aerial vehicle using the irradiation direction of the millimeter-wave radar, the measured positions of the three or more reflectors, and the placement positions of the three or more reflectors relative to the center position of the port.

[0007] According to the present disclosure, a technique can be provided that can estimate the center position of a port at low cost.

[0008] FIG. 1 is a diagram showing the overall configuration of a flight control system. FIG. 2 is a diagram showing an image of millimeter-wave radar signal processing. FIG. 3 is a flow chart showing a method for estimating the center position of a port. FIG. 4 is a reference diagram for estimating the center position and inclination of a port. FIG. 5 is a flow chart showing a method for estimating the inclination of a port. FIG. 6 is a perspective view of the flight control system 1 used during the experiment. FIG. 7 is a top view of the flight control system 1 used during the experiment. FIG. 8 is a diagram showing a distance intensity distribution calculated by millimeter-wave radar measurement. FIG. 9 is a diagram showing an example of the estimation result of port inclination. FIG. 10 is a diagram showing an example of the hardware configuration of a control device.

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0010] [Summary of the Present Disclosure] The present disclosure arranges three or more reflectors on a port, measures the three-dimensional positions of the reflectors using a millimeter-wave radar, and estimates the center position of the port using the measured positions of the reflectors and the arrangement positions (position vectors) of the multiple reflectors relative to the center position of the port.

[0011] Furthermore, the present disclosure uses a millimeter-wave radar to measure the distances between the millimeter-wave radar and multiple reflectors, and estimates the inclination of the port using the measured distances between the reflectors, the accurate measured distances between the reflectors, and the estimated center positions of the ports.

[0012] This allows the center position and inclination of the port to be estimated simply by processing the millimeter-wave radar signal (including estimation processing), even if the relative position between the drone and the port is unknown. In addition, since the reflector does not require a power source, estimation can be performed at low cost.

[0013] 1 is a diagram showing the overall configuration of a flight control system 1 according to this embodiment. The flight control system 1 includes a drone 10, a control device 20, and a port 30.

[0014] The drone 10 is an unmanned aerial vehicle (UAV). The drone 10 is equipped with a millimeter wave radar 11.

[0015] The port 30 is installed on, for example, a sloping ground. The port 30 may also be installed on the bed of a moving truck, the deck of a ship that is rocking with waves, or the like. N reflectors 31 (N is a natural number equal to or greater than 3) are arranged on the surface of the port 30. The reflectors 31 are, for example, small corner reflectors.

[0016] The control device 20 is a controller (e.g., a computer, software, or circuit) that is mounted on the drone 10 and controls the operation of the drone 10. The control device 20 may be not mounted on the drone 10 and may control the flight of the drone 10 via wireless communication.

[0017] As shown in FIGS. 1 and 2, the control device 20 determines the irradiation direction of the millimeter wave radar 11 and the arrangement positions s of the N reflectors 31 relative to the center position p of the port 30. ｉ Using the above, the center position p and the inclination θ of the port 30 as seen from the drone 10 are calculated. Ｐ For example, the control device 20 includes a measurement unit 21, an estimation unit 22, a control unit 23, and a storage unit 24.

[0018] The measurement unit 21 measures the three-dimensional positions v of N reflectors 31 using the millimeter wave radar 11. ｉ It has the function to measure (i=1, 2, ..., N).

[0019] The measurement unit 21 measures the measurement positions v ｉ Using the above, the distance d between the millimeter wave radar 11 and the N reflectors 31 is calculated. ｉ It has the function to measure.

[0020] The estimation unit 22 estimates the irradiation direction of the millimeter wave radar 11 and the measurement positions v of the N reflectors 31 measured by the measurement unit 21. ｉ and the arrangement positions s of the N reflectors 31 relative to the center position p of the port 30. ｉ and has the function of estimating and calculating the center position p of the port 30 as seen from the drone 10.

[0021] The arrangement positions s of the N reflectors 31 relative to the center position p of the port 30 ｉ is a position vector s of the reflector 31 starting from the center position p of the port 30. ｉ Therefore, the positions v of the N reflectors 31 are ｉ Once measured, the position v ｉ position vector s ｉ By converting the value of the center position p of the port 30 into the value of the center position p of the port 30, the center position p of the port 30 can be estimated.

[0022] The estimation unit 22 estimates and calculates the center position p of the port 30 and the measured distance d between the millimeter wave radar 11 and the N reflectors 31 measured by the measurement unit 21. ｉ and the accurate measurement distance d between the millimeter wave radar 11 and the N reflectors 31. ｉ－ｍｅｓ (mes is an abbreviation for measurement, not shown in FIG. 2 ) and the inclination θ of the port 30 as seen from the drone 10 Ｐ It has the function to estimate and calculate the following.

[0023] If the center position p of the port 30 is estimated, the rotation angles of the port 30 around the x-axis, y-axis, and z-axis are determined by minimizing the error in the distance from the millimeter-wave radar 11 to the N reflectors 31. Therefore, the inclination θ of the port 30 is Ｐ can be estimated.

[0024] The control unit 23 estimates and calculates the center position p and inclination θ of the port 30. Ｐ It has the function of controlling the position and inclination of the drone 10 using the above.

[0025] The storage unit 24 stores, for example, the central position p and the inclination θ of the port 30. Ｐ Various formulas for calculating the above, various data used in the calculation of the various formulas, the center position p of the port 30 and the inclination θ Ｐ It has a function to store the estimated value of

[0026] The measurement unit 21 and the estimation unit 22 may be a single functional unit.

[0027] [Method for Estimating the Center Position p of the Port 30 as Seen by the Drone 10] Fig. 3 is a flow diagram showing a method for estimating the center position p of the port 30. Fig. 4 is a flow diagram showing a method for estimating the center position p of the port 30 and the inclination θ Ｐ This is a reference diagram for estimating V(x ｖ , y ｖ , z ｖ ) is the coordinate system of the drone 10 (millimeter wave radar 11). ｖ’ , y ｖ’ , z ｖ’ ) is the coordinate system of the non-tilted port 30. ｐ , y ｐ , z ｐ ) is the coordinate system of the tilted port 30.

[0028] Step S101: The user places N reflectors 31 on the port 30. Each reflector 31 is formed to be strongly reflective in three dimensions so that the drone 10 can measure the position of each reflector 31 using the millimeter-wave radar 11.

[0029] Step S102: When the center position p of the port 30 is set as the origin of the coordinate system V′, P of the port 30, the arrangement position s of each reflector 31 relative to the origin is calculated. ｉ In other words, since the center position p of the port 30 is on the same plane as the N reflectors 31, the arrangement positions of all the reflectors 31 (=position vectors of each reflector 31 starting from the center position p of the port 30) s ｉIt can be expressed as a linear sum of

[0030] Specifically, the center position p of the port 30 is determined by the arrangement positions (position vectors) s of the N reflectors 31. ｉ The coefficient of k is known ｉ Then, the position (position vector) s of each reflector 31 is ｉ Using this, it can be expressed as in equation (1).

[0031]

[0032] In addition, N in the formula (1) is a natural number of 4 or more.

[0033] The control device 20 stores the formula (1) in advance in the storage unit 24 .

[0034] The above is a preliminary preparation for estimating the center position p of the port 30. The following is a process for estimating the center position p of the port 30.

[0035] Step S103: The drone 10 irradiates millimeter waves from the millimeter wave radar 11 toward the port 30. The measurement unit 21 of the control device 20 analyzes the reflected signals from the N reflectors 31 arranged on the port 30 to calculate the three-dimensional position v ｉ Measure.

[0036] Step S104: The estimation unit 22 of the control device 20 reads out the equation (1) from the storage unit 24 and calculates the measured positions v of the N reflectors 31 measured in step S103. ｉ The placement position (position vector) s ｉ and calculating the equation (2) after substitution, the center position p of the port 30 is calculated.

[0037]

[0038] It should be noted that N in equation (2) is also a natural number of 4 or more.

[0039] Step S104 will now be explained in more detail.

[0040] In this embodiment, the center position p of the port 30 as seen from the drone 10 is calculated. This means that the center position p of the port 30 is obtained in the coordinate system V of the drone 10, which is defined by the position and attitude of the drone 10.

[0041] The measurement positions v of the N reflectors 31 measured in step S103 ｉ is the measurement position as seen from the coordinate system of the millimeter-wave radar 11 (a coordinate system in which the irradiation direction of the millimeter-wave radar 11 is the X-axis; not shown in FIG. 4). Therefore, the center position p of the port 30 calculated by equation (2) is the center position p of the port 30 as seen from the coordinate system of the millimeter-wave radar 11.

[0042] The estimation unit 22 of the control device 20 calculates the center position p of the port 30 as seen from the drone 10 by converting the center position p of the port 30 as seen from the coordinate system of the millimeter wave radar 11 into a position in the coordinate system V of the drone 10.

[0043] It is assumed that the irradiation direction of the millimeter-wave radar 11 from the drone 10 with respect to the attitude of the drone 10 is known. The center position p of the port 30 obtained by signal analysis of the millimeter-wave radar 11 is calculated using distance, azimuth angle, and elevation angle with the irradiation point of the millimeter waves as a reference. Therefore, in order to know the relative position between the drone 10 and the center position p of the port 30 from the center position p of the port 30 obtained by signal analysis of the millimeter-wave radar 11, it is necessary to know in advance the irradiation direction of the millimeter-wave radar 11 with respect to the attitude of the drone 10.

[0044] [The inclination θ of the port 30 as seen from the drone 10 Ｐ 5 shows the method for estimating the inclination θ of the port 30. Ｐ FIG. 10 is a flow chart showing a method for estimating

[0045] Step S201: Determine the attitude of the port 30 relative to the attitude of the drone 10 (tilt θ Ｐ ) are the roll angles θ around the x-axis, y-axis, and z-axis in the coordinate system V′, P of the port 30. ｒｏｌｌ ｐ , pitch angle θ ｐｉｔｃｈ ｐ , yaw angle θ ｙａｗ ｐ When this is done, (θ ｒｏｌｌｐ , θ ｐｉｔｃｈ ｐ , θ ｙａｗ ｐ When the center position p of the port 30 is known, the distance d between the millimeter wave output end e of the millimeter wave radar 11 and the N reflectors 31 can be expressed as ｉ can be expressed as in equation (3).

[0046]

[0047] In addition, R ｘ , R ｙ , R ｚ are rotation matrices around the x-axis, y-axis, and z-axis in the coordinate systems V′ and P of the port 30, respectively.

[0048] The control device 20 stores the formula (3) in advance in the storage unit 24 .

[0049] Step S202: Also, the distance d in equation (3) ｉ The exact distance d calculated separately ｉ－ｍｅｓ Equation (4) is generated in advance to solve the equation with the least square error.

[0050]

[0051] It should be noted that N in equation (4) is also a natural number of 4 or more.

[0052] The control device 20 stores the formula (4) in advance in the storage unit 24 .

[0053] Up to this point, the inclination θ of the landing port PT Ｐ This is a preliminary preparation for estimating the inclination θ of the port 30. Ｐ This is the process for estimating the

[0054] Step S203: The drone 10 irradiates millimeter waves from the millimeter wave radar 11 toward the port 30. The measurement unit 21 of the control device 20 analyzes the reflected signals from the N reflectors 31 installed on the port 30 to calculate the three-dimensional position v ｉ Measure.

[0055] Thereafter, the measurement unit 21 of the control device 20 calculates the measurement positions v of the measured N reflectors 31. ｉUsing the above, the distance d between the millimeter wave output terminal e of the millimeter wave radar 11 and the N reflectors 31 is calculated. ｉ Measure.

[0056] Step S204: The estimation unit 22 of the control device 20 reads out the formula (3) from the storage unit 24, and substitutes the center position p of the port 30 calculated in step S104 into the formula (3).

[0057] The estimation unit 22 also reads out the equation (4) from the storage unit 24 and calculates the distance d between the millimeter wave output terminal e of the millimeter wave radar 11 and the N reflectors 31 calculated in step S203. ｉ and the accurate distance d between the millimeter wave radar 11 and the N reflectors 31, which has been measured in advance. ｉ－ｍｅｓ and are substituted into equation (4).

[0058] Then, the estimation unit 22 calculates the distance d ｉ－ｍｅｓ and distance d ｉ Minimize the difference between (θ ｒｏｌｌ ｐ , θ ｐｉｔｃｈ ｐ , θ ｙａｗ ｐ ) and calculate the inclination θ of the port 30 as seen from the drone 10. Ｐ Calculate.

[0059] Note that steps S101 to S104 and steps S201 to S204 may be combined as appropriate.

[0060] [Experimental Results] Fig. 6 is a perspective view of the flight control system 1 used in the experiment. Fig. 7 is a top view of the flight control system 1 used in the experiment.

[0061] The millimeter-wave radar 11 was placed at a position (x, y, z) = (1 m, 0 m, 1 m) in a coordinate system with the center position of the port 30 shown in Figure 6 as the origin. The irradiation direction of the millimeter-wave radar 11 was fixed so as to face the center of the port 30.

[0062] The number of reflectors 31 was four. The reflectors 31 were arranged at positions s ｉ polar coordinates (r ｉ , φ ｉ , θ ｉ ) (0≦i<4), (rｉ , θ ｉ ) = (0.4m, 90°), (φ ０ , φ １ , φ ２ , φ ３ ) = (0°, 300°, 120°, 180°). A disk-shaped polystyrene foam with a thickness of 3 cm and a radius of 0.5 m was used as the reflector 31. A hole was cut out above the port 30, and the reflector 31 was fitted and fixed.

[0063] The millimeter-wave radar 11 uses a 79-GHz millimeter-wave FMCW radar, and utilizes a frequency band from 77 to 81 GHz. It is designed to achieve a distance resolution of 37.5 mm. The MIMO antenna, which has three transmitting antennas and four receiving antennas, calculates the azimuth and elevation angles at which the reflector 31 is located, and calculates the three-dimensional position v of the reflector 31. ｉ and distance d ｉ was measured.

[0064] The three-dimensional position v of the reflector 31 ｉ The calculation was performed using Root-MUSIC, which is disclosed in "Tatsuya Iizuka and three others, "Root-MUSIC Based Power Estimation Method with Super-Resolution FMCW Radar," 2020 IEEE / MTT-S International Microwave Symposium (IMS). IEEE, 2020, pp. 1027-1030."

[0065] Figure 8 shows the distance-intensity distribution calculated by millimeter-wave radar measurement. The horizontal axis represents the distance from the millimeter-wave radar, and the vertical axis represents the intensity. Four intensities were obtained near the port (near 1.4 m), indicating that four reflectors were detected.

[0066] 9 is a diagram showing an example of the estimation result of the inclination of the port. The three-dimensional position of the reflector is measured, and the degree of inclination of the port can be determined.

[0067] [Effects of the embodiment] According to the present embodiment, the center position of the port as seen from the drone is estimated using the irradiation direction of the millimeter wave radar, the measurement positions of the N reflectors, and the placement positions of the N reflectors relative to the center position of the port, so that the center position of the port can be estimated at low cost.

[0068] Furthermore, according to this embodiment, the inclination of the port as seen from the drone is estimated using the center position of the port, the measured distance between the millimeter-wave radar and the N reflectors, and the accurate measured distance between the millimeter-wave radar and the N reflectors, thereby enabling the inclination of the port to be estimated at low cost.

[0069] In other words, the center position and inclination of the port can be estimated with high accuracy and low cost using only a power-free landing port with an embedded reflector and millimeter-wave radar signal processing in the drone's control device.

[0070] This will enable drones to land on moving surfaces such as truck beds, ship decks, and offshore floating ports in all weather conditions, including poor visibility conditions such as fog, rain, and nighttime, thereby expanding the scope of drone use in the delivery industry and marine environment observation fields.

[0071] [Others] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0072] The control device 20 of the present embodiment described above can be realized, for example, by using a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in Fig. 10. The memory 902 and the storage 903 are storage devices. In the computer system, the CPU 901 executes a predetermined program loaded onto the memory 902, thereby realizing each function of the control device 20.

[0073] The control device 20 may be implemented by one computer, or by multiple computers, or may be a virtual machine implemented on a computer.

[0074] The program for the control device 20 can be stored in a computer-readable recording medium such as a HDD, an SSD, a Universal Serial Bus (USB) memory, a Compact Disc (CD), or a Digital Versatile Disc (DVD). The computer-readable recording medium is, for example, a non-transitory recording medium. The program for the control device 20 can also be distributed via a communication network.

[0075] REFERENCE SIGNS LIST 1 Flight control system 10 Drone 11 Millimeter wave radar 20 Control device 21 Measurement unit 22 Estimation unit 23 Control unit 24 Storage unit 30 Port 31 Reflector 901 CPU 902 Memory 903 Storage 904 Communication device 905 Input device 906 Output device

Claims

1. A flight control system comprising an unmanned aerial vehicle equipped with a millimeter-wave radar, a control device that controls the operation of the unmanned aerial vehicle, and a port with three or more reflectors arranged on its surface, wherein the control device comprises: a measurement unit that measures the three-dimensional positions of the three or more reflectors using the millimeter-wave radar; and an estimation unit that estimates the center position of the port as seen from the unmanned aerial vehicle using the irradiation direction of the millimeter-wave radar, the measured positions of the three or more reflectors, and the arrangement positions of the three or more reflectors relative to the center position of the port.

2. A flight control system as described in claim 1, wherein the measurement unit measures the distance between the millimeter-wave radar and the three or more reflectors using the measured positions of the three or more reflectors, and the estimation unit estimates the inclination of the port as seen from the unmanned aerial vehicle using the estimated center position of the port, the measured distance between the millimeter-wave radar and the three or more reflectors, and the accurate measured distance between the millimeter-wave radar and the three or more reflectors.

Images