Take-off and landing method and equipment of unmanned aerial vehicle on naval vessel, medium and program product

Abstract

The embodiment of the invention discloses a take-off and landing method and equipment of an unmanned aerial vehicle on a naval vessel, a medium and a program product. The method comprises the following steps: respectively acquiring first environment data and second environment data of the unmanned aerial vehicle through a camera and a laser radar arranged on the unmanned aerial vehicle; fusing the first environment data with the second environment data to obtain multi-dimensional sensing environment data; planning the flight path of the unmanned aerial vehicle according to the multi-dimensional perception environment data, the flight task demand of the unmanned aerial vehicle and the position of the naval vessel; the relative swing amplitude of the naval vessel and the unmanned aerial vehicle is collected, and the take-off speed and the landing speed of the unmanned aerial vehicle are determined according to the multi-dimensional sensing environment data and the relative swing amplitude; and according to the take-off speed, the flight path and the landing speed, the unmanned aerial vehicle is controlled to take off and fly on the naval vessel to execute a task and land on the naval vessel. Take-off and landing safety of the unmanned aerial vehicle can be guaranteed through data fusion of the camera and the laser radar, and take-off and landing safety and stability of the unmanned aerial vehicle can be guaranteed through speed adjustment of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles (UAVs), and in particular to a method, equipment, medium, and program product for taking off and landing a UAV on a ship. Background Art

[0002] Due to the excellent functions of drones, they are widely used in surveying and mapping, geology, petroleum, agriculture and forestry and other fields.

[0003] In existing technologies, autonomous drone takeoff and landing primarily relies on camera image acquisition to plan the drone's path. However, purely visual drone takeoff and landing control lacks safety due to its inability to obtain accurate environmental data. Furthermore, existing autonomous drone takeoff and landing systems typically fail to account for the instability caused by changes in the takeoff and landing environment. Summary of the Invention

[0004] The present invention provides a method, equipment, medium and program product for taking off and landing a UAV on a ship, so as to ensure the stability and safety of the UAV during autonomous take-off and landing.

[0005] According to one aspect of the present invention, a method for taking off and landing a UAV on a ship is provided, the method comprising:

[0006] The first environment data and the second environment data of the drone are collected respectively by a camera and a laser radar provided on the drone;

[0007] fusing the first environmental data with the second environmental data to obtain multi-dimensional perceived environmental data;

[0008] Planning a flight path for the UAV based on the multi-dimensional perceived environmental data, the UAV's flight mission requirements, and the ship's position;

[0009] collecting the relative swing amplitude between the ship and the UAV, and determining the takeoff speed and landing speed of the UAV based on the multi-dimensional sensed environment data and the relative swing amplitude;

[0010] According to the take-off speed, the flight path, and the landing speed, the UAV is controlled to take off from the ship, fly to perform a mission, and land on the ship.

[0011] According to another aspect of the present invention, a take-off and landing device for a UAV on a ship is provided, the device comprising:

[0012] An environmental data acquisition module is used to respectively collect first environmental data and second environmental data of the UAV through a camera and a lidar provided on the UAV;

[0013] An environmental data fusion module, configured to fuse the first environmental data with the second environmental data to obtain multi-dimensional perceived environmental data;

[0014] A flight path planning module is used to plan the flight path of the UAV based on the multi-dimensional perception environment data, the UAV flight mission requirements and the ship position;

[0015] a flight speed determination module, configured to collect the relative swing amplitude between the ship and the UAV, and determine the takeoff speed and landing speed of the UAV based on the multi-dimensional sensed environment data and the relative swing amplitude;

[0016] The UAV control module is used to control the UAV to take off from the ship, fly to perform the mission, and land on the ship according to the take-off speed, the flight path, and the landing speed.

[0017] According to another aspect of the present invention, an electronic device is provided, comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for taking off and landing a drone on a ship as described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for taking off and landing a drone on a ship as described in any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method for taking off and landing a UAV on a ship as described in any embodiment of the present invention.

[0023] The technical solution of the embodiment of the present invention is to collect first environmental data and second environmental data of the drone through a camera and a laser radar set on the drone; the first environmental data and the second environmental data are integrated to obtain multi-dimensional perception environmental data; the flight path of the drone is planned according to the multi-dimensional perception environmental data, the drone flight mission requirements and the position of the ship; the relative swing amplitude between the ship and the drone is collected, and the take-off speed and landing speed of the drone are determined according to the multi-dimensional perception environmental data and the relative swing amplitude; according to the take-off speed, flight path, and landing speed, the drone is controlled to take off from the ship, fly to perform missions, and land on the ship, thereby solving the safety and stability problems in the autonomous take-off and landing of the drone. The safety of the drone's take-off and landing can be guaranteed by fusing the camera and laser radar data, and the safety and stability of the drone's take-off and landing can be guaranteed by adjusting the drone speed according to the integrated environmental data and the relative swing amplitude.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 This is a flow chart of a method for taking off and landing a UAV on a ship according to the first embodiment of the present invention;

[0027] Figure 2 This is a flow chart of a method for taking off and landing a UAV on a ship according to a second embodiment of the present invention;

[0028] Figure 3 This is a schematic structural diagram of a take-off and landing device for a UAV on a ship according to a third embodiment of the present invention;

[0029] Figure 4 The figure is a schematic diagram of the structure of an electronic device for implementing the method for taking off and landing a UAV on a ship according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Example 1

[0033] Figure 1 This is a flow chart of a method for taking off and landing a UAV on a ship according to the first embodiment of the present invention. This embodiment is applicable to the case where a UAV automatically takes off and lands on a ship. The method can be executed by a UAV take-off and landing device on a ship. The UAV take-off and landing device on a ship can be implemented in the form of hardware and / or software. The UAV take-off and landing device on a ship can be configured in electronic equipment such as computers, servers, UAV controllers, etc. Figure 1 As shown, the method includes:

[0034] Step 110: The first environment data and the second environment data of the UAV are collected respectively by using the camera and the lidar provided on the UAV.

[0035] The camera may be a single camera or multiple cameras. The first environmental data collected by the camera may be two-dimensional image data. The second environmental data collected by the lidar may be three-dimensional point cloud data. The first environmental data and the second environmental data may be used to identify obstacle data and calibration data during the take-off and landing of the drone and during flight. During the take-off and landing of the drone and the flight, the first environmental data and the second environmental data may be collected in real time by the camera and the lidar, respectively, so as to perform precise drone flight control based on the real-time environmental data. The drone may be a fixed-wing drone. A fixed-wing drone refers to a type of drone with fixed wings whose outer wing sweep angle can be adjusted automatically or manually with speed.

[0036] Step 120: Fuse the first environmental data with the second environmental data to obtain multi-dimensional perception environmental data.

[0037] The fusion of the first and second environmental data can be achieved by splicing. For example, when the first and second environmental data are aligned in time and space, complementary information extraction is performed to obtain multi-dimensional perception environmental data. The first environmental data collected by the camera can provide high-resolution texture information. The second environmental data collected by the lidar can provide accurate distance and obstacle outline data. The multi-dimensional perception environmental data obtained by fusing the first and second environmental data can provide the drone with precise obstacle information and can obtain highly accurate obstacle identification in various weather conditions.

[0038] Step 130: Plan the flight path of the UAV based on the multi-dimensional perception environment data, the UAV flight mission requirements, and the ship's position.

[0039] Multi-dimensional perception environment data can be used to avoid obstacles or accurately land in a designated area when planning a flight path. The UAV flight mission requirements can be used to control the UAV to reach the designated mission location and stay there when planning a flight path. The ship position can be the take-off and landing position of the UAV in the flight path. Therefore, based on the multi-dimensional perception environment data, the UAV flight mission requirements and the ship position, a path planning algorithm can be used to obtain the flight path of the UAV. For example, through the path breadth-first search algorithm (Dijkstra), the heuristic search algorithm (A * ), ant colony algorithm, genetic algorithm or deep learning algorithm, combined with multi-dimensional perception environment data, UAV flight mission requirements and ship position, to plan the UAV's flight path.

[0040] Step 140: Collect the relative swing amplitude between the ship and the UAV, and determine the take-off speed and landing speed of the UAV based on the multi-dimensional environmental data and the relative swing amplitude.

[0041] In the application scenario of an embodiment of the present invention, a drone takes off from a ship to perform a mission and then lands on the ship after completing the mission. Due to the impact of waves and wind, the ship will experience varying degrees of sway. Sensors can be installed on the ship to detect the amplitude of the ship's sway. Simultaneously, sensors can be installed on the drone to detect the drone's sway. The ship's sway amplitude can be transmitted to the drone. Alternatively, the drone's sway status can be transmitted to the ship. Alternatively, both the ship's sway amplitude and the drone's sway status can be transmitted to a designated electronic device. The relative sway amplitudes of the ship and drone can be determined based on the ship's sway amplitude and the drone's sway status.

[0042] When determining a drone's takeoff and landing speeds, relative sway amplitude and multi-dimensional environmental data can be considered. For example, as the relative sway amplitude increases, the drone's vertical ascent speed can be reduced; as the relative sway amplitude decreases, the drone's vertical ascent speed can be increased. If the multi-dimensional environmental data indicates that the distance between the obstacle and the drone is not within a safe range, the drone's speed can be reduced; if the distance is within a safe range, the drone's speed can be increased. In specific applications, multi-dimensional environmental data and relative sway amplitude can be combined to comprehensively determine the drone's takeoff and landing speeds.

[0043] Step 150: Control the UAV to take off from the ship, fly to perform the mission, and land on the ship based on the takeoff speed, flight path, and landing speed.

[0044] The drone can autonomously take off from the ship at its takeoff speed, execute its mission according to its flight path, and autonomously land on the ship at its landing speed. It should be noted that the takeoff speed, flight path, and landing speed are all determined in real time based on real-time environmental data, ensuring the drone's effective safety.

[0045] The technical solution of the embodiment of the present invention is to collect first environmental data and second environmental data of the drone through a camera and a laser radar set on the drone; the first environmental data and the second environmental data are integrated to obtain multi-dimensional perception environmental data; the flight path of the drone is planned according to the multi-dimensional perception environmental data, the drone flight mission requirements and the position of the ship; the relative swing amplitude between the ship and the drone is collected, and the take-off speed and landing speed of the drone are determined according to the multi-dimensional perception environmental data and the relative swing amplitude; according to the take-off speed, flight path, and landing speed, the drone is controlled to take off from the ship, fly to perform missions, and land on the ship, thereby solving the safety and stability problems in the autonomous take-off and landing of the drone. The safety of the drone's take-off and landing can be guaranteed by fusing the camera and laser radar data, and the safety and stability of the drone's take-off and landing can be guaranteed by adjusting the drone speed according to the integrated environmental data and the relative swing amplitude.

[0046] Example 2

[0047] Figure 2 This is a flow chart of a method for taking off and landing a UAV on a ship provided according to the second embodiment of the present invention. This embodiment is a further refinement of the above technical solution. The technical solution in this embodiment can be combined with various optional solutions in one or more of the above embodiments.

[0048] Optionally, the first environmental data and the second environmental data are fused to obtain multidimensional perceived environmental data, including: synchronizing the timestamps and aligning the coordinate systems of the first environmental data and the second environmental data to obtain spatial and temporally consistent initial environmental fusion data; determining the fusion weight of the first environmental data and the second environmental data based on the environmental parameters of the UAV; wherein the environmental parameters include at least one of the following: wind, light, rain, snow, and environmental stability; performing image feature extraction on the first environmental data in the initial environmental fusion data to obtain a first environmental feature, and performing point cloud feature extraction on the second environmental data in the initial environmental fusion data to obtain a second environmental feature; and performing a multimodal attention mechanism to fuse the first environmental feature and the second environmental feature according to the fusion weight to obtain multidimensional perceived environmental data.

[0049] like Figure 2 As shown, the method includes:

[0050] Step 210: The first environment data and the second environment data of the UAV are collected respectively by a camera and a lidar provided on the UAV.

[0051] Step 220: synchronize the timestamps of the first environment data and align the coordinate systems with the second environment data to obtain initial environment fusion data that is consistent in time and space.

[0052] The timestamp synchronization of the first environmental data and the second environmental data can be achieved through network time, such as global positioning system (GPS) time; or, they can be triggered by the same pulse signal so that the camera and the lidar collect data at the same time, thereby achieving time synchronization between the first environmental data and the second environmental data. The coordinate systems of the first environmental data and the second environmental data can be aligned through pre-calibrated coordinates. When the first environmental data and the second environmental data are obtained, the spatial alignment of the first environmental data and the second environmental data can be achieved based on time synchronization according to the calibrated coordinate conversion method, thereby obtaining initial environmental fusion data that is consistent in time and space.

[0053] Step 230: Determine the fusion weight of the first environmental data and the second environmental data based on the environmental parameters of the UAV.

[0054] Environmental parameters include at least one of the following: wind speed, light, rain and snow, and environmental stability. Different environmental parameters have different effects on cameras and lidar. To ensure the accuracy of obstacle detection, a fusion weight can be set when fusing the first and second environmental data. The fusion weight is set based on environmental parameters. For example, when the wind is strong, the light is insufficient, or there is heavy rain or snow, the second environmental data obtained by the lidar can be used first. When the environmental stability is high, the first environmental data obtained by the camera can be used first.

[0055] For example, sensors can be used to collect environmental parameters of the drone, such as wind speed, light, rain and snow, and environmental stability. Once each environmental parameter is obtained, it can be normalized. A fusion weight expression is constructed based on the relationship between each environmental parameter and the camera and lidar. For example, the fusion weight of the second environmental data is γ = α1x1 - α2x2 + α3x3 - α4x4. Here, α1, α2, α3, and α4 are preset weights for wind speed, light, rain and snow, and environmental stability, respectively, and can be constants. x1, x2, x3, and x4 are the normalized values ​​of wind speed, light, rain and snow, and environmental stability, respectively. The fusion weight of the first environmental data is 1 - γ. Determining the fusion weight using environmental parameters can improve the reliability of multi-dimensional perception environmental data, thereby ensuring the flight safety of the drone.

[0056] Step 240: Perform image feature extraction on the first environmental data in the initial environmental fusion data to obtain a first environmental feature, and perform point cloud feature extraction on the second environmental data in the initial environmental fusion data to obtain a second environmental feature.

[0057] Among them, both image feature extraction and point cloud feature extraction can be achieved through image recognition using neural network models.

[0058] Step 250: Perform a multimodal attention mechanism fusion on the first environmental feature and the second environmental feature according to the fusion weight to obtain multidimensional perception environment data.

[0059] When performing multimodal attention fusion, the data dimension can be transformed first. For example, attention matrices are calculated from image to point cloud and from point cloud to image. The first and second environmental features are then fused using the attention matrices according to the fusion weights to obtain multidimensional perceived environmental data.

[0060] Step 260: Plan the flight path of the UAV based on the multi-dimensional perception environment data, the UAV flight mission requirements, and the ship's position.

[0061] Optionally, the flight path of the UAV is planned based on the multi-dimensional perception environment data, the UAV flight mission requirements and the ship position, including: generating the UAV initial route using a heuristic search algorithm based on the UAV flight mission requirements and the ship position; and optimizing the initial route using a dynamic window method based on the real-time multi-dimensional perception environment data to obtain the flight path.

[0062] Among them, the heuristic search algorithm can be A * Algorithm. * When the UAV mission requirements and take-off and landing locations are known, the algorithm can perform direct static path search to obtain the global UAV initial route.

[0063] Dynamic Window Approach (DWA) is a local real-time path planning algorithm. Based on the initial route of the UAV, the DWA algorithm can dynamically adjust the path according to the multi-dimensional perception environment data to obtain a real-time flight path. * The combination of the algorithm and the DWA algorithm can optimize the UAV flight path in real time and enable the UAV to dynamically avoid obstacles.

[0064] On the basis of the above implementation mode, optionally, a dynamic window method is used to optimize the path of the initial route according to real-time multi-dimensional perception environment data to obtain a flight path, including: determining the safety boundary of the UAV and the take-off and landing area of ​​the UAV according to real-time multi-dimensional perception environment data; based on the real-time safety boundary and take-off and landing area, a dynamic window method is used to optimize the path of the initial route to obtain a flight path.

[0065] Environmental data collected by lidar helps determine the drone's safety boundaries. Environmental data collected by cameras typically includes ship deck markings, such as guide lines and markers, which help determine the drone's takeoff and landing area. In this embodiment of the present invention, multi-dimensional environmental data can be used to determine the drone's safety boundaries and takeoff and landing area. This allows the dynamic windowing method to ensure the drone's safety by ensuring it flies within the safety boundaries and lands in the takeoff and landing area.

[0066] Step 270: Collect the relative swing amplitude between the ship and the UAV, and determine the take-off speed and landing speed of the UAV based on the multi-dimensional perception environment data and the relative swing amplitude.

[0067] In an embodiment of the present invention, the take-off speed and landing speed of the UAV may be determined separately.

[0068] Optionally, the take-off speed of the UAV is determined based on the multi-dimensional perception environment data and the relative swing amplitude, including: when it is determined based on the multi-dimensional perception environment data that there is an obstacle within a preset range from the UAV, and / or the relative swing amplitude is greater than a first preset swing amplitude, determining the degree of reduction in the take-off speed of the UAV based on the first distance of the obstacle from the UAV and the relative swing amplitude; adjusting the take-off speed based on the degree of reduction, and adjusting the UAV to the cruising speed when the preset take-off height is reached.

[0069] The first preset swing amplitude can be the swing amplitude when the drone can take off normally, but the take-off speed of the drone needs to be monitored. The first preset swing amplitude and the preset range can be determined through experience or experimental observation.

[0070] For example, the greater the first distance, the smaller the degree of reduction; the smaller the first distance, the greater the degree of reduction. The greater the relative swing amplitude, the greater the degree of reduction; the smaller the relative swing amplitude, the smaller the degree of reduction. When the drone takes off, the impact of the first distance on the drone's takeoff can be given priority. For example, a larger weight is set for the first distance, and a smaller weight is set for the relative swing amplitude, and the degree of reduction in the drone's takeoff speed is determined by combining the first distance and the relative swing amplitude. When the first distance is less than the drone's safe takeoff distance, or the relative swing amplitude is greater than the drone's safe takeoff amplitude, the drone's takeoff speed can be reduced to 0, that is, the drone is hovered and waits for the environment to stabilize to ensure the drone's takeoff safety.

[0071] Optionally, the landing speed of the UAV is determined based on the multi-dimensional perception environment data and the relative swing amplitude, including: when the relative swing amplitude is less than a second preset swing amplitude, determining the second distance between the UAV and the ship based on the multi-dimensional perception environment data; determining the landing acceleration of the UAV based on the current speed of the UAV and the second distance, and adjusting the current speed according to the landing acceleration to control the landing speed of the UAV; when the relative swing amplitude is greater than the second preset swing amplitude, triggering the UAV to hover, and waiting for the relative swing amplitude to be less than the second preset swing amplitude, and returning to the step of determining the second distance between the UAV and the ship based on the multi-dimensional perception environment data when the relative swing amplitude is less than the second preset swing amplitude.

[0072] In this embodiment of the present invention, to ensure a smooth landing of the drone, the drone's velocity at the moment of contact is V. V can be a number equal to or close to 0. For example, V can be a value within the range [-0.1 m / s, 0.1 m / s]. The drone can descend at a set initial landing velocity, and during descent, the landing acceleration can be adjusted in real time based on the second distance and current velocity to ensure that the drone's velocity at the moment of contact is close to 0.

[0073] The second preset swing amplitude can be the upper limit of the swing amplitude required for the drone to safely land on the ship. The second preset swing amplitude can be determined through manual experience or experimental testing. If the relative swing amplitude is greater than the second preset swing amplitude, the drone's landing safety cannot be guaranteed. If the relative swing amplitude is greater than the second preset swing amplitude during landing, the drone can be triggered to hover. The drone can then continue landing operations until the relative swing amplitude is less than the second preset swing amplitude. During landing, the drone's landing acceleration is determined based on the drone's current speed and the second distance, and the current speed is adjusted based on the landing acceleration to control the drone's landing speed.

[0074] By adjusting the landing speed of the drone according to the relative swing amplitude and the distance between the drone and the ship, the safety and stability of the drone landing can be guaranteed, avoiding drone failures caused by unstable ship decks and avoiding drone failures caused by the drone landing on the ship at too high a speed.

[0075] Step 280: Control the UAV to take off from the ship, fly to perform the mission, and land on the ship based on the takeoff speed, flight path, and landing speed.

[0076] Based on the above embodiment, a buffer device may optionally be provided on the ship. Controlling the drone to take off from the ship, perform its mission, and land on the ship based on the takeoff speed, flight path, and landing speed includes controlling the drone to land on the ship's buffer device based on the landing speed. The buffer device provided on the ship can improve the ship's stability on the sea surface, thereby preventing deck shaking caused by external factors such as wind and waves while the ship is sailing at sea, thereby enabling a smooth landing of the drone on the ship.

[0077] The technical solution of the embodiment of the present invention is to respectively collect first and second environmental data of the drone through a camera and a laser radar provided on the drone; synchronize the timestamps of the first and second environmental data and align the coordinate systems to obtain initial environmental fusion data that is consistent in time and space; determine the fusion weight of the first and second environmental data according to the environmental parameters of the drone; extract image features from the first environmental data in the initial environmental fusion data to obtain first environmental features, and extract point cloud features from the second environmental data in the initial environmental fusion data to obtain second environmental features; and fuse the first and second environmental features through a multimodal attention mechanism according to the fusion weight. Obtain multi-dimensional perception environment data; plan the flight path of the UAV according to the multi-dimensional perception environment data, the UAV flight mission requirements and the position of the ship; collect the relative swing amplitude between the ship and the UAV, and determine the take-off speed and landing speed of the UAV based on the multi-dimensional perception environment data and the relative swing amplitude; control the UAV to take off from the ship, fly to perform the mission, and land on the ship according to the take-off speed, flight path, and landing speed, solving the safety and stability problems in the autonomous take-off and landing of the UAV. The safety of the UAV take-off and landing can be guaranteed by fusing the camera and lidar data, and the safety and stability of the UAV take-off and landing can be guaranteed by adjusting the UAV speed through the fused environment data and the relative swing amplitude.

[0078] An application example of the method for taking off and landing a UAV on a ship provided by an embodiment of the present invention may be: during the take-off preparation stage, the take-off and landing area on the ship can be cleaned to ensure that the take-off and landing area is clean and obstacle-free, meeting the take-off and landing requirements of the UAV; and, the various systems of the vertical take-off fixed-wing UAV, such as the power system, navigation system, and communication system, can be checked for operation, and the operating status of each system can be adjusted to the initial operating state; finally, the take-off parameters of the UAV, such as the take-off altitude, initial take-off speed, and take-off heading, can be set.

[0079] During takeoff, the drone is docked at a designated location in the ship's takeoff and landing area. The drone's power system is then activated, causing its rotors to rotate and generate lift. Under the control of the automatic navigation system, the drone ascends vertically until it reaches a safe takeoff altitude. During takeoff, cameras capture real-time images from both sides of the drone, and lidar detects obstacles in front of and behind the drone. The drone's flight path is then planned based on the image and obstacle data captured by the cameras and lidar, respectively, the mission requirements, and the ship's position. Upon reaching takeoff altitude, the drone switches to cruise mode, stopping its rotors and allowing the fixed-wing section to take over, generating lift using aerodynamic principles to achieve high-speed, low-energy, long-distance flight.

[0080] During the landing phase, following the flight path, upon completing the mission, the drone slowly descends toward the ship's take-off and landing area, eventually landing smoothly there. Specifically, during the descent, the ship's automatic take-off and landing guidance system activates, preparing to receive the drone. When the drone approaches the ship, it switches to vertical take-off and landing mode, restarting its rotors. At this point, under the precise control of the drone's automatic navigation system, it slowly descends toward the ship's take-off and landing area. Once the drone touches the ship's deck, the lift generated by the drone's rotors and the ship's cushioning system ensure a smooth landing.

[0081] Example 3

[0082] Figure 3 FIG. 1 is a schematic diagram of a take-off and landing device for a UAV on a ship according to the third embodiment of the present invention. Figure 3 As shown, the device includes: an environmental data acquisition module 310, an environmental data fusion module 320, a flight path planning module 330, a flight speed determination module 340 and a UAV control module 350. Among them:

[0083] The environmental data collection module 310 is used to collect first environmental data and second environmental data of the UAV through a camera and a laser radar provided on the UAV;

[0084] The environmental data fusion module 320 is used to fuse the first environmental data with the second environmental data to obtain multi-dimensional perception environmental data;

[0085] The flight path planning module 330 is used to plan the flight path of the UAV based on the multi-dimensional perception environment data, the UAV flight mission requirements and the ship position;

[0086] The flight speed determination module 340 is used to collect the relative swing amplitude between the ship and the UAV and determine the takeoff speed and landing speed of the UAV based on the multi-dimensional environmental data and the relative swing amplitude;

[0087] The UAV control module 350 is used to control the UAV to take off, fly to perform missions, and land on the ship based on the takeoff speed, flight path, and landing speed.

[0088] Optionally, the environmental data fusion module 320 includes:

[0089] A spatiotemporal synchronization unit, configured to synchronize the timestamps of the first environment data and align the coordinate systems of the second environment data to obtain spatiotemporally consistent initial fusion data of the environment;

[0090] a fusion weight determination unit, configured to determine a fusion weight of the first environmental data and the second environmental data based on environmental parameters of the UAV; wherein the environmental parameters include at least one of the following: wind speed, light, rain and snow, and environmental stability;

[0091] a feature extraction unit, configured to perform image feature extraction on the first environment data in the initial environment fusion data to obtain a first environment feature, and perform point cloud feature extraction on the second environment data in the initial environment fusion data to obtain a second environment feature;

[0092] The environmental data fusion unit is used to perform a multimodal attention mechanism fusion on the first environmental feature and the second environmental feature according to the fusion weight to obtain multidimensional perception environmental data.

[0093] Optionally, the flight path planning module 330 includes:

[0094] The initial route generation unit is used to generate the initial route of the UAV using a heuristic search algorithm according to the UAV flight mission requirements and the ship's position;

[0095] The flight path planning unit is used to optimize the initial route using a dynamic window method based on real-time multi-dimensional perception environment data to obtain a flight path.

[0096] Optional, flight path planning unit, including:

[0097] The area determination subunit is used to determine the safety boundary of the UAV and the take-off and landing area of ​​the UAV based on real-time multi-dimensional perception environment data;

[0098] The flight path planning subunit is used to optimize the initial route using the dynamic window method based on the real-time safety boundary and take-off and landing area to obtain the flight path.

[0099] Optionally, the flight speed determination module 340 includes:

[0100] a reduction degree determining unit, configured to determine a degree of reduction in the takeoff speed of the drone based on a first distance between the obstacle and the drone and the relative swing amplitude when it is determined based on the multi-dimensional sensing environment data that there is an obstacle within a preset range from the drone and / or the relative swing amplitude is greater than a first preset swing amplitude;

[0101] The take-off speed adjustment unit is used to adjust the take-off speed according to the degree of descent and adjust the drone to the cruising speed when the preset take-off altitude is reached.

[0102] Optionally, the flight speed determination module 340 includes:

[0103] a distance determining unit, configured to determine a second distance between the UAV and the ship based on the multi-dimensional sensed environment data when the relative swing amplitude is less than a second preset swing amplitude;

[0104] A landing speed adjustment unit, configured to determine a landing acceleration of the drone based on the current speed of the drone and the second distance, and adjust the current speed based on the landing acceleration to control the landing speed of the drone;

[0105] The hovering control unit is used to trigger the drone to hover when the relative swing amplitude is greater than the second preset swing amplitude, and wait for the relative swing amplitude to be less than the second preset swing amplitude, and return to the step of determining the second distance between the drone and the ship based on the multi-dimensional perception environment data when the relative swing amplitude is less than the second preset swing amplitude.

[0106] Optionally, a buffer device is provided on the ship;

[0107] The drone control module 350 includes:

[0108] The UAV control unit is used to control the UAV to land on the ship's buffer device according to the landing speed.

[0109] The take-off and landing device for a UAV on a ship provided in an embodiment of the present invention can execute the take-off and landing method for a UAV on a ship provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0110] Example 4

[0111] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0112] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0113] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0114] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for taking off and landing a drone on a ship.

[0115] In some embodiments, the method for taking off and landing a drone on a ship can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method for taking off and landing a drone on a ship described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the method for taking off and landing a drone on a ship through any other suitable means (e.g., via firmware).

[0116] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0118] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0120] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0121] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0122] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0123] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for taking off and landing a UAV on a ship, characterized in that: include: The first environment data and the second environment data of the drone are collected respectively by a camera and a laser radar provided on the drone; fusing the first environmental data with the second environmental data to obtain multi-dimensional perceived environmental data; Planning a flight path for the UAV based on the multi-dimensional perceived environmental data, the UAV's flight mission requirements, and the ship's position; collecting the relative swing amplitude between the ship and the UAV, and determining the takeoff speed and landing speed of the UAV based on the multi-dimensional sensed environment data and the relative swing amplitude; According to the take-off speed, the flight path, and the landing speed, the UAV is controlled to take off from the ship, fly to perform a mission, and land on the ship.

2. The method according to claim 1, characterized in that The first environmental data and the second environmental data are integrated to obtain multi-dimensional perception environmental data, including: Performing time stamp synchronization and coordinate system alignment on the first environment data and the second environment data to obtain temporally and spatially consistent initial environment fusion data; Determining a fusion weight of the first environmental data and the second environmental data based on environmental parameters of the UAV; wherein the environmental parameters include at least one of the following: wind speed, light, rain and snow, and environmental stability; Performing image feature extraction on the first environmental data in the initial environmental fusion data to obtain a first environmental feature, and performing point cloud feature extraction on the second environmental data in the initial environmental fusion data to obtain a second environmental feature; The first environmental feature and the second environmental feature are fused using a multimodal attention mechanism according to the fusion weight to obtain multidimensional perceived environmental data.

3. The method according to claim 1, characterized in that Planning the flight path of the UAV based on the multi-dimensional perception environment data, the UAV flight mission requirements, and the ship's position, including: According to the UAV flight mission requirements and the ship's position, a heuristic search algorithm is used to generate the UAV's initial route; According to the real-time multi-dimensional perception environment data, the initial route is optimized using a dynamic window method to obtain the flight path.

4. The method according to claim 3, characterized in that According to the real-time multi-dimensional sensing environment data, the initial route is optimized using a dynamic window method to obtain the flight path, including: Determine the safety boundary of the drone and the take-off and landing area of ​​the drone based on the real-time multi-dimensional perception environment data; According to the real-time safety boundary and take-off and landing area, the dynamic window method is used to optimize the path of the initial route to obtain the flight path.

5. The method according to claim 1, wherein Determining a takeoff speed of the UAV according to the multi-dimensional sensed environment data and the relative swing amplitude includes: When it is determined according to the multi-dimensional environmental perception data that an obstacle exists within a preset range from the drone, and / or the relative swing amplitude is greater than a first preset swing amplitude, determining a degree of reduction in the take-off speed of the drone according to the first distance between the obstacle and the drone and the relative swing amplitude; The take-off speed is adjusted according to the degree of reduction, and when the preset take-off altitude is reached, the UAV is adjusted to a cruising speed.

6. The method according to claim 1, characterized in that Determining a landing speed of the UAV according to the multi-dimensional sensed environment data and the relative swing amplitude includes: When the relative swing amplitude is less than a second preset swing amplitude, determining a second distance between the UAV and the ship according to the multi-dimensional sensed environment data; Determining a landing acceleration of the drone based on the current speed of the drone and the second distance, and adjusting the current speed based on the landing acceleration to control the landing speed of the drone; When the relative swing amplitude is greater than the second preset swing amplitude, the drone is triggered to hover, and waits until the relative swing amplitude is less than the second preset swing amplitude, and returns to the step of determining the second distance between the drone and the ship based on the multi-dimensional perception environment data when the relative swing amplitude is less than the second preset swing amplitude.

7. The method according to claim 1, characterized in that A buffer device is provided on the ship; Controlling the UAV to take off from the ship, fly to perform a mission, and land on the ship according to the takeoff speed, the flight path, and the landing speed, including: According to the landing speed, the UAV is controlled to land on the buffer device of the ship.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for taking off and landing a drone on a ship according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for taking off and landing a UAV on a ship according to any one of claims 1 to 7 when executed.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method for taking off and landing a UAV on a ship according to any one of claims 1 to 7.

Images