A method and system for visual navigation of a rotorcraft unmanned aerial vehicle in an unknown forest region

Abstract

The application discloses a kind of rotor unmanned aerial vehicle visual navigation method and system for unknown forest area, first utilize depth camera in combination with Sobel gradient filtering mechanism, effectively eliminate interference information;Secondly through the cascade model of segmentation convolution network and residual network, realize tree trunk pixel level segmentation and three-dimensional positioning, accurately map visual information to world coordinate system, solve the positioning drift problem caused by similar texture in forest scene;Further based on the minimum jerk trajectory planning of starting point-target point-end point, generate smooth trajectory with polynomial optimization and quadratic programming, fundamentally suppress flight jitter, while reducing mechanical loss, guarantee image acquisition quality;Finally, through the real-time target point update and trajectory dynamic correction of depth camera, form " perception-planning-execution " closed loop, so that unmanned aerial vehicle has dynamic obstacle avoidance and path online optimization ability, finally realizes autonomous crossing, significantly improves the navigation safety and stability under complex environment.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) path planning technology, and in particular relates to a visual navigation method and system for rotary-wing UAVs in unknown forest areas. Background Technology

[0002] In recent years, rotorcraft have become a hot research topic in the field of unmanned aerial vehicles (UAVs) due to their superior mission execution capabilities. With the rapid development of artificial intelligence, computer vision, and control technology, the application scope of rotorcraft has been continuously expanding, covering multiple fields such as search and rescue, environmental monitoring, intelligent inspection, agricultural plant protection, and military reconnaissance. In many application scenarios, rotorcraft demonstrate outstanding performance in complex mission execution due to their excellent hovering performance and agile maneuverability. However, rotorcraft face many challenges when flying at low altitudes, such as limited environmental perception capabilities, high obstacle avoidance accuracy requirements, and susceptibility to navigation stability interference. These factors directly affect their flight safety and mission execution efficiency.

[0003] During mission execution, the autonomous navigation and obstacle avoidance capabilities of rotorcraft are particularly crucial. They not only determine whether the aircraft can successfully reach its target location, but also relate to its ability to operate stably in complex environments, avoid collisions, and complete the mission. Therefore, improving the autonomous navigation capabilities of rotorcraft to ensure their safe and efficient mission execution in unknown or dynamic environments has become one of the core research issues.

[0004] Currently, mainstream autonomous navigation methods for rotorcraft typically break down the problem into multiple sub-tasks, including perception, mapping, and planning. The perception phase mainly relies on various sensors to acquire environmental information, such as lidar, depth cameras, and inertial measurement units (IMUs), to build an understanding of the surrounding environment. In recent years, vision-based navigation methods have attracted considerable attention due to their low cost, lightweight nature, and wide applicability. Vision-based navigation uses only cameras as environmental perception sensors and has achieved remarkable results in both laboratory and real-world environments. Among them, Simultaneous Localization and Mapping (SLAM) technology is one of the most advanced vision-based navigation methods. It utilizes image information acquired by cameras to create an environmental map and performs local path planning within the mapped area, thereby achieving autonomous navigation for rotorcraft. Building on this, researchers have further proposed the Euclidean Signed Distance Field (ESDF) to calculate collision costs and gradients, enabling efficient planning in unknown environments. ESDF can provide a more accurate representation of the environment, allowing the aircraft to optimize its path based on obstacle distance information. However, the construction of ESDF maps requires significant computational resources, greatly impacting the real-time performance of local path planning. This issue has become a major factor limiting the widespread application of visual SLAM-based rotorcraft autonomous navigation systems. To address this challenge, deep learning-based methods have emerged in recent years, utilizing neural networks to directly learn environmental features from visual information and guide rotorcraft flight path planning accordingly. Compared to traditional mapping and planning methods, learning-based methods can achieve superior navigation performance in specific environments and possess stronger adaptability. However, these methods require substantial prior environmental data during training, and the model's generalization ability remains a significant challenge in unknown environments. Furthermore, the high computational resource demands of learning methods mean that improving model reliability while maintaining real-time performance remains a key research focus.

[0005] In summary, autonomous navigation for rotorcraft still faces numerous challenges. Researchers need to continuously optimize algorithms across multiple stages, including perception, mapping, planning, and control, to improve the adaptability and stability of aircraft in complex environments. To further promote the application of UAVs in navigation, traversal, and search in unknown forest areas, there is an urgent need to invent efficient and safe visual navigation methods for UAVs. Summary of the Invention

[0006] To address the above technical problems, this invention provides a visual navigation method and system for rotary-wing unmanned aerial vehicles (UAVs) in unknown forest areas.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] A visual navigation method for rotary-wing unmanned aerial vehicles (UAVs) in unknown forest areas, the method comprising the following steps:

[0009] S100: Uses a drone depth camera to acquire RGB-D images, calculates the image gradient using the Sobel operator, and filters the image using a gradient filter;

[0010] S200: The fused and filtered RGB-D image is input into the segmentation convolutional neural network model to obtain a segmentation map output containing only tree trunks;

[0011] S300: Input the segmentation map into the residual network model to obtain the 3D target points in the camera coordinate system; obtain the coordinate transformation matrix based on the UAV attitude and position, and transform the target points to the world coordinate system;

[0012] S400: Given a starting point, a target point, and an ending point, construct a trajectory planning problem that minimizes jerk.

[0013] S500: During flight, the target point is updated in real time based on images acquired by the depth camera, thereby updating the trajectory until the destination is reached, completing the drone navigation in unknown forest areas.

[0014] Preferably, S100 includes:

[0015] S110: The relative positions of the depth camera and the drone are as follows: Use a depth camera to acquire RGB and depth images at a preset frame rate;

[0016] S120: Align the RGB image and depth image according to the timestamp, use the Sobel operator as an image filter, and calculate the gradient of the depth image in the x-direction:

[0017] ;

[0018] in Indicates depth image in gradient of direction, Represents the original depth image;

[0019] S130: Determine whether the gradient image meets the preset gradient requirements through the gradient filter. If it does not meet the gradient requirements, discard the current image; if it meets the gradient requirements, receive the RGB image and the depth image.

[0020] Preferably, S200 includes:

[0021] S210: Merges depth image data with RGB image data, upscaling 3D RGB data to 4D RGB-D data;

[0022] S220: Input four-dimensional RGB-D data into a segmentation convolutional neural network model, remove branches and leaves from tree images and retain the trunk, reduce the data from four-dimensional RGB-D to single-channel one-dimensional depth segmentation data, and output a depth image containing only tree branches and trunks.

[0023] S230: Perform color mapping transformation on the one-dimensional depth segmentation data to generate an RGB image, thereby achieving tree branch and background segmentation, where the color depth of the branches represents their distance from the camera.

[0024] Preferably, S300 includes:

[0025] S310: Input the segmented image into a pre-trained residual network, and output a pair of safe and feasible target points in the pixel coordinate system. They are located on both sides of the branches;

[0026] S320: Intercept between target points The rectangular region is processed using a depth filter to obtain the average depth of the remaining region. If the average depth of the remaining area is less than the preset viewing distance, the average depth will be used. As the Z coordinate of the camera coordinate system, the pixel plane coordinates are transformed to the camera coordinate system plane coordinates by combining the focal length and optical center deviation.

[0027] S330: Based on the UAV's world coordinate system position With attitude quaternions Calculate the heading angle;

[0028] S340: Based on the UAV's position and attitude, construct the coordinate transformation matrix from the camera coordinate system to the world coordinate system. ;

[0029] S350: Transforms the target point coordinates in the camera coordinate system to the target point coordinates in the world coordinate system using a coordinate transformation matrix. , ;

[0030] S360: Calculate the perpendicular distance from each target point to the line connecting the start and end points, select the safest target point with the closest distance, and finally determine the start point in the world coordinate system. Target point and the endpoint .

[0031] Preferably, S320 specifically comprises:

[0032] ;

[0033] ;

[0034] in These are the pixel coordinates in the pixel coordinate system. The median deviation of the camera. For the camera Focal length, to obtain a pair of target point coordinates in the camera coordinate system. , The average depth, , These are the planar coordinates in the camera coordinate system;

[0035] The S330 is specifically:

[0036] ;

[0037] in, For the drone's heading angle, For attitude quaternions;

[0038] S340 specifically refers to:

[0039] ;

[0040] The S350 is specifically:

[0041] .

[0042] Preferably, S400 includes:

[0043] S410: Flight system receives the starting point in the world coordinate system. Target point and the end point And recorded as the target position matrix. Initialize the initial boundary state based on the target waypoint. With the termination boundary state The boundary conditions include position, velocity, and acceleration boundary conditions;

[0044] S420: Calculate two adjacent waypoints Euclidean distance between Calculate the total distance Based on the preset average speed Calculate total time Distribute the time segments according to their proportions in the total distance, and obtain their respective time segment vectors. Where N is the total number of waypoints;

[0045] S430: Construct a 5th-order time-dependent locus polynomial from the starting point to the target point, and then from the target point to the end point. :

[0046] ;

[0047] in, For polynomial parameters, based on time segments Calculate the time basis function: ;

[0048] S440: To ensure image acquisition quality, a strategy of minimizing jerk is adopted to suppress flight jitter; a polynomial coefficient matrix is ​​constructed. Trajectory mapping matrix and boundary condition matrix ;in ;

[0049] S450: Construct the Hessian matrix that minimizes the jerk. Defined as:

[0050] ;

[0051] in for The third derivative;

[0052] The trajectory problem is transformed into a quadratic programming (QP) problem, which, in its standard form, yields:

[0053] ;

[0054] The trajectory polynomial coefficient matrix is ​​calculated quickly using the QP solver. ;

[0055] S460: Generates a planned position trajectory based on the trajectory polynomial coefficient matrix, discretizes the trajectory to obtain a trajectory position list, and maintains a UAV trajectory transmission stack list. Following the first-in-first-out principle, if a discrete trajectory exists in the stack list, it is popped and sent to the UAV for trajectory tracking.

[0056] Preferably, S500 includes:

[0057] S510: Real-time subscription to the drone's 3D coordinates When the drone reaches the target point of Within the neighborhood, resume receiving RGB-D images;

[0058] S520: After acquiring the image, re-execute S100, S200, S300, and S400. If a new discrete trajectory is generated, update the stack list; otherwise, continue along the original trajectory until the endpoint is reached. .

[0059] A visual navigation system for rotary-wing unmanned aerial vehicles (UAVs) in unknown forest areas includes:

[0060] The image filtering module is used to acquire RGB-D images using a drone depth camera, calculate the image gradient according to the Sobel operator, and filter the image using a gradient filter.

[0061] The image segmentation module is used to fuse the filtered RGB-D images and input them into the segmentation convolutional neural network model to obtain a segmentation map output containing only tree trunks;

[0062] The coordinate system transformation module is used to input the segmentation map into the residual network model to obtain the 3D target points in the camera coordinate system; and to obtain the coordinate transformation matrix based on the UAV attitude and position to transform the target points to the world coordinate system.

[0063] The trajectory planning module is used to construct a trajectory planning problem that minimizes jerk given a start point, target point, and end point.

[0064] The trajectory update module is used to update the target point in real time based on the images acquired by the depth camera during flight, thereby updating the trajectory until the destination is reached, completing the drone navigation in unknown forest areas.

[0065] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a visual navigation method for a rotary-wing unmanned aerial vehicle in an unknown forest area.

[0066] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a visual navigation method for a rotary-wing unmanned aerial vehicle (UAV) in an unknown forest area.

[0067] The aforementioned visual navigation method and system for rotary-wing UAVs in unknown forest areas proposes a visual navigation framework for rotary-wing UAVs in data-driven unknown forests. It designs a front-end perception system and a back-end flight system. The front-end perception system uses RGB-D images to acquire safe and feasible target points, while the back-end flight system plans and tracks the trajectory based on the safe and feasible target points to minimize jerk. This enables UAVs to safely navigate from the starting point to the destination in unknown jungles, providing a technical foundation for UAVs to traverse, search, and explore unknown jungles lacking prior knowledge. Attached Figure Description

[0068] Figure 1 This is a flowchart of a visual navigation method for a rotary-wing unmanned aerial vehicle (UAV) in an unknown forest area according to an embodiment of the present invention;

[0069] Figure 2This is a flowchart illustrating a visual navigation method for a rotary-wing unmanned aerial vehicle (UAV) in an unknown forest area, as described in another embodiment of the present invention.

[0070] Figure 3 This is an environmental map for the simulation experiment of this invention;

[0071] Figure 4 The results are the operation results of the front-end perception system of the present invention, wherein (a) is the RGB image acquired by the depth camera, (b) is the depth image acquired by the depth camera, (c) is the output image segmentation result, and (d) is the output target point recognition result.

[0072] Figure 5 This is a simulation experiment trajectory diagram of the present invention in a simulation scenario;

[0073] Figure 6 For the present invention in Figure 5 The graph shows the change of jerk over time in the two sets of simulation experiments. Detailed Implementation

[0074] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0075] In one embodiment, such as Figure 1 As shown, a visual navigation method for a rotary-wing unmanned aerial vehicle (UAV) in an unknown forest area includes the following steps:

[0076] S100: Uses a drone depth camera to acquire three primary colors-depth (RGB-D) images, calculates the image gradient according to the Sobel operator, and filters the image through a gradient filter;

[0077] S200: The fused and filtered RGB-D image is input into the segmentation convolutional neural network model to obtain a segmentation map output containing only tree trunks;

[0078] S300: Input the segmentation map into the residual network model to obtain the 3D target points in the camera coordinate system; obtain the coordinate transformation matrix based on the UAV attitude and position, and transform the target points to the world coordinate system;

[0079] S400: Given a starting point, a target point, and an ending point, construct a trajectory planning problem that minimizes jerk.

[0080] S500: During flight, the target point is updated in real time based on images acquired by the depth camera, thereby updating the trajectory until the destination is reached, completing the drone navigation in unknown forest areas.

[0081] This invention proposes a visual navigation method and system for rotary-wing UAVs in unknown forest areas, constructing a data-driven visual navigation framework for rotary-wing UAVs in unknown forest environments. For example... Figure 2 As shown, this framework designs a front-end perception system and a back-end flight system: the front-end perception system uses RGB-D images to identify and extract safe and feasible target points; the back-end flight system then plans and tracks a trajectory with minimal jerk based on these points. This effectively enables safe autonomous navigation of UAVs from start to finish in unknown jungle environments, providing a solid technical foundation for UAVs to perform tasks such as traversing, searching, and exploring in unknown jungle areas lacking prior information.

[0082] In one embodiment, S100 includes:

[0083] S110: The relative positions of the depth camera and the drone are as follows: Use a depth camera to acquire RGB and depth images at a preset frame rate;

[0084] S120: Align the RGB image and depth image according to the timestamp, use the Sobel operator as an image filter, and calculate the gradient of the depth image in the x-direction:

[0085] ;

[0086] in Indicates depth image in gradient of direction, This represents the original depth image; furthermore, calculating the gradient of the depth image enables rapid detection of the presence of obstacles such as trees within the line of sight.

[0087] S130: Determine whether the gradient image meets the preset gradient requirements through the gradient filter. If it does not meet the gradient requirements, discard the current image; if it meets the gradient requirements, receive the RGB image and the depth image.

[0088] In one embodiment, S200 includes:

[0089] S210: Merges depth image data with RGB image data, upscaling 3D RGB data to 4D RGB-D data;

[0090] S220: Input four-dimensional RGB-D data into a segmentation convolutional neural network model, remove branches and leaves from tree images and retain the trunk, reduce the data from four-dimensional RGB-D to single-channel one-dimensional depth segmentation data, and output a depth image containing only tree branches and trunks.

[0091] S230: Perform color mapping transformation on the one-dimensional depth segmentation data to generate an RGB image, thereby achieving tree branch and background segmentation, where the color depth of the branches represents their distance from the camera.

[0092] In one embodiment, S300 includes:

[0093] S310: Input the segmented image into a pre-trained residual network, and output a pair of safe and feasible target points in the pixel coordinate system. They are located on both sides of the branches;

[0094] S320: Intercept between target points The rectangular region is processed using a depth filter to obtain the average depth of the remaining region. If the average depth of the remaining area is less than the preset viewing distance, the average depth will be used. As the Z-coordinate of the camera coordinate system, the pixel plane coordinates are transformed to the camera coordinate system plane coordinates by combining the focal length and optical center deviation; further, in this embodiment, the preset viewing distance is 0.9 * viewing distance, if the average depth If the distance is greater than 0.9 * viewing distance, it indicates that the distance to the obstacle tree is too far, and the image processing should be abandoned.

[0095] S330: Based on the UAV's world coordinate system position With attitude quaternions Calculate the heading angle;

[0096] S340: Based on the UAV's position and attitude, construct the coordinate transformation matrix from the camera coordinate system to the world coordinate system. ;

[0097] S350: Transforms the target point coordinates in the camera coordinate system to the target point coordinates in the world coordinate system using a coordinate transformation matrix. , ;

[0098] S360: Calculate the perpendicular distance from each target point to the line connecting the start and end points, select the safest target point with the closest distance, and finally determine the start point in the world coordinate system. Target point and the endpoint .

[0099] In one embodiment, S320 specifically includes:

[0100] ;

[0101] ;

[0102] in These are the pixel coordinates in the pixel coordinate system. The median deviation of the camera. For the camera Focal length, to obtain a pair of target point coordinates in the camera coordinate system. , The average depth, , These are the planar coordinates in the camera coordinate system;

[0103] The S330 is specifically:

[0104] ;

[0105] in, For the drone's heading angle , For attitude quaternions;

[0106] S340 specifically refers to

[0107] ;

[0108] The S350 is specifically:

[0109] .

[0110] In one embodiment, S400 includes:

[0111] S410: Flight system receives the starting point in the world coordinate system. Target point and the end point And recorded as the target position matrix. Initialize the initial boundary state based on the target waypoint. With the termination boundary state The boundary states include position, velocity, and acceleration boundary conditions; the target waypoints include the start point, the target point, and the end point.

[0112] S420: Calculate two adjacent waypoints Euclidean distance between Calculate the total distance Based on the preset average speed Calculate total time Distribute the time segments according to their proportions in the total distance, and obtain their respective time segment vectors. Where N is the total number of waypoints;

[0113] S430: Construct a 5th-order time-dependent locus polynomial from the starting point to the target point, and then from the target point to the end point:

[0114] ;

[0115] in, For polynomial parameters, based on time segments Calculate the time basis function: Based on polynomial coefficients and time basis functions simplify:

[0116] ;

[0117] S440: To ensure image acquisition quality, a strategy of minimizing jerk is adopted to suppress flight jitter; a polynomial coefficient matrix is ​​constructed. Trajectory mapping matrix and boundary condition matrix ;in (Including the starting point) ,end Position, velocity, acceleration constraints and target point (continuity constraints);

[0118] S450: Construct the Hessian matrix that minimizes the jerk. Defined as:

[0119] ;

[0120] in for The third derivative;

[0121] The trajectory problem is transformed into a quadratic programming (QP) problem, which, in its standard form, yields:

[0122] ;

[0123] The trajectory polynomial coefficient matrix is ​​quickly calculated using a quadratic programming (QP) solver. ;

[0124] S460: Generates a planned position trajectory based on the trajectory polynomial coefficient matrix, discretizes the trajectory to obtain a trajectory position list, and maintains a UAV trajectory transmission stack list. Following the first-in-first-out principle, if a discrete trajectory exists in the stack list, it is popped and sent to the UAV for trajectory tracking.

[0125] In one embodiment, S500 includes:

[0126] S510: Real-time subscription to the drone's 3D coordinates When the drone reaches the target point of Within the neighborhood, resume receiving RGB-D images;

[0127] S520: After acquiring the image, re-execute S100, S200, S300, and S400. If a new discrete trajectory is generated, update the stack list; otherwise, continue along the original trajectory until the endpoint is reached. .

[0128] Specifically, such as Figure 3 The image shown is the environment map of the simulation experiment, and the front-end perception results are shown in the image below. Figure 4 As shown, to verify the method proposed in this invention, in Figure 3 The simulation environment shown performs a physical simulation from two different starting points to different ending points. The cylinders represent the locations of trees in the simulation environment, and the trajectory diagram is as follows. Figure 5 As shown, in each set of trajectories, the solid line represents the actual flight trajectory of the rotary-wing UAV after passing the real-time obstacle detection, and the dashed line represents the original flight trajectory before the obstacle was detected. The graphs showing the change in jerk over time during the two sets of simulation experiments are shown below. Figure 6 As shown, the trajectory planning module of this invention plays a role in the flight trajectory.

[0129] The aforementioned visual navigation method for rotary-wing UAVs in unknown forest areas significantly improves navigation safety and stability in complex environments by integrating depth perception and intelligent decision-making. First, by utilizing a depth camera combined with the Sobel gradient filtering mechanism, interference information such as occlusion by branches and leaves is effectively removed, enhancing the robust extraction of tree trunk features. Second, through a cascaded model of segmentation convolutional networks and residual networks, pixel-level segmentation and 3D localization of tree trunks are achieved, accurately mapping visual information to the world coordinate system and solving the localization drift problem caused by similar textures in forest scenes. Furthermore, based on the minimum jerk trajectory planning of the start-target-end point, a smooth trajectory is generated through polynomial optimization and quadratic programming, fundamentally suppressing flight jitter and ensuring image acquisition quality while reducing mechanical wear. Finally, through real-time target point updates and dynamic trajectory correction by the depth camera, a "perception-planning-execution" closed loop is formed, enabling the UAV to have dynamic obstacle avoidance and online path optimization capabilities. Ultimately, it achieves autonomous traversal of dense forest areas in environments without a Global Navigation Satellite System (GNSS), effectively improving navigation success rate and reducing trajectory oscillation amplitude, providing highly reliable navigation support for scenarios such as resource exploration and disaster relief.

[0130] In one embodiment, a visual navigation system for a rotary-wing unmanned aerial vehicle (UAV) in an unknown forest area is also provided, comprising:

[0131] Image filtering module, image segmentation module, coordinate system transformation module, trajectory planning module, and trajectory update module;

[0132] The image filtering module is used to acquire RGB-D images using a drone depth camera, calculate the image gradient according to the Sobel operator, and filter the image using a gradient filter.

[0133] The image segmentation module is used to fuse the filtered RGB-D images and input them into the segmentation convolutional neural network model to obtain a segmentation map output containing only tree trunks;

[0134] The coordinate system transformation module is used to input the segmentation map into the residual network model to obtain the 3D target points in the camera coordinate system; and to obtain the coordinate transformation matrix based on the UAV attitude and position to transform the target points to the world coordinate system.

[0135] The trajectory planning module is used to construct a trajectory planning problem that minimizes jerk given a start point, target point, and end point.

[0136] The trajectory update module is used to update the target point in real time based on the images acquired by the depth camera during flight, thereby updating the trajectory until the destination is reached, completing the drone navigation in unknown forest areas.

[0137] Specific limitations regarding a visual navigation system for a rotary-wing UAV used in unknown forest areas can be found in the above description of a visual navigation method for a rotary-wing UAV used in unknown forest areas, and will not be repeated here. The modules in the aforementioned visual navigation system for a rotary-wing UAV used in unknown forest areas can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0138] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a visual navigation method for a rotary-wing unmanned aerial vehicle in an unknown forest area.

[0139] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a visual navigation method for a rotary-wing unmanned aerial vehicle (UAV) in an unknown forest area.

[0140] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0141] The present invention provides a detailed description of a visual navigation system and method for rotary-wing unmanned aerial vehicles (UAVs) in unknown forest areas. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are merely illustrative of the core ideas of the invention. It should be noted that those skilled in the art can make various improvements and modifications to the invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims.

Claims

1. A visual navigation method for rotary-wing unmanned aerial vehicles (UAVs) in unknown forest areas, characterized in that, The method includes the following steps: S100: Uses a drone depth camera to acquire RGB-D images, calculates the image gradient using the Sobel operator, and filters the image using a gradient filter; S200: The fused and filtered RGB-D image is input into the segmentation convolutional neural network model to obtain a segmentation map output containing only tree trunks; S300: Input the segmentation map into the residual network model to obtain the 3D target points in the camera coordinate system; obtain the coordinate transformation matrix based on the UAV attitude and position, and transform the target points to the world coordinate system; S400: Given a starting point, a target point, and an ending point, construct a trajectory planning problem that minimizes jerk. S500: During flight, the target point is updated in real time based on images acquired by the depth camera, thereby updating the trajectory until the destination is reached, completing the drone navigation in unknown forest areas.

2. The method according to claim 1, characterized in that, S100 includes: S110: The relative positions of the depth camera and the drone are as follows: Use a depth camera to acquire RGB and depth images at a preset frame rate; S120: Align the RGB image and depth image according to the timestamp, use the Sobel operator as an image filter, and calculate the gradient of the depth image in the x-direction: ； in Indicates depth image in gradient of direction, Represents the original depth image; S130: Determine whether the gradient image meets the preset gradient requirements through the gradient filter. If it does not meet the gradient requirements, discard the current image; if it meets the gradient requirements, receive the RGB image and the depth image.

3. The method according to claim 2, characterized in that, S200 includes: S210: Merges depth image data with RGB image data, upscaling 3D RGB data to 4D RGB-D data; S220: Input four-dimensional RGB-D data into a segmentation convolutional neural network model, remove branches and leaves from tree images and retain the trunk, reduce the data from four-dimensional RGB-D to single-channel one-dimensional depth segmentation data, and output a depth image containing only tree branches and trunks. S230: Perform color mapping transformation on the one-dimensional depth segmentation data to generate an RGB image, thereby achieving tree branch and background segmentation, where the color depth of the branches represents their distance from the camera.

4. The method according to claim 3, characterized in that, The S300 includes: S310: Input the segmented image into a pre-trained residual network, and output a pair of safe and feasible target points in the pixel coordinate system. They are located on both sides of the branches; S320: Intercept between target points The rectangular region is processed using a depth filter to obtain the average depth of the remaining region. If the average depth of the remaining area is less than the preset viewing distance, the average depth will be used. As the Z coordinate of the camera coordinate system, the pixel plane coordinates are transformed to the camera coordinate system plane coordinates by combining the focal length and optical center deviation. S330: Based on the UAV's world coordinate system position With attitude quaternions Calculate the heading angle; S340: Based on the UAV's position and attitude, construct the coordinate transformation matrix from the camera coordinate system to the world coordinate system. ; S350: Transforms the target point coordinates in the camera coordinate system to the target point coordinates in the world coordinate system using a coordinate transformation matrix. , ; S360: Calculate the perpendicular distance from each target point to the line connecting the start and end points, select the safest target point with the closest distance, and finally determine the start point in the world coordinate system. Target point and the endpoint .

5. The method according to claim 4, characterized in that, The S320 specifically refers to: ； ； in These are the pixel coordinates in the pixel coordinate system. The median deviation of the camera. For the camera Focal length, to obtain a pair of target point coordinates in the camera coordinate system. , The average depth, , These are the planar coordinates in the camera coordinate system; The S330 is specifically: ； in, For the drone's heading angle, For attitude quaternions; S340 specifically refers to: ； The S350 is specifically: 。 6. The method according to claim 5, characterized in that, The S400 includes: S410: Flight system receives the starting point in the world coordinate system. Target point and the end point And recorded as the target position matrix. Initialize the initial boundary state based on the target waypoint. With the termination boundary state The boundary conditions include position, velocity, and acceleration boundary conditions; S420: Calculate two adjacent waypoints Euclidean distance between Calculate the total distance Based on the preset average speed Calculate total time Distribute the time segments according to their proportions in the total distance, and obtain their respective time segment vectors. Where N is the total number of waypoints; S430: Construct a 5th-order time-dependent locus polynomial from the starting point to the target point, and then from the target point to the end point. : ； Based on time segmentation Calculate the time basis function : Based on polynomial coefficients and time basis functions Simplify the locus polynomial with respect to time: ； in, These are the polynomial coefficients; S440: To ensure image acquisition quality, a strategy of minimizing jerk is adopted to suppress flight jitter; a polynomial coefficient matrix is ​​constructed. Trajectory mapping matrix and boundary condition matrix ;in ; S450: Construct the Hessian matrix that minimizes the jerk. Defined as: ； in for The third derivative; The trajectory problem is transformed into a quadratic programming (QP) problem, which, in its standard form, yields: ； The trajectory polynomial coefficient matrix is ​​calculated quickly using the QP solver. ; S460: Generates a planned position trajectory based on the trajectory polynomial coefficient matrix, discretizes the trajectory to obtain a trajectory position list, and maintains a UAV trajectory transmission stack list. Following the first-in-first-out principle, if a discrete trajectory exists in the stack list, it is popped and sent to the UAV for trajectory tracking.

7. The method according to claim 6, characterized in that, The S500 includes: S510: Real-time subscription to the drone's 3D coordinates When the drone reaches the target point of Within the neighborhood, resume receiving RGB-D images; S520: After acquiring the image, re-execute S100, S200, S300, and S400. If a new discrete trajectory is generated, update the stack list; otherwise, continue along the original trajectory until the endpoint is reached. .

8. A visual navigation system for rotary-wing unmanned aerial vehicles (UAVs) in unknown forest areas, characterized in that, include: The image filtering module is used to acquire RGB-D images using a drone depth camera, calculate the image gradient according to the Sobel operator, and filter the image using a gradient filter. The image segmentation module is used to fuse the filtered RGB-D images and input them into the segmentation convolutional neural network model to obtain a segmentation map output containing only tree trunks; The coordinate system transformation module is used to input the segmentation map into the residual network model to obtain the 3D target points in the camera coordinate system; and to obtain the coordinate transformation matrix based on the UAV attitude and position to transform the target points to the world coordinate system. The trajectory planning module is used to construct a trajectory planning problem that minimizes jerk given a start point, target point, and end point. The trajectory update module is used to update the target point in real time based on the images acquired by the depth camera during flight, thereby updating the trajectory until the destination is reached, completing the drone navigation in unknown forest areas.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images