A structured light scintillation visual navigation method for near-Earth scenes of manned aircraft

Abstract

This invention discloses a structured light scintillation visual navigation method for manned aircraft in near-ground scenarios. The method includes designing a scintillation control mechanism and a timing synchronization mechanism for the structured light emitter, acquiring image data using an image acquisition device, splitting the image streams after verification, and preprocessing the split passive visual and structured light images separately. Finally, flight navigation control is achieved based on the passive visual and structured light images. This solution innovatively introduces an alternating structured light scintillation mechanism, building upon traditional passive vision and continuous structured light assistance. While maintaining high-precision visual positioning capabilities, it significantly improves the real-time obstacle avoidance capability of manned aircraft in complex near-ground scenarios with uneven lighting and weak textures, effectively reducing collision risks and enhancing the robustness and engineering practicality of autonomous navigation for manned aircraft in low-altitude scenarios.

Description

Technical Field

[0001] This invention belongs to the field of aircraft navigation, specifically relating to a structured light scintillation visual navigation method for near-ground scenarios of manned aircraft. Background Technology

[0002] With the rapid development of the low-altitude economy, manned aircraft (such as vertical takeoff and landing aircraft and flying cars) are showing broad application prospects in the low-altitude field. The near-ground flight scenarios for these aircraft are complex and diverse, with typical scenarios including low-altitude commuting in urban canyons, low-altitude cruising in mountainous areas, autonomous landing between building clusters, and emergency landings in unforeseen circumstances. Among these, autonomous positioning and real-time obstacle avoidance, as core functions of autonomous navigation in near-ground environments, provide key technical support for the aircraft to achieve autonomous flight. Currently, commonly used sensing solutions in the industry include LiDAR and visual cameras. Visual solutions, due to their advantages of light weight, low power consumption, cost-effectiveness, and the ability to simultaneously provide texture and depth information, highly meet the payload, endurance, and cost requirements of manned aircraft, making them extremely suitable for low-altitude near-ground flight scenarios.

[0003] However, the lighting and obstacle environment in near-ground scenes is extremely complex: the alternation of daylight / nighttime lighting and shadow / dark areas in urban canyons, as well as the complex lighting conditions such as morning and evening backlighting, can significantly reduce the signal-to-noise ratio of visual images; weakly textured obstacles such as uniform plain walls and smooth metal surfaces can also increase the difficulty of feature extraction, resulting in a large number of holes or noise in the depth map, which may cause collision risks in high reliability scenarios such as near-ground low-altitude flight and precision landing.

[0004] Active structured light technology can enhance depth perception in areas with weak texture to a certain extent, becoming one of the mainstream approaches to address the aforementioned shortcomings. Traditional structured light projection typically operates in a continuous mode, and the stable, unchanging light spots generated in the image interfere with visual feature extraction, affecting positioning performance. If structured light is completely turned off to ensure positioning accuracy, the depth perception advantage of structured light under weak texture and complex lighting conditions will be lost, weakening obstacle avoidance capabilities. Therefore, achieving synergistic optimization of positioning and obstacle avoidance performance under structured light assistance is of great significance for improving the environmental adaptability and flight safety of manned aircraft in complex near-ground scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide a structured light scintillation visual navigation method for near-ground scenarios of manned aircraft. By coordinating the alternating operation of structured light projection and passive vision, this method overcomes the dual defects of traditional continuous structured light interference affecting visual feature matching and the decline in perception capability and obstacle avoidance safety of passive vision in complex lighting and weak texture environments. This provides a technical solution with engineering value for improving the environmental adaptability, near-ground autonomous navigation and safe flight capabilities of manned aircraft.

[0006] The specific technical solution for achieving the objective of this invention is as follows:

[0007] A structured light scintillation visual navigation method for near-Earth scenes of manned aircraft includes the following steps:

[0008] Step 1: Design the scintillation control mechanism and timing synchronization mechanism for the structured light emitter;

[0009] Step 2: Acquire image data using an image acquisition device;

[0010] Step 3: Verify the acquired image data, split the image streams, and preprocess the split passive vision and structured light images separately:

[0011] Step 4: Implement flight navigation control based on passive vision images and structured light images.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0013] This solution innovatively introduces an alternating flashing structured light mechanism on top of traditional passive vision and continuous structured light assistance. By inputting passive vision images with the structured light temporarily turned off into the positioning module, the interference of continuous structured light on visual feature matching is effectively avoided, ensuring visual positioning accuracy. At the same time, the depth images with the structured light projected for a short time are input into the obstacle avoidance module, enhancing the depth perception capability in weak texture and complex lighting environments, and improving the reliability of near-ground obstacle perception.

[0014] Compared to traditional solutions, the visual navigation scheme with periodic structured light projection in this invention can significantly improve the real-time obstacle avoidance capability of manned aircraft in complex near-ground scenes such as uneven lighting and weak textures while maintaining high-precision visual positioning capabilities. This effectively reduces the risk of collisions and enhances the robustness and engineering practicality of autonomous navigation for manned aircraft in low-altitude scenarios.

[0015] The present invention will be further described below with reference to specific embodiments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the principle of the structured light scintillation visual navigation method for near-ground scenarios of manned aircraft in this scheme.

[0017] Figure 2 This is a timing diagram of the processing logic of the structured light scintillation visual navigation method for near-ground scenarios of manned aircraft in this embodiment of the solution. Detailed Implementation

[0018] Example

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0021] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0022] This solution provides a structured light scintillation visual navigation method for near-Earth scenarios of manned aircraft. It is designed from two aspects: a multi-parameter configuration mechanism at the structured light control end and a structured light power verification mechanism at the camera data processing and publishing end. These two mechanisms work together to achieve structured light scintillation control, ensuring the accuracy of depth image data while avoiding interference from structured light on passive visual image data. It also meets the usage requirements of traditional modes (structured light always on or always off mode), ultimately achieving structured light scintillation-based visual navigation. Specifically, it includes the following steps, such as... Figure 1 As shown:

[0023] Step 1: Design the flicker control mechanism and timing synchronization mechanism for the structured light emitter.

[0024] The structured light emitter uses a multi-parameter configuration mechanism to switch between structured light modes and flashing states, and uses a timing synchronization mechanism to achieve hard synchronization between the structured light projector and the image acquisition device.

[0025] First, the switching between structured light mode and flashing state control is achieved based on a multi-parameter configuration mechanism:

[0026] The first parameter, enable, controls the main switch of the structured light transmitter, where 0 represents disabling structured light and 1 represents enabling structured light.

[0027] The alternating switching mode of the structured light emitter is controlled by the second parameter on-off, where 0 represents turning off the alternating switching mode of the structured light and 1 represents enabling the structured light flashing mode.

[0028] The structured light emission power is controlled by the third parameter set_laser_power, where 0 represents emission power of 0, i.e. no emission, and max represents emission using the maximum power.

[0029] The multiple parameters are set in a progressive manner. The setting of the second parameter is only effective when the first parameter, enable, is set to 1, and the setting of the third parameter is only effective when the second parameter, enable, is set to 1.

[0030] The parameter settings are as follows for different modes in this embodiment.

[0031] Structured light flashing modes:

[0032] Structured light off: enable=1, on-off=1, set_laser_power=0;

[0033] Structured lighting: enable=1, on-off=1, set_laser_power=max.

[0034] Traditional mode (structured light is always on or always off):

[0035] Structured light is always off: enable=0, on-off=0, set_laser_power=0;

[0036] Structured light is always on: enable=1, on-off=0, set_laser_power is the native value.

[0037] In some embodiments, the native value of set_laser_power in the conventional mode is set to 200 (mW) based on the structured light emitter rating.

[0038] Secondly, by configuring preset timing parameters for different scenarios and employing a synchronization mechanism between the structured light projector and image acquisition, the image acquisition device can output stable and orderly passive vision and structured light images.

[0039] Based on different flight scenarios of manned aircraft, preset basic timing parameters are used to set the flashing period of the structured light emitter. For example, the flashing period is set to 30Hz in an environment where sunlight and shadow alternate.

[0040] The design employs a timing synchronization mechanism to achieve hard synchronization between the structured light projector and the image acquisition device through dual signals: the first signal is triggered by the image acquisition device, controlling the periodic exposure of the image acquisition device according to a set period; the second signal is triggered by the structured light projector, forming a fixed phase difference with the first signal, controlling the structured light projector to achieve flashing according to the parameter configuration in step 1-1.

[0041] Step 2: Acquire image data using image acquisition equipment:

[0042] That is, the structured light projector can set the structured light flashing mode according to the preset parameters of different scenarios. By configuring the preset timing parameters of different scenarios and using the synchronization mechanism between the structured light projector and the image acquisition, the image acquisition device can acquire image data.

[0043] Furthermore, the structured light mode of the structured light emitter is adaptively adjusted according to the flight environment and state of the manned spacecraft, including:

[0044] When the aircraft is in a safe cruise state, it will use satellite navigation as the primary method and visual navigation as a secondary method, and will reduce the flashing frequency or turn off the structured light flashing mode to save energy consumption of the manned aircraft.

[0045] When the aircraft is in near-ground flight, satellite navigation faces the problem of reduced accuracy due to signal blockage. To balance positioning accuracy and obstacle avoidance reliability, the structured light flashing mode is activated. In environments with increased complexity, timing parameters are adjusted, flashing frequency is increased, or structured light emission power is appropriately increased (not exceeding the maximum value) to improve obstacle perception accuracy and obstacle avoidance safety.

[0046] Step 3: Verify the acquired image data, split the image streams, and preprocess the split passive vision and structured light images separately:

[0047] Step 3-1: For the data acquired by the image acquisition device, set up a structured light power verification mechanism. Based on the verification result, split the acquired image into passive vision image and structured light image, specifically:

[0048] At each moment, the actual value of the structured light power, frame_laser_power, is compared with a preset threshold (e.g., 100 mW):

[0049] If the actual value of the structured light power frame_laser_power at this moment is greater than or equal to the threshold, it means that the structured light emitter is on at this moment and the structured light emitter was off at the previous moment. The acquired image has no structured light pollution, so the image is used as a passive vision image.

[0050] like Figure 2 In this process, images acquired at time t0 are processed and published at time t1, and so on.

[0051] If the actual value of the structured light power at this moment is less than the threshold, it means that the structured light is off at this moment and was on at the previous moment. The acquired image depth data is more accurate, so the image is used as a structured light image.

[0052] like Figure 2 In the process, images acquired at time t1 are processed and published at time t2, and so on.

[0053] Step 3-2: Preprocess the passive vision image and structured light image after splitting, including: For the passive vision image, remove distortion according to camera intrinsic parameters, optionally, perform image denoising, and avoid overly bright or dark images through adaptive histogram equalization to provide the localization module with raw data that makes corner features easier to detect; For the structured light image, generate a depth map through a stereo matching algorithm to provide the necessary obstacle depth information for the obstacle avoidance module's trajectory planning.

[0054] Step 4: Implement flight navigation control based on passive vision images and structured light images:

[0055] Step 4-1: Extract and track visual features from the passive vision image, calculate the local pose of the current state based on the geometric model, and solve for a more accurate global pose of the aircraft through data fusion optimization methods, including:

[0056] Extract and track inter-frame feature points of passive vision images, and remove mismatched items. In this embodiment, after extracting Shi-Tomasi corner points, the pyramid LK optical flow method is used for tracking. Combined with the RANSAC algorithm and the bidirectional optical flow checking mechanism of previous and next frames and different camera frames, the feature points that are mismatched are removed.

[0057] Based on the known historical feature 3D spatial points and the current 2D pixel points obtained from passive vision images, the local pose of the aircraft relative to the reference coordinate system is solved by the PnP algorithm (Perspective n Points) of multi-view geometry.

[0058] Spatial alignment between the vision and inertial sensors is completed, and global pose optimization is performed. For example, in this embodiment, spatial alignment is first completed using extrinsic parameters between the vision and inertial sensors. These extrinsic parameters are the rotation and translation amounts between the origins of the camera coordinate system and the IMU coordinate system, which can be obtained using camera calibration tools such as Kalibr. Then, the global pose is calculated using bundle adjustment on the state variables within a fixed-size sliding window to reduce accumulated errors. A fixed-size sliding window avoids the computational complexity from increasing linearly with the amount of data.

[0059] Step 4-2: Generate a depth image using structured light images. Based on the depth image, generate a global guidance path and optimize local collision-free trajectories. Obtain executable discrete trajectory points through time reallocation. Finally, output executable control commands for the flight controller, including:

[0060] An initial path from the starting point to a given destination is generated using the A* search algorithm, serving as a global guide;

[0061] Based on the initial path, and combined with the real-time obstacle-occupied grid map information obtained from the depth image, local collision detection is performed and a collision-free path is generated.

[0062] Considering the kinematic and dynamic constraints of manned aircraft, we solve an optimization problem that includes collision penalties, trajectory smoothness penalties, and dynamic feasibility penalties. We complete the local trajectory optimization of the collision-free path to obtain a safe, feasible, and smooth trajectory. We then verify whether the trajectory meets the dynamic feasibility requirements. Furthermore, we sample the optimized trajectory at fixed time intervals to obtain discrete executable trajectory points, which are used to generate control commands under the desired velocity and desired acceleration.

[0063] Step 4-3: The flight control module receives real-time attitude data and control commands, calculates control quantities based on the current state of the aircraft, and drives the actuators to complete autonomous navigation flight.

[0064] This solution innovatively introduces an alternating flashing structured light mechanism on the basis of traditional passive vision and continuous structured light assistance. While maintaining high-precision visual positioning capabilities, it can significantly improve the real-time obstacle avoidance capability of manned aircraft in complex near-ground scenes such as uneven lighting and weak textures, effectively reducing the risk of collision and thus enhancing the robustness and engineering practicality of autonomous navigation of manned aircraft in low-altitude scenarios.

[0065] This solution also provides a structured light scintillation visual navigation system for near-Earth scenarios of manned aircraft, including the following modules:

[0066] Image acquisition module: used to design the flicker control mechanism and timing synchronization mechanism of the structured light emitter, and to acquire image data using image acquisition equipment;

[0067] Image splitting module: Used to verify the acquired image data, split the image stream, and preprocess the split passive vision and structured light images respectively.

[0068] Navigation module: Used to achieve flight navigation control based on passive vision images and structured light images.

[0069] This solution also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor performs the following steps when executing the computer program:

[0070] Step 1: Design the scintillation control mechanism and timing synchronization mechanism for the structured light emitter;

[0071] Step 2: Acquire image data using an image acquisition device;

[0072] Step 3: Verify the acquired image data, split the image streams, and preprocess the split passive vision and structured light images separately:

[0073] Step 4: Implement flight navigation control based on passive vision images and structured light images.

[0074] This solution also provides a computer-storable medium on which a computer program is stored, which, when executed by a processor, performs the following steps:

[0075] Step 1: Design the scintillation control mechanism and timing synchronization mechanism for the structured light emitter;

[0076] Step 2: Acquire image data using an image acquisition device;

[0077] Step 3: Verify the acquired image data, split the image streams, and preprocess the split passive vision and structured light images separately:

[0078] Step 4: Implement flight navigation control based on passive vision images and structured light images.

[0079] The embodiments described above are merely one implementation method of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A structured light scintillation visual navigation method for near-Earth scenes of a manned aircraft, characterized in that, Includes the following steps: Step 1: Design the scintillation control mechanism and timing synchronization mechanism for the structured light emitter; Step 2: Acquire image data using an image acquisition device; Step 3: Verify the acquired image data, split the image streams, and preprocess the split passive vision and structured light images separately: Step 4: Implement flight navigation control based on passive vision images and structured light images.

2. The structured light scintillation visual navigation method for near-Earth scenarios of manned aircraft according to claim 1, characterized in that, The flicker control mechanism and timing synchronization mechanism in step 1 are as follows: The structured light emitter uses a multi-parameter configuration mechanism to switch between structured light modes and flashing states, and uses a timing synchronization mechanism to achieve hard synchronization between the structured light projector and the image acquisition device. The multi-parameter configuration mechanism is as follows: The first parameter, enable, controls the main switch of the structured light transmitter, where 0 represents disabling structured light and 1 represents enabling structured light. The alternating switching mode of the structured light emitter is controlled by the second parameter on-off, where 0 represents turning off the alternating switching mode of the structured light and 1 represents enabling the structured light flashing mode. The structured light emission power is controlled by the third parameter set_laser_power, where 0 represents emission power of 0, i.e. no emission, and max represents emission using the maximum power. Multiple parameters are set in a progressive manner. The setting of the second parameter is only effective when the first parameter, enable, is set to 1, and the setting of the third parameter is only effective when the second parameter, enable, is set to 1. The timing synchronization mechanism achieves hard synchronization between the structured light projector and the image acquisition device through dual signals: the first signal is triggered by the image acquisition device and controls the periodic exposure of the image acquisition device according to a set period; the second signal is triggered by the structured light projector and forms a fixed phase difference with the first signal, controlling the structured light projector to flash according to preset parameter configuration.

3. The structured light scintillation visual navigation method for near-Earth scenarios of manned aircraft according to claim 2, characterized in that, Step 2 involves acquiring image data using an image acquisition device, specifically as follows: The structured light projector sets the structured light flicker mode according to the preset parameters of different scenarios. By configuring the preset timing parameters of different scenarios and using the synchronization mechanism between the structured light projector and the image acquisition device, the image acquisition device acquires image data.

4. The structured light scintillation visual navigation method for near-Earth scenarios of manned aircraft according to claim 3, characterized in that, The structured light emitter's structured light scintillation mode is adaptively adjusted according to the manned spacecraft's flight environment and status, including: When the aircraft is in a safe cruise state, it will use satellite navigation as the primary method and visual navigation as a secondary method, and will reduce the flashing frequency or turn off the structured light flashing mode to save energy consumption of the manned aircraft. When an aircraft is flying near the ground, satellite navigation faces the problem of reduced accuracy due to signal blockage. To address this, the structured light flashing mode is activated to balance positioning accuracy and obstacle avoidance reliability. In environments with increased complexity, timing parameters can be adjusted, flashing frequency increased, or structured light emission power can be appropriately increased to improve obstacle perception accuracy and obstacle avoidance safety.

5. The structured light scintillation visual navigation method for near-Earth scenarios of manned aircraft according to claim 1, characterized in that, The split image stream in step 3 is specifically as follows: Step 3-1: For the data acquired by the image acquisition device, set up a structured light power verification mechanism, and according to the verification result, split the acquired image into passive vision image and structured light image; Step 3-2: Perform preprocessing on the split passive vision image and structured light image respectively, including: distortion removal, noise reduction, and brightness adjustment of the passive vision image; Generation of depth images from structured light images.

6. The structured light scintillation visual navigation method for near-Earth scenarios of manned aircraft according to claim 5, characterized in that, The process of splitting the acquired image into passive vision image and structured light image specifically involves: At each moment, the actual value of the structured light power, frame_laser_power, is compared with a preset threshold: If the actual value of the structured light power frame_laser_power at this moment is greater than or equal to the threshold, it means that the structured light emitter is on at this moment and the structured light emitter was off at the previous moment. The acquired image has no structured light pollution, so the image is used as a passive vision image. If the actual value of the structured light power at this moment is less than the threshold, it means that the structured light is off at this moment and was on at the previous moment. The acquired image depth data is more accurate, so the image is used as a structured light image.

7. The structured light scintillation visual navigation method for near-Earth scenarios of manned aircraft according to claim 1, characterized in that, Step 4, which involves implementing flight navigation control based on passive vision and structured light images, specifically includes: Step 4-1: Extract and track visual features from the passive vision image, calculate the local pose of the current state based on the geometric model, and solve for a more accurate global pose of the aircraft through data fusion optimization methods, including: Extract and track inter-frame feature points of passive visual images, and remove mismatches; Based on the known historical feature 3D spatial points and the current 2D pixel points obtained from passive vision images, the local pose of the aircraft relative to the reference coordinate system is solved by the PnP algorithm of multi-view geometry. Complete spatial alignment between the vision and inertial sensors, and perform global pose optimization; Step 4-2: Generate a depth image using structured light images. Based on the depth image, generate a global guidance path and optimize local collision-free trajectories. Obtain executable discrete trajectory points through time reallocation. Finally, output executable control commands for the flight controller, including: An initial path from the starting point to a given destination is generated using the A* search algorithm, serving as a global guide; Based on the initial path, and combined with the real-time obstacle-occupied grid map information obtained from the depth image, local collision detection is performed and a collision-free path is generated. Considering the kinematic and dynamic constraints of manned aircraft, we solve an optimization problem that includes collision penalties, trajectory smoothness penalties, and dynamic feasibility penalties. We complete the local trajectory optimization of the collision-free path to obtain a safe, feasible, and smooth trajectory. We then verify whether the trajectory meets the dynamic feasibility requirements. Furthermore, we sample the optimized trajectory at fixed time intervals to obtain discrete executable trajectory points, which are used to generate control commands under the desired velocity and desired acceleration. Step 4-3: The flight control module receives real-time attitude data and control commands, calculates control quantities based on the current state of the aircraft, and drives the actuators to complete autonomous navigation flight.

8. A structured light scintillation visual navigation system for near-Earth scenarios of a manned aircraft, characterized in that, Includes the following modules: Image acquisition module: used to design the flicker control mechanism and timing synchronization mechanism of the structured light emitter, and to acquire image data using image acquisition equipment; Image splitting module: Used to verify the acquired image data, split the image stream, and preprocess the split passive vision and structured light images respectively. Navigation module: Used to achieve flight navigation control based on passive vision images and structured light images.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

10. A computer-storable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

Images