An Optical Scene Estimation Method for Fresnel-Assisted Landing Systems Based on Geometric Modeling Perspective Constraints

Abstract

This invention discloses an optical scene estimation method for a Fresnel-assisted landing system based on geometric modeling and perspective constraints. The method includes: acquiring UAV pose data, internal parameters, and Fresnel-assisted landing system image data; correcting the UAV pose data and Fresnel-assisted landing system image data based on the UAV's internal parameters; using the corrected data as input to obtain ray trajectories; selecting results, normalizing and assigning weights to the rays by measuring the relationship between multiple rays; calculating the three-dimensional world coordinates of virtual points; projecting the virtual points onto the image plane according to an imaging model; comparing the virtual point positions with reference point positions in the image to determine the deviation between the UAV and the ideal approach position, thereby generating UAV control commands. This invention employs image ray tracing and geometric modeling, using perspective constraints to process and derive the original optical scene, constructing and estimating the virtual image point positions viewed by the human eye at the same angle, thus optimizing the application of the landing system.

Description

Technical Field

[0001] This invention relates to the field of visual navigation imaging information processing technology, and in particular to an optical scene estimation method for Fresnel-assisted landing systems based on geometric modeling perspective constraints. Background Technology

[0002] When aircraft conduct visual landing operations on a ship, the Fresnel lens system is an indispensable guidance device and a key component in assisting with the manual landing. Its guidance performance is a crucial factor affecting the success rate of manual landings. The Fresnel lens system projects a light layer into the air through an optical lens group. Combined with a light array system and a servo system that compensates for the ship's pitch, roll, and heave, it provides a glide path guide beam for the returning aircraft in various sea conditions, guiding the pilot for a visual landing.

[0003] Compared to manned aircraft, drones now offer advantages such as smaller size, lower cost, ease of use, and lower environmental requirements, leading to a shift in application demand from manned aircraft to drones. However, drones have poor compatibility with the original Fresnel system. This is mainly because the human eye's imaging system produces a virtual image during operation. When the position determination methods and calibration information from manned aircraft are applied to cameras, it is found that the camera cannot directly reproduce this virtual image. Therefore, a landing assistance system model suitable for cameras is needed.

[0004] Existing Fresnel landing aid systems are mainly designed for manned aircraft applications. For UAV landing aid guidance, if visual information is directly analyzed to generate commands, the navigation results will have large errors due to the defects and biases of camera vision compared to human vision. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an optical scene estimation method for Fresnel landing systems based on geometric modeling perspective constraints. Based on UAVs and ground platforms, and using image ray tracing and geometric modeling, the method uses perspective constraints to process and derive the original optical scene, thereby constructing and estimating the virtual image point position seen by the human eye at the same angle, so as to achieve application optimization of the landing system.

[0006] The present invention solves the technical problem by adopting the following technical solution:

[0007] A method for estimating the optical scene of a Fresnel-assisted landing system based on geometric modeling perspective constraints includes the following steps:

[0008] Step S1: Acquire UAV pose data, internal parameters, and Fresnel landing system image data;

[0009] Step S2: Correct the UAV pose data and Fresnel landing system image data based on the UAV internal parameters obtained in step S1.

[0010] Step S3: Using the corrected data obtained in step S2 as input, obtain the ray trajectory based on the geometric modeling perspective constraint image ray detection system;

[0011] Step S4: Select the results from step S3 based on geometric modeling perspective constraints, and normalize and assign weights to the rays by measuring the relationship between multiple rays.

[0012] Step S5: Use ray tracing and minimum circle of confusion to calculate the 3D world coordinates of the virtual point;

[0013] Step S6: Project the virtual points onto the image plane according to the imaging model; compare the position of the virtual points with the position of the reference points in the image to give the deviation relationship between the UAV and the ideal approach position, and then generate UAV control commands.

[0014] Furthermore, step S1, the method for acquiring UAV pose data, internal parameters, and Fresnel landing system image data includes:

[0015] Step S11: Ground-based monitoring stations sample data to obtain UAV pose data;

[0016] Step S12: The UAV captures and collects image data of the Fresnel landing system in real time, records its own pose in real time, and sends it to the main control station.

[0017] In step S13, the UAV sends the internal parameter measurements of its inertial, visual, satellite, and meteorological navigation units to the main control station.

[0018] Further, in step S2, the correction method includes: correcting the UAV pose data based on the UAV's internal parameters, and combining the corrected pose data to perform data correction including image denoising and information enhancement based on the differences in the pose data.

[0019] Further, in step S4, the results of step S3 are selected. The methods for normalizing and weighting the rays by measuring the relationship between multiple rays include:

[0020] Step S41: Select the optimal effective beam of the system based on the maximum variance principle from the obtained ray trajectories;

[0021] Step S42: Determine the confidence level based on the correlation between light rays, calculate the information allocation weight for each beam, and then normalize the weights.

[0022] Furthermore, step S5, the method for calculating the 3D world coordinates of the virtual point using ray tracing and the minimum circle of confusion position, includes:

[0023] Step S51: Perform geometric modeling based on perspective constraints to obtain the intersection point of the reverse tracing of light trajectories;

[0024] Step S52: Calculate the three-dimensional world coordinates of the virtual point using the position of the minimum circle of confusion.

[0025] The optical scene estimation method for Fresnel-assisted descent systems based on geometric modeling perspective constraints disclosed in this invention has the following beneficial effects:

[0026] This invention employs geometric modeling and perspective constraints to address the application of unmanned aerial vehicles (UAVs) in manned aircraft landing assistance systems by processing optical images. Initial image data and pose information are provided by the UAV platform and the ground platform. Compared to existing solutions, this invention addresses the limitation of direct application transfer due to the difference between camera vision and human vision. It proposes using reasoning from camera vision information to obtain all the information available in human vision, thereby achieving landing assistance guidance and enabling highly efficient engineering applications. Attached Figure Description

[0027] Figure 1 This is a flowchart of the method of the present invention;

[0028] Figure 2 A schematic diagram illustrating the algorithm framework for the Fresnel-assisted landing system optical scene estimation method based on geometric modeling perspective constraints, as described in this invention.

[0029] Figure 3 This is a schematic diagram of the Fresnel lens system guiding principle of the present invention;

[0030] Figure 4 This is a diagram showing the relationship between the Fresnel lens indication and aircraft altitude in this invention;

[0031] Figure 5 This is a three-dimensional view of the geometric perspective model constructed in this invention;

[0032] Figure 6 This is a plan view of the geometric perspective model constructed in this invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This invention uses a sea-based landing platform as the ground monitoring component. The ground monitoring component consists of a main control station, a monitoring station, an injection station, and communication and auxiliary systems. The UAV carries an inertial navigation unit, a visual navigation unit, a satellite navigation unit, and an auxiliary navigation unit.

[0035] This invention provides a method for estimating the optical scene of a Fresnel-assisted landing system based on geometric modeling perspective constraints, comprising the following steps:

[0036] Step S1: Acquire UAV pose data, internal parameters, and Fresnel landing system image data;

[0037] Step S2: Correct the UAV pose data and Fresnel landing system image data based on the UAV internal parameters obtained in step S1.

[0038] Step S3: Using the corrected data obtained in step S2 as input, obtain the ray trajectory based on the geometric modeling perspective constraint image ray detection system;

[0039] Step S4: Select the results from step S3 based on geometric modeling perspective constraints, and normalize and assign weights to the rays by measuring the relationship between multiple rays.

[0040] Step S5: Use ray tracing and minimum circle of confusion to calculate the 3D world coordinates of the virtual point;

[0041] Step S6: Project the virtual points onto the image plane according to the imaging model; compare the position of the virtual points with the position of the reference points in the image to give the deviation relationship between the UAV and the ideal approach position, and then generate UAV control commands.

[0042] Furthermore, step S1, the method for acquiring UAV pose data, internal parameters, and Fresnel landing system image data includes:

[0043] Step S11: Ground-based monitoring stations sample data to obtain UAV pose data;

[0044] Step S12: The UAV captures and collects image data of the Fresnel landing system in real time, records its own pose in real time, and sends it to the main control station; the acquisition of Fresnel landing system image data is achieved using a vision method based on a visible light tracing binocular camera.

[0045] In step S13, the UAV sends the internal parameter measurements of its inertial, visual, satellite, and meteorological navigation units to the main control station.

[0046] Further, in step S2, the correction method includes: correcting the UAV pose data based on the UAV's internal parameters, and combining the corrected pose data to perform data correction including image denoising and information enhancement based on the differences in the pose data.

[0047] Further, in step S4, the results of step S3 are selected. The methods for normalizing and weighting the rays by measuring the relationship between multiple rays include:

[0048] Step S41: Select the optimal effective beam of the system based on the maximum variance principle from the obtained ray trajectories;

[0049] Step S42: Determine the confidence level based on the correlation between light rays, calculate the information allocation weight for each beam, and then normalize the weights.

[0050] Furthermore, step S5, the method for calculating the 3D world coordinates of the virtual point using ray tracing and the minimum circle of confusion position, includes:

[0051] Step S51: Perform geometric modeling based on perspective constraints to obtain the intersection point of the reverse tracing of light trajectories;

[0052] Step S52: Calculate the three-dimensional world coordinates of the virtual point using the position of the minimum circle of confusion.

[0053] This invention employs geometric modeling and perspective constraints to address the application of unmanned aerial vehicles (UAVs) in manned aircraft landing assistance systems by processing optical images. Initial image data and pose information are provided by the UAV platform and the ground platform. Compared to existing solutions, this invention addresses the limitation of direct application transfer due to the difference between camera vision and human vision. It proposes using reasoning from camera vision information to obtain all the information available in human vision, thereby achieving landing assistance guidance and enabling highly efficient engineering applications.

[0054] This invention addresses the problem that redesigning the physical systems of unmanned aerial vehicles (UAVs) is too costly and cumbersome. By studying the perspective relationships in imaging to estimate the optical scene of the landing aid system, it enables UAVs and manned aircraft to share a single system, which is of great significance for the consistency, simplicity, and efficiency of applications.

[0055] Example

[0056] like Figure 1 and Figure 2 As shown, this invention proposes an optical scene estimation method for Fresnel-assisted landing systems based on geometric modeling perspective constraints. Based on a UAV and a ground platform, the method includes the following stages: receiving external environmental parameters through sensors on the UAV and ground platform; correcting and processing the input data to form ray trajectories; constructing a geometric perspective constraint model; measuring multi-line relationships and assigning weights based on the model; calculating the three-dimensional coordinates of virtual points using ray tracing and the minimum circle of confusion; projecting the virtual points onto the image position according to the imaging model; comparing the virtual point position with the reference point position in the image to determine the deviation relationship between the UAV and the ideal approach position; generating UAV control commands; and achieving UAV landing.

[0057] This invention addresses the impact of differences between camera imaging and human eye imaging, constructing a geometric model with perspective constraints; forming ray tracing trajectories based on a ray tracing model; selecting effective ray bundles using the maximum variance principle and calculating the weights of each ray bundle using a hybrid evaluation model; estimating virtual image point positions using the minimum circle of confusion algorithm; constructing the relationship between estimated point positions and the original image based on system internal parameters; comparing the virtual image point positions with reference points to obtain the deviation, and generating control commands.

[0058] The optical scene estimation method for Fresnel-assisted landing systems based on geometric modeling perspective constraints of the present invention specifically includes the following steps:

[0059] Step 1: Using the offshore landing platform as the ground monitoring unit, the ground monitoring unit consists of a main control station, a monitoring station, an injection station, and communication and auxiliary systems. The UAV carries an inertial navigation unit, a visual navigation unit, a satellite navigation unit, and an auxiliary navigation unit. The specific steps are as follows:

[0060] 1.1 The ground platform is considered stationary (or moving at a constant speed) and data is sampled by the monitoring station to obtain the UAV pose data z. g ;

[0061] 1.2 The UAV captures and acquires image data P0 of the Fresnel landing system in real time, and records its own pose, camera intrinsic parameter matrix K, and extrinsic parameter matrix in real time. Send to the master station, where R is the rotation matrix describing the representation of the camera coordinate system in the world coordinate system, and t is the translation vector describing the position of the camera center in the world coordinate system;

[0062] 1.3 The UAV sends its internal parameter measurements of inertial, visual, satellite, and meteorological navigation units, SINS0, SVNS0, SGNSS0, and SWR0, to the main control station;

[0063] Step 2: Based on the internal parameters and other data in 1.3, correct the UAV pose data and acquired image information constructed in 1.1 and 1.2 to obtain the corrected image P. y The corrected pose is R,t;

[0064] Step 3: Input the data obtained in Step 2, and obtain the ray trajectory based on the geometric modeling perspective constraint image ray detection system. The specific steps are as follows:

[0065] 3.1 First, the corrected image P... y The Fresnel lamp group image P was obtained through recognition and segmentation. f (M*N) (where M*N is the original image size), convert it from RGB space to HSI space to obtain the intensity of each point. Where R i G i B iImage matrices for red, green, and blue channels are given, and the average intensity of the images is used as a threshold I. In image P... f Within (M*N), select the region with intensity greater than I (m*n, where m < M, n < N) (m*n is the size after selection) as the key point.

[0066] 3.2 Regarding the key points selected in 3.1, taking point A as an example:

[0067] 3.2.1 From the rotation and translation matrices, the relationship between the camera coordinate system and the world coordinate system can be obtained as follows:

[0068]

[0069] Where [X] cam Y cam Z cam [X, Y, Z] represents the coordinates of the actual point object in the camera coordinate system, [X, Y, Z] represents the coordinates of the actual point object in the world coordinate system, R is the rotation matrix describing the representation of the camera coordinate system in the world coordinate system, and t is the translation vector describing the position of the camera center in the world coordinate system.

[0070] Then, the camera coordinate system is transformed into the pixel coordinate system:

[0071]

[0072] in Z is the camera intrinsic parameter matrix, determined by the camera parameters. cam Let [u,v] be the depth, and [u,v] be the pixel coordinates of the actual point object. That is, let a be the pixel coordinate data, A' be the camera plane data, and A be the world coordinate data, then we can have:

[0073]

[0074] Given the camera's unit pixel resolution calibration value K (i.e., one pixel value represents an offset distance K in the camera plane), and the world coordinates A at the camera's origin from the ground station calibration information. gps Then the coordinates of point A corresponding to the camera plane are A':

[0075]

[0076] Based on the above information, calculate the world coordinates of each point and the corresponding coordinates on the camera plane. Connecting points A and A' yields the desired ray trajectory. Construct the following geometric perspective model (e.g.) Figure 5 and Figure 6 As shown, Figure 5 For a 3D graph, the depth difference between A and A'; d represents the parallax of the image captured by the binocular camera. L -u R(d is the camera baseline, f is the camera focal length), u L Let u be the coordinates of the left imaging plane. R The coordinates of the right imaging plane are:

[0077] Figure 5 In the diagram, A and B are actual three-dimensional points, and A' and B' are the projection points of these actual points onto the camera plane. Based on the principle of virtual image imaging, the intersection point O of AA' and BB' is considered the desired virtual point.

[0078] Step 4: Based on the perspective constraints of geometric modeling, select the results from Step 3, and normalize and assign weights to the rays by measuring the relationship between multiple rays. The specific steps are as follows:

[0079] 4.1 Assign confidence levels {w1, w2, ..., w} to the light trajectories in the geometric perspective model of step three. n}

[0080] (Total n items), with confidence levels {w1, w2, ..., w n Based on the correlation of each ray (its ability to intersect with other rays) and the previously calculated light intensity at the corresponding point... A joint decision.

[0081] 4.2 Remove the wire bundles and their weights from 4.1 whose confidence levels are below the preset threshold ε, to obtain a weight subset {w1, w2, ..., w l (l≤n)

[0082] 4.3 The weight subset {w1, w2, ..., w} obtained in 4.2 is used to... l The data is normalized, and to highlight the information of the main light rays, the density of the distribution is rearranged using a normal distribution before normalization. The hyperparameters μ and ∑ of the Gaussian distribution are determined empirically, and new weights {σ1, σ2, ..., σ} are obtained. l}

[0083] Step 5: Use ray tracing and minimum circle of confusion to calculate the 3D coordinates of the virtual point;

[0084] 5.1 Perform energy transformation on the wire harness in step four based on its corresponding weights, I i =I i *σ i The energy level here is selected from the image's intensity I in the HIV space.

[0085] 5.2 Extend the harness in reverse based on ray tracing (e.g.) Figure 3 This yields the intersection point data in three-dimensional space.

[0086] 5.3 Use the root mean square (RMS) method to determine the spatial position of each point at the k smallest point (k = 3 or other selected values) before the diameter of the circle of confusion, and use this position as the image point coordinates {o1, o2, ..., o n}(k≤l), and assign weights to the corresponding points by the corresponding diffusion circle, following the principle that the smaller the diameter, the larger the weight;

[0087] Step 6: Project the virtual point onto the image position according to the imaging model; compare the position of the virtual point with the position of the reference point in the image to give the deviation relationship between the UAV and the ideal approach position, and then generate UAV control commands to control the UAV to land at the current altitude, or to raise, lower, or go around.

[0088] 6.1 The coordinates obtained in step 5 are mapped back to the original image using the UAV camera's intrinsic parameter matrix K and extrinsic parameter matrix P to obtain the corresponding coordinate points;

[0089] 6.2 Calculate the deviation between each coordinate point and the reference point, including the distance {d1, d2, ..., d...} k}, angles {θ1, θ2, ..., θ k Then, the final deviation relationship is obtained by assigning weights in 5.3. Where D is the total distance offset and θ is the total angle offset;

[0090] 6.3 From the deviation relationship, such as Figure 4 Perform control judgments and generate instructions.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for estimating the optical scene of a Fresnel-assisted landing system based on geometric modeling perspective constraints, characterized in that, Includes the following steps: Step S1: Acquire UAV pose data, internal parameters, and Fresnel landing system image data; Step S2: Correct the UAV pose data and Fresnel landing system image data based on the UAV internal parameters obtained in step S1. Step S3: Using the corrected data obtained in step S2 as input, obtain the ray trajectory based on the geometric modeling perspective constraint image ray detection system; Step S4: Select the results from step S3 based on geometric modeling perspective constraints, and normalize and assign weights to the rays by measuring the relationship between multiple rays. Step S5: Use ray tracing and minimum circle of confusion to calculate the 3D world coordinates of the virtual point; Step S6: Project the virtual points onto the image plane according to the imaging model; compare the position of the virtual points with the position of the reference points in the image to give the deviation relationship between the UAV and the ideal approach position, and then generate UAV control commands.

2. The optical scene estimation method for a Fresnel-assisted landing system based on geometric modeling perspective constraints according to claim 1, characterized in that, Step S1, the method for acquiring UAV pose data, internal parameters, and Fresnel landing system image data includes: Step S11: Ground-based monitoring stations sample data to obtain UAV pose data; Step S12: The UAV captures and collects image data of the Fresnel landing system in real time, records its own pose in real time, and sends it to the main control station. In step S13, the UAV sends the internal parameter measurements of its inertial, visual, satellite, and meteorological navigation units to the main control station.

3. The optical scene estimation method for a Fresnel-assisted landing system based on geometric modeling perspective constraints according to claim 2, characterized in that, Step S2, the correction method includes: correcting the UAV pose data based on the UAV's internal parameters, and combining the corrected pose data to perform data correction including image denoising and information enhancement based on the differences in pose data.

4. The optical scene estimation method for a Fresnel-assisted landing system based on geometric modeling perspective constraints according to claim 3, characterized in that, Step S4 involves selecting the results from step S3. The methods for normalizing and weighting the rays by measuring the relationship between multiple rays include: Step S41: Select the optimal effective beam of the system based on the maximum variance principle from the obtained ray trajectories; Step S42: Determine the confidence level based on the correlation between light rays, calculate the information allocation weight for each beam, and then normalize the weights.

5. The optical scene estimation method for a Fresnel-assisted landing system based on geometric modeling perspective constraints according to claim 4, characterized in that, Step S5, the method for calculating the 3D world coordinates of the virtual point using ray tracing and the minimum circle of confusion position includes: Step S51: Perform geometric modeling based on perspective constraints to obtain the intersection point of reverse ray tracing; Step S52: Calculate the three-dimensional world coordinates of the virtual point using the position of the minimum circle of confusion.

Images