An unmanned vehicle-based fuselage skin viewpoint generation and detection method and device

Abstract

The application discloses a fuselage skin viewpoint generation and detection method and device based on an unmanned vehicle, and the method comprises the following steps: acquiring a fuselage point cloud model of the airplane; performing cell decomposition on the contour of the fuselage point cloud model according to the fuselage point cloud model, a camera coverage range and a shooting overlap rate requirement; acquiring a normal vector and a center point of an effective cell; calculating initial pose information of a camera according to orthoview and equidistance shooting constraint conditions; generating an optimized target viewpoint according to minimum safety distance constraint conditions and lifting device height constraint conditions; and detecting the viewpoint coverage rate of the target viewpoint according to a view cone elimination strategy and an occlusion elimination strategy. The fuselage surface coverage problem can be well solved by using the unmanned vehicle system to automatically collect data, perform cell decomposition and generate and optimize the viewpoint, so that the efficiency and safety are improved, the coverage degree is ensured, and the data collection quality is ensured.

Description

Technical Field

[0001] This invention relates to the field of mobile robot coverage planning technology, specifically to a method and apparatus for generating and detecting viewpoints on the fuselage skin of an unmanned vehicle. Background Technology

[0002] Damage to aircraft skin (such as impact dents, lightning strikes, and cracks) is a significant hazard to flight safety. To ensure the continued airworthiness of aircraft, regular inspections of the aircraft skin structure are essential. Currently, approximately 90% of aircraft skin damage detection tasks rely primarily on manual visual inspection, supplemented by some handheld inspection equipment. Methods such as walking around the aircraft, using ladders, and climbing ropes to approach the area to be inspected suffer from low efficiency, poor safety, difficulty in ensuring comprehensive coverage, low identification accuracy, and the inability to provide dynamic early warnings.

[0003] With the rapid development of robotics technology, solving the aforementioned challenges has become possible. Firstly, automated assembly line operations can improve efficiency and safety. Secondly, by rationally planning sensor poses, not only can full coverage of the inspection area be achieved, but also strict constraints on sensor data acquisition can be met, improving data quality and ensuring damage detection accuracy. Finally, repetitive operations can create a historical database of various parts of an aircraft, and spatiotemporal dynamic analysis can be used for hazard warning, further improving flight safety. Unmanned Ground Vehicles (UGVs) have strong load-bearing capacity, endurance, and stable movement, making them a key research area in aircraft skin inspection robots. UGVs can utilize lifting devices and gimbals to carry various inspection equipment such as visible light, infrared, and 3D scanners. Based on the viewpoint generated by coverage planning, the movement of the UGV, lifting device, and gimbal can be controlled to change sensor poses, thereby achieving automatic data acquisition of skin within a specified fuselage area. However, aircraft skin inspection UGVs mainly focus on fixed-point inspection of 21 key parts specified in the aircraft maintenance manual, and there is currently no coverage planning for the fuselage, making full-coverage inspection impossible. Furthermore, due to the limited movement of UGVs on the ground, it is necessary to equip them with lifting devices, gimbals, and other equipment to increase the sensor's field of view. However, the movement of UGVs is obstructed by the wings, and there are limitations on the height of the lifting devices and the rotation angle of the gimbal. These factors make it difficult to achieve strictly direct, equidistant shooting of some parts of the fuselage.

[0004] Therefore, those skilled in the art urgently need to find a new technical solution to address the aforementioned problems. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this invention discloses a method and apparatus for generating and detecting viewpoints of the fuselage skin of unmanned vehicles.

[0006] According to a first aspect of the disclosed embodiments of the present invention, a method for generating and detecting viewpoints on the fuselage skin of an unmanned vehicle is provided, the method comprising:

[0007] Based on the pose information of the unmanned vehicle at each preset observation position and the fuselage detection points collected by the lidar on the unmanned vehicle, the fuselage point cloud model of the aircraft is obtained.

[0008] Based on the fuselage point cloud model, the camera coverage of each unmanned vehicle, and the shooting overlap rate requirements, the outline of the fuselage point cloud model projected onto a preset plane is decomposed into cells.

[0009] Determine the number of point clouds contained in each decomposed cell, identify cells with a number of point clouds greater than a preset threshold as valid cells, and obtain the normal vector and center point of the valid cells.

[0010] Based on the normal vector and center point of each valid cell, the initial pose information of the camera for each unmanned vehicle is calculated according to the shooting constraints of frontal view and equidistant distance.

[0011] Based on the minimum safe distance constraint and the height constraint of the lifting device, the initial pose information is optimized to generate an optimized target viewpoint;

[0012] Based on the view frustum culling strategy and the occlusion culling strategy, the range of fuselage skin that the optimized target viewpoint can cover is calculated in order to detect the viewpoint coverage of the target viewpoint.

[0013] Optionally, obtaining the fuselage point cloud model of the aircraft based on the pose information of the unmanned vehicle at each preset observation position and the fuselage detection points collected by the lidar on the unmanned vehicle includes:

[0014] After the unmanned vehicle reaches the preset observation position, the pose information of the unmanned vehicle is acquired;

[0015] The lidar installed on the unmanned vehicle collects the fuselage detection points of the aircraft in the aircraft coordinate system.

[0016] Convert the fuselage detection points in the aircraft coordinate system to fuselage detection points in the world coordinate system to obtain a fuselage point cloud model composed of all fuselage detection points in the world coordinate system.

[0017] Optionally, the step of decomposing the outline of the fuselage point cloud model projected onto a preset plane into cells based on the fuselage point cloud model, the camera coverage area of ​​each unmanned vehicle, and the required shooting overlap rate includes:

[0018] Based on the fuselage point cloud model and the preset fuselage detection range, and using the least squares method, the first contour of the fuselage detection points in the fuselage point cloud model in the xy plane is represented by a three-segment function expression of quadratic curve-straight line-quadratic curve:

[0019]

[0020] in,

[0021] The second contour of the fuselage detection points in the fuselage point cloud model in the xz plane is represented by an arc approximation function: f xz (x,z)=(xx O ) 2 +(zz O ) 2 -r 2 ,z min ≤z≤z max ;

[0022] Based on the camera coverage and shooting overlap requirements of each unmanned vehicle, the start and end points of all rows of the first contour in the xy plane and the start and end points of all rows of the second contour in the xy plane are determined respectively, so as to decompose the contour of the fuselage point cloud model projected on the preset plane into cells.

[0023] Optionally, determining the number of point clouds contained in each decomposed cell, identifying cells with a point cloud number greater than a preset threshold as valid cells, and obtaining the normal vector and center point of the valid cells includes:

[0024] Determine the number of point clouds contained in each decomposed unit;

[0025] If the number of point clouds contained in the cell is greater than or equal to a preset threshold N e The cell is determined as a valid cell U. ij ;

[0026] Calculate the valid cell U ij center point O ij ;

[0027] The effective cell U is calculated using principal component analysis. ij normal vector n ij .

[0028] Optionally, the step of calculating the initial camera pose information of each autonomous vehicle based on the normal vector and center point of each valid cell, according to the shooting constraints of frontal and equidistant views, includes:

[0029]

[0030] Among them, C ij d represents the position of the camera's optical center. ij This represents the distance between the center point of the cell and the optical center of the camera; d0 represents the nominal shooting distance of the camera; f ij f0 indicates the camera's focal length, where f0 represents the nominal camera focal length. This represents the centroid coordinates of the autonomous vehicle. Pan represents the yaw angle of an autonomous vehicle. ij and tilt ij The gimbal rotation angle is represented by atan2(y,x)∈[0,2π], which is the tangent function, and arctan(y / x)∈[π / 2,-π / 2], which is the arctangent function. ij c represents the height of the boom. l Indicates a constant bias, This represents the angle between the y-axis of the aircraft coordinate system and the y-axis of the unmanned vehicle coordinate system. If the y-axis of the aircraft coordinate system is rotated counterclockwise to the y-axis of the unmanned vehicle coordinate system, then... For positive values, 'pan' represents the angle between the projection of the camera coordinate system's x-axis onto the horizontal plane and the gimbal coordinate system's x-axis. If the gimbal coordinate system's x-axis rotates counter-clockwise to the camera coordinate system's x-axis, 'pan' is positive. 'tilt' represents the angle between the camera coordinate system's z-axis and the gimbal coordinate system's xz plane. If the camera coordinate system's z-axis lies above the gimbal coordinate system's xz plane, 'tilt' is positive. min ,pan max [tilt] indicates the variable range of the pan angle. min ,tilt max ] indicates the variable range of the tilt angle.

[0031] Optionally, optimizing the initial pose information based on the minimum safe distance constraint and the height constraint of the lifting device to generate an optimized target viewpoint includes:

[0032] Determine the minimum safe distance d between the unmanned vehicle and the aircraft. s ;

[0033] With each fuselage detection point as the center and the minimum safe distance d as the boundary, s Draw circles with a radius, calculate the outer contour S of all circles, and determine the area outside the outer contour S as the safe zone where the unmanned vehicle can move freely.

[0034] Determine whether the projection of each valid cell onto the horizontal plane is within the safe area;

[0035] If not within the aforementioned safe zone, via C ij =O ij +d ij nij ,(C ij (1),C ij (2))∈S adjusts the position of the camera's optical center, and through f ij =f0d ij / d0 adjusts the camera's focal length;

[0036] Determine whether each viewpoint satisfies f based on the height constraint of the lifting device. min ≤f ij ≤f max ,h min ≤C ij (3)≤h max , where f min f represents the minimum focal length. max h represents the maximum focal length. min h represents the minimum height of the camera's optical center under the adjustment of the lifting device. max This indicates the maximum height of the camera's optical center under the adjustment of the lifting device;

[0037] If the constraints of the lifting device are not met, and when C ij (3) > h max At that time, let C ij (3) = h max ;

[0038] at tilt ij Based on this, sampling is performed in increments of Δtilt, for each tilt. ij The tilt angle calculation ij The angle can capture the skin area, and it determines whether the cell to be detected can be captured, until the tilt point. ij >tilt max Sampling will stop if either of the two conditions is met and the cell to be detected cannot be captured.

[0039] Let Tilt ij This represents the set of tilt angles that can be captured by the cell to be detected, and the calculation is performed. in Let c(S) represent the set of tilt angles ultimately selected. ij ) represents S ij The achievable coverage of the cells to be detected, card(S) ij () represents the cardinality of the set, e s Indicates the maximum permissible shooting tilt angle;

[0040] Obtain the optimized target viewpoint.

[0041] Optionally, the step of calculating the fuselage skin range that the optimized target viewpoint can cover based on the view frustum culling strategy and the occlusion culling strategy, in order to detect the viewpoint coverage of the target viewpoint, includes:

[0042] Projecting points on the fuselage skin onto a pixel coordinate system, if 0 ≤ P can be satisfied simultaneously... imag (1) / P imag (3)≤x p and 0≤P imag (2) / P imag (3)≤y p Then it means P s It can be photographed without obstruction, where x p The horizontal resolution is represented by y. p P represents the vertical resolution. s A P on the fuselage skin s =[p x p y p z ], P imag =K P W C P s , P W C Let K represent the transformation matrix between the camera coordinate system and the aircraft coordinate system, and let K represent the camera's intrinsic parameter matrix.

[0043]

[0044] Determine whether the camera's coverage area includes point I, which is not on the camera body skin, based on the camera's pose information. ij k (k = 1, 2, ..., n) c );

[0045] If it is determined that points not on the fuselage skin are included, then occlusion has occurred and the occluded fuselage area is calculated.

[0046] Through R ij k (l)=C ij +l ij k N ij k , indicating connection to optical center C ij and I ij k The light rays, of which N ij k =D ij k / norm(D ij k ), norm(Dij k ) represents the magnitude of the vector, D ij k =[I ij k (1)-C ij (1),I ij k (2)-C ij (2),I ij k (3)-C ij (3)];

[0047] Calculate the relationship between the light ray and the effective cell U ij The intersection points are fitted using the least squares method based on the point cloud within the valid cells. The complete coordinates of the four vertices are calculated using the fitting results. Each cell is decomposed into two triangles along the diagonal, one in the upper left and one in the lower right. Let V0, V1, and V2 represent the three vertices of the triangles. Then, the points on the triangular plane can be represented as:

[0048] T ij k (u,v)=(1-uv)V0+uV1+vV2,u≥0,v≥0,u+v≤1,

[0049] If the ray intersects the triangular plane, then C ij +l ij k N ij k = (1-uv)V0+uV1+vV2, obtain the matrix form:

[0050]

[0051] Where E1 = V1 - V0, E2 = V2 - V0, T = C ij -V0, if it satisfies {u≥0,v≥0,u+v≤1}, then it means that point T is on the triangular plane. ij k (u,v) is obscured;

[0052] After removing all obscured points, determine the covered and uncovered areas of the fuselage skin and calculate the coverage rate.

[0053] According to a second aspect of the embodiments disclosed in this invention, a viewpoint generation and detection device for fuselage skin of an unmanned vehicle is provided, the device comprising:

[0054] The point cloud model acquisition module acquires the fuselage point cloud model of the aircraft based on the pose information of the unmanned vehicle at each preset observation position and the fuselage detection points collected by the lidar on the unmanned vehicle.

[0055] The cell decomposition module, connected to the point cloud acquisition module, decomposes the outline of the fuselage point cloud model projected onto a preset plane into cells based on the fuselage point cloud model, the camera coverage of each unmanned vehicle, and the shooting overlap rate requirements.

[0056] The effective cell determination module is connected to the cell decomposition module. It determines the number of point clouds contained in each decomposed cell, determines the cells with a number of point clouds greater than a preset threshold as effective cells, and obtains the normal vector and center point of the effective cells.

[0057] The initial pose determination module is connected to the effective cell determination module. Based on the normal vector and center point of each effective cell, it calculates the initial pose information of the camera for each unmanned vehicle according to the shooting constraints of frontal view and equidistant distance.

[0058] The viewpoint optimization module, connected to the initial pose determination module, optimizes the initial pose information based on the minimum safe distance constraint and the height constraint of the lifting device to generate an optimized target viewpoint.

[0059] The coverage detection module, connected to the viewpoint optimization module, calculates the range of fuselage skin that the optimized target viewpoint can cover based on the view frustum culling strategy and the occlusion culling strategy, so as to detect the viewpoint coverage of the target viewpoint.

[0060] Optionally, the point cloud model acquisition module includes:

[0061] The autonomous vehicle pose acquisition unit acquires the pose information of the autonomous vehicle after it reaches the preset observation position.

[0062] The detection point acquisition unit is connected to the unmanned vehicle pose acquisition unit and collects the fuselage detection points of the aircraft in the aircraft coordinate system through the lidar installed on the unmanned vehicle.

[0063] The point cloud model acquisition unit is connected to the detection point acquisition unit. It converts the fuselage detection points in the aircraft coordinate system into fuselage detection points in the world coordinate system, and acquires a fuselage point cloud model composed of all fuselage detection points in the world coordinate system.

[0064] Optionally, the valid cell determination module includes:

[0065] Point cloud quantity acquisition unit, which determines the number of point clouds contained in each decomposed unit;

[0066] The valid cell determination unit is connected to the point cloud quantity acquisition unit. If the number of point clouds contained in the cell is greater than or equal to a preset threshold N, then the valid cell is determined. eThe cell is determined as a valid cell U. ij ;

[0067] The center point determination unit is connected to the valid cell determination unit, and calculates the valid cell U. ij center point O ij ;

[0068] The normal vector determination unit, connected to the center point determination unit, calculates the effective cell U using principal component analysis. ij normal vector n ij .

[0069] In summary, the technical solution disclosed in this invention can bring the following beneficial effects:

[0070] (1) The ability to automatically collect data according to the planned viewpoint through the unmanned vehicle system can improve efficiency, safety, coverage and data quality;

[0071] (2) The cell decomposition method effectively solves the problem of covering the fuselage curved surface based on the aircraft outline, UGV motion and camera characteristics;

[0072] (3) The optimal sensor pose under many practical constraints such as safe working distance, camera focal length, lifting device height and gimbal attitude can be calculated through viewpoint generation and optimization methods.

[0073] Other features and advantages disclosed in this invention will be described in detail in the following detailed description section. Attached Figure Description

[0074] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0075] Figure 1 This is a flowchart illustrating a method for generating and detecting viewpoints on the fuselage skin of an unmanned vehicle, according to an exemplary embodiment.

[0076] Figure 2 It is based on Figure 1 The flowchart shown is a method for obtaining a point cloud model;

[0077] Figure 3 It is based on Figure 1 The flowchart shown is a method for decomposing cells;

[0078] Figure 4 It is based on Figure 3 A schematic diagram illustrating cell decomposition in the z-direction is shown.

[0079] Figure 5 It is based on Figure 1 A flowchart illustrating an effective cell analysis method is shown.

[0080] Figure 6 This is a structural block diagram of a viewpoint generation and detection device for the fuselage skin of an unmanned vehicle, according to an exemplary embodiment.

[0081] Figure 7 It is based on Figure 6 The diagram shown is a structural block diagram of a point cloud model acquisition module.

[0082] Figure 8 It is based on Figure 6 The diagram shows the structure of an effective cell determination module. Detailed Implementation

[0083] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present disclosure.

[0084] Before introducing the viewpoint generation and detection method for the fuselage skin of an unmanned vehicle in the embodiments disclosed in this invention, the various coordinate systems involved in this invention will be introduced first:

[0085] World coordinate system: The coordinate system is established based on the initial pose of the UGV in the virtual simulation environment. The centroid of the UGV is the origin of the coordinate system. The y-axis is located in the symmetry plane of the UGV and is parallel to the horizontal plane. The positive direction is from the rear of the vehicle to the front of the vehicle. The z-axis is perpendicular to the horizontal plane and vertically upward. The x-axis satisfies the right-hand rule.

[0086] Aircraft coordinate system: The vertex of the aircraft nose projected onto the horizontal plane is the origin of the coordinate system. The y-axis is the intersection of the aircraft's plane of symmetry and the horizontal plane, with the positive direction pointing from the nose to the tail. The z-axis is perpendicular to the horizontal plane and points vertically upwards. The x-axis follows the right-hand rule.

[0087] UGV coordinate system: The centroid of the UGV is the origin of the coordinate system. The y-axis is located in the UGV's plane of symmetry and is parallel to the horizontal plane. The positive direction is from the rear of the vehicle to the front. The z-axis is perpendicular to the horizontal plane and points vertically upward. The x-axis satisfies the right-hand rule.

[0088] Gimbal coordinate system: The lens optical center is the origin of the coordinate system. The UGV coordinate system is rotated 90° counterclockwise around its z-axis to obtain x′y′z, and then rotated 90° clockwise around its x′ axis to obtain x′y″z′. x′, y″, and z′ are the x-axis, y-axis, and z-axis of the gimbal coordinate system.

[0089] Camera coordinate system: The optical center of the lens is the origin of the coordinate system, the optical axis of the camera is the z-axis, pointing outwards is the positive direction, the x-axis and y-axis are parallel to the two vertical sides of the image coordinate system respectively, the positive direction of the y-axis is downwards, and the x-axis satisfies the right-hand rule.

[0090] Figure 1 This is a flowchart illustrating a method for generating and detecting viewpoints on the fuselage skin of an unmanned vehicle, according to an exemplary embodiment. Figure 1 As shown, the method includes:

[0091] In step 101, the fuselage point cloud model of the aircraft is obtained based on the pose information of the unmanned vehicle at each preset observation position and the fuselage detection points collected by the lidar on the unmanned vehicle.

[0092] For example, a dataset of points on the aircraft fuselage in a preset coordinate system (a dataset consisting of the coordinate values ​​of each point) constitutes the fuselage point cloud model of the aircraft. The fuselage point cloud model is the basis for generating viewpoints in subsequent steps 102-106. In the disclosed embodiment of the present invention, fuselage detection points are determined and collected by a lidar installed on an unmanned vehicle to obtain the fuselage point cloud model.

[0093] It should be noted that aircraft models are typically CAD or SolidWorks-based design models. Converting an aircraft model to an STL model yields some 3D points and their coordinates on the aircraft skin surface. However, these 3D points are mainly concentrated on aircraft components and textures, resulting in an uneven distribution that is difficult to use for viewpoint generation. Therefore, the embodiments disclosed in this invention employ virtual simulation to obtain a uniformly distributed aircraft point cloud. Specific steps include: establishing a virtual simulation environment comprising an aircraft model and an UGV (Unmanned Ground Vehicle) equipped with a LiDAR. This simulation environment is built based on Gazebo and includes the aircraft model to be inspected and the UGV equipped with a multi-line LiDAR. Then, the UGV is controlled to move to multiple preset positions within the simulation environment and its onboard LiDAR scans the aircraft model to obtain multiple sets of point clouds. Finally, these multiple point clouds are fused to obtain a complete aircraft point cloud model.

[0094] Specifically, Figure 2 It is based on Figure 1 A flowchart of a point cloud model acquisition method is shown, as follows: Figure 2 As shown, step 101 includes:

[0095] In step 1011, after the unmanned vehicle reaches the preset observation position, the pose information of the unmanned vehicle is acquired.

[0096] For example, each UGV is equipped with multiple lidars at different heights. After determining multiple different unmanned vehicle observation points in the area around the aircraft (each unmanned vehicle observation point corresponds one-to-one with the unmanned vehicle), the chassis drives the UGV to move to the designated unmanned vehicle observation point, and records the pose information of the UGV and the point cloud information (fuselage detection points) collected by each lidar installed on the unmanned vehicle. Then, based on the pose information of each lidar, all the point clouds collected by each lidar are transformed to the same coordinate system, thereby obtaining a complete aircraft point cloud model.

[0097] In step 1012, the lidar installed on the unmanned vehicle collects the fuselage detection points of the aircraft in the aircraft coordinate system.

[0098] For example, let m devices with heights of h be mounted on the UGV simultaneously. i A lidar array (i = 1, 2, ..., m) is used, and n unmanned vehicle (UGV) observation points are selected around the aircraft. After the UGV moves to the designated observation point via chassis drive, the pose of the UGV and the data collected by each lidar are recorded. Then, the quaternions and position information provided by the Gazebo environment are used to transform all aircraft point clouds into the world coordinate system.

[0099] Specifically, let Q ij ∈R 4×1 Let P represent the quaternion of the i-th lidar at the j-th observation point. ij ∈R 1×3 Let C represent the position of the i-th lidar at the j-th observation point, and C... ij b ∈R 1×3 This represents the b-th (b = 1, 2, ..., k) obtained by the i-th lidar at the j-th observation point. ij The fuselage point cloud model in the aircraft coordinate system (i.e., fuselage detection points in the aircraft coordinate system) obtained by fusing the above quaternions, positions, and points can be represented as:

[0100] S w ={C ij b T ij +P ij +H i ,i=1,2,…,m,j=1,2,…,n,b=1,2,…,k ij},

[0101] Among them, T ij =quat2dcm(Q ij )∈R 3×3 H represents a rotation matrix calculated based on quaternions. i =[0 0 hi ].

[0102] In addition, to facilitate subsequent viewpoint generation calculations, the fuselage point cloud model is represented in the aircraft coordinate system.

[0103] In step 1013, the fuselage detection points in the aircraft coordinate system are converted into fuselage detection points in the world coordinate system to obtain a fuselage point cloud model composed of all fuselage detection points in the world coordinate system.

[0104] For example, let P W W The transformation matrix between the aircraft coordinate system and the world coordinate system is represented in the following form:

[0105]

[0106] in, φ represents the coordinates of the origin in the aircraft coordinate system in the world coordinate system, and φ represents the y-axis of the aircraft coordinate system obtained by rotating the world coordinate system counterclockwise around its y-axis by φ.

[0107] Let P W ∈R 3×N Indicates by S w The matrix formed by all elements in P is then P p = P W W P W This represents the point cloud coordinates in the aircraft coordinate system.

[0108] In step 102, based on the fuselage point cloud model, the camera coverage of each unmanned vehicle, and the shooting overlap rate requirements, the outline of the fuselage point cloud model projected onto a preset plane is decomposed into cells.

[0109] For example, firstly, the contours of the camera's point cloud in the top-view and front-view planes are extracted, and then a function is fitted to these contours. Depending on the camera's single-image coverage (i.e., camera coverage area) and the required overlap rate, the user-specified detection area is typically also considered. The camera coordinate system is decomposed along the vertical axis to obtain the start and end points of all rows covering the specified vertical height range. Simultaneously, the same method is used to decompose along the vertical axis to obtain the start and end points of all columns covering the specified vertical length range. Based on the start and end points of all rows and columns, a set of cells that can cover the specified camera area with a certain overlap rate is obtained (i.e., the cell decomposition process).

[0110] Specifically, Figure 3 It is based on Figure 1 A flowchart of a cell decomposition method is shown, such as Figure 3 As shown, step 102 includes:

[0111] In step 1021, based on the fuselage point cloud model and the preset fuselage detection range, and using the least squares method, the first contour of the fuselage detection points in the fuselage point cloud model in the xy plane is represented by a three-segment function expression of quadratic curve-straight line-quadratic curve as follows:

[0112]

[0113] in,

[0114] Specifically, the fuselage detection range can be represented as {(y,z)|0≤y≤y max ,z min ≤z≤z max As can be seen, the range of y and z coordinate values ​​constitutes the constraint condition for the first contour of the fuselage detection point in the xy plane. This first contour is approximated by a three-segment function expression of quadratic curve-straight line-quadratic curve, and the function parameters in the above expression are calculated based on the least squares method.

[0115] In step 1022, the second contour of the fuselage detection points in the fuselage point cloud model in the xz plane is represented by the circular arc approximation function.

[0116] f xz (x,z)=(xx O ) 2 +(zz O ) 2 -r 2 ,z min ≤z≤z max .

[0117] For example, in the disclosed embodiment of the present invention, the second profile of the xz plane is approximated by an arc curve, and the arc parameters are calculated by obtaining the cylindrical fitting of the fuselage using the M-estimator Sample Consensus (MSAC) method.

[0118] In step 1023, based on the camera coverage and shooting overlap requirements of each unmanned vehicle, the start and end points of all rows of the first contour in the xy plane and the start and end points of all rows of the second contour in the xz plane are determined respectively, so as to decompose the contour of the fuselage point cloud model projected on the preset plane into cells.

[0119] For example, after obtaining the first contour of the fuselage point cloud model in the xy plane and the second contour in the xz plane through steps 1021-1022 above, the contour of the fuselage point cloud model projected onto the preset plane can be decomposed into cells based on the first and second contours. During the decomposition process, it is also necessary to obtain the start and end points of all rows of the first and second contours.

[0120] Specifically, the first step is to perform a decomposition in the z-direction, as follows: Figure 4 As shown, let (x i ,z i ) and (x i ′,z i ′) represent the start and end points of the i-th row, respectively. Furthermore, The following formula can be used to recursively solve for the start and end points of all rows in the z-direction:

[0121]

[0122] Among them, L w σ represents the width covered by a single image. w The first row of equations represents the shooting overlap rate in the width direction. The first row of equations indicates that the distance between the start and end points is equal to the width covered by a single image, and the second row of equations indicates that the shooting overlap rate requirement is met.

[0123] For the same reason, let (x j ,y j ) and (x j * ,y j * Let (x1, y1) represent the start and end points of the j-th column, respectively. Let (x1, y1) = (0, 0). The following formula is used to recursively calculate the start and end points of all columns covering the x-direction:

[0124]

[0125] Among them, L l σ represents the length covered by a single image. l The first row of equations represents the shooting overlap rate along the length direction. The first row of equations indicates that the distance between the start and end points is equal to the length covered by a single image. The second row of equations indicates that the shooting overlap rate requirement is met.

[0126] It is understandable that, through the above method, the cell U in the i-th row and j-th column can be obtained. ij The (y, z) coordinates of the four vertices are (y j ,z i ), (y j+1 ,z i ), (y j ,zi+1 ) and (y j+1 ,z i+1 This allows us to obtain the coordinates of the four vertices of each cell after cell decomposition.

[0127] In step 103, the number of point clouds contained in each decomposed cell is determined, and cells with a number of point clouds greater than or equal to a preset number threshold are determined as valid cells, and the normal vector and center point of the valid cell are obtained.

[0128] For example, after cell decomposition, the number of point clouds (number of fuselage detection points) in each cell is searched and recorded. Cells with fewer point clouds than a preset threshold are identified as invalid cells. Invalid cells are filtered out, valid cells are identified, and the normal vector and center point of the valid cells are calculated.

[0129] Specifically, Figure 5 It is based on Figure 1 A flowchart of an effective cell analysis method is shown, such as Figure 5 As shown, step 103 includes:

[0130] In step 1031, the number of point clouds contained in each decomposed unit is determined.

[0131] In step 1032, if the number of point clouds contained in the cell is greater than a preset threshold N... e Define this cell as a valid cell U ij .

[0132] In step 1033, the valid cell U is calculated. ij center point O ij .

[0133] In step 1034, the effective cell U is calculated using principal component analysis. ij normal vector n ij .

[0134] For example, in the disclosed embodiments of the present invention, for valid cell U ij Calculate the valid cell U ij center point O ij And the effective cell U is calculated using principal component analysis. ij normal vector n ij .

[0135] In step 104, the initial pose information of the camera for each autonomous vehicle is calculated based on the normal vector and center point of each valid cell, according to the shooting constraints of frontal view and equidistant distance.

[0136] For example, determine the valid cell U.ij normal vector n ij and center point O ij Then, based on the normal vector n ij and center point O ij The generated viewpoints (i.e., the initial pose information of the cameras for each unmanned vehicle) satisfy the shooting constraints of frontal view and equidistant view. This can improve the image quality acquired when shooting the aircraft body based on the above viewpoints and is beneficial for the 3D reconstruction of the aircraft model based on the images.

[0137] Specifically, the initial pose information of the camera includes the position C of the camera's optical center. ij The distance d between the center point of the cell and the optical center of the camera ij Yaw angle of autonomous vehicles The height of the lifting boom h ij The angle pan between the projection of the camera coordinate system's x-axis onto the horizontal plane and the gimbal coordinate system's x-axis. z The angle tilt between the axis and the xz plane of the gimbal coordinate system.

[0138] Based on the normal vector and center point of each valid cell, the initial pose information of the camera for each autonomous vehicle, calculated according to the shooting constraints of frontal and equidistant views, can be expressed as:

[0139]

[0140] Among them, C ij d represents the position of the camera's optical center. ij This represents the distance between the center point of the cell and the optical center of the camera; d0 represents the nominal shooting distance of the camera; f ij f0 indicates the camera's focal length, where f0 represents the nominal camera focal length. This represents the centroid coordinates of the autonomous vehicle. Pan represents the yaw angle of an autonomous vehicle. ij and tilt ij The gimbal rotation angle is represented by atan2(y,x)∈[0,2π], which is the tangent function, and arctan(y / x)∈[π / 2,-π / 2], which is the arctangent function. ij c represents the height of the boom. l Indicates a constant bias, This represents the angle between the y-axis of the aircraft coordinate system and the y-axis of the unmanned vehicle coordinate system. Specifically, if the y-axis of the aircraft coordinate system is rotated counterclockwise to the y-axis of the unmanned vehicle coordinate system... For positive values, 'pan' represents the angle between the projection of the camera coordinate system's x-axis onto the horizontal plane and the gimbal coordinate system's x-axis. 'pan' is positive if the gimbal coordinate system's x-axis rotates counterclockwise to the camera coordinate system's x-axis. 'tilt' represents the angle between the camera coordinate system's z-axis and the gimbal coordinate system's xz plane. 'tilt' is positive if the camera coordinate system's z-axis lies on the gimbal coordinate system's xz plane. min ,pan max ] and [tilt min ,tilt max ] represent the variable range of the two angles respectively.

[0141] In step 105, the initial pose information is optimized based on the minimum safe distance constraint and the height constraint of the lifting device to generate an optimized target viewpoint.

[0142] For example, considering constraints such as the safety zone, camera focal length, elevator height, and gimbal attitude, the viewpoint generated above needs to be adjusted. Specifically, the viewpoint is first adjusted using the safety zone constraint. The aircraft's inner contour is calculated using the alphaShape method, based on the projected coordinates of the aircraft point cloud on the horizontal plane.

[0143] Determine the minimum safe distance d between the unmanned vehicle and the aircraft. s Taking each fuselage detection point as the center and the minimum safe distance d as the boundary, s Draw circles with a radius of 1, calculate the outer contour S of all circles, and define the area outside S as the safe zone where the autonomous vehicle can move freely; determine whether the projection of each valid cell onto the horizontal plane falls within this safe zone; if it does not fall within the safe zone, use C... ij =O ij +d ij n ij (C) ij (1),C ij (2))∈S adjusts the position of the camera's optical center, and through f ij =f0d ij / d0 adjusts the camera's focal length; determines whether each viewpoint satisfies f based on the height constraints of the lifting device. min ≤f ij ≤f max ,h min ≤C ij (3)≤h max , where f min f represents the minimum focal length. max h represents the maximum focal length. min h represents the minimum height of the camera's optical center under the adjustment of the lifting device. maxThis indicates the maximum height of the camera's optical center under the adjustment of the lifting device; if this is not met, a new viewpoint needs to be selected. Currently, PTZ cameras have strong zoom capabilities, and zoom constraints are usually met. However, due to factors such as manufacturing difficulty, safety, and stability, the lifting device cannot be too high or too low. Therefore, viewpoints corresponding to the higher and lower parts of the fuselage and the upper part of the wing generally do not meet the height constraints of the lifting device.

[0144] When C ij (3) > h max At that time, let C ij (3) = h max Simultaneously increase the tilt angle to capture the cell to be detected. At tilt... ij Based on this, sampling is performed in increments of Δtilt. For each tilt angle, the skin area that can be captured is calculated, and it is determined whether the cell to be detected can be captured. Specifically, at tilt... ij Based on this, sampling is performed in increments of Δtilt, for each tilt. ij Calculate the tilt angle ij The angle can capture the skin area, and it determines whether the cell to be detected can be captured, until the tilt point. ij tilt ij >tilt max And sampling will stop if no cell in the test area meets any of the conditions; it should be noted that, to reduce computation, when tilt... ij >tilt max Sampling may stop when the cell to be detected changes from being able to be captured to being unable to be captured.

[0145] Let Tilt ij This represents the set of tilt angles that can be captured by the cell to be detected, and the calculation is performed. in Let c(S) represent the set of tilt angles ultimately selected. ij ) represents S ij The achievable coverage of the cells to be detected, card(S) ij () represents the cardinality of the set, e s This indicates the maximum permissible shooting tilt angle; similarly, when C ij (3) < h min At that time, let C ij (3) = h min At the same time, the tilt angle is reduced to capture the cell to be detected. In tilt... ij Based on this, sampling is performed in increments of -Δtilt. For each tilt angle, the skin area that can be captured is calculated, and it is determined whether the cell to be detected can be captured. To reduce computational load, when tilt...ij <tilt min Sampling stops when the cell to be detected changes from being able to be captured to being unable to be captured. Then, an optimized tilt angle set is obtained by solving the minimum set coverage problem. The optimized target viewpoint is then obtained.

[0146] In step 106, the range of fuselage skin that the optimized target viewpoint can cover is calculated based on the viewpoint frustum culling strategy and the occlusion culling strategy, so as to detect the viewpoint coverage of the target viewpoint.

[0147] First, perform cone culling. For each viewpoint, calculate the skinned area that the camera can capture at that viewpoint (if it has already been calculated in step four, it does not need to be calculated again). Let... P W UGV The transformation matrix between the UGV coordinate system and the aircraft coordinate system is shown below:

[0148]

[0149] Where h UGV This indicates the height of the UGV's center of mass. Let... UGV W C The transformation matrix between the camera coordinate system and the UGV coordinate system is:

[0150]

[0151] in UGV T C =[0 0 C ij (3)-h UGV ] T , UGV R C = UGV R PTU PTU R C This represents the rotation matrix between the camera coordinate system and the UGV coordinate system, where UGV R PTU and PTU R C Represent the rotation matrices between the gimbal coordinate system and the UGV coordinate system, and between the camera coordinate system and the gimbal coordinate system, respectively, in the form of:

[0152] R C =R y (pan ij )R x (tilt ij )

[0153] in

[0154]

[0155]

[0156] This yields the transformation matrix between the camera coordinate system and the aircraft coordinate system:

[0157] P W C = P W UGV UGV W C

[0158] Let K denote the intrinsic parameter matrix of the camera:

[0159]

[0160] Where dx and dy represent the horizontal and vertical pixel sizes, respectively, and x p and y p Let P represent the horizontal and vertical resolutions, respectively. Then, the following formula can be used to determine the resolution of a point P on the fuselage skin. s =[p x p y p z Projected onto the pixel coordinate system:

[0161] P imag =K P W C P s

[0162] If 0 ≤ P is satisfied at the same time imag (1) / P imag (3)≤x p and 0≤P imag (2) / P imag (3)≤y p Then P s It can be photographed when there is no obstruction.

[0163] Then, occlusion removal is performed. Due to the relatively regular fuselage structure, occlusion during camera shooting is mainly caused by the wings and engines. For a given camera pose, when its coverage points include non-fleet points I... ij k (k = 1, 2, ..., n) c If this occurs, obstruction may have occurred, and the area of ​​the obstructed fuselage needs to be calculated. Connect the optical center C... ij and I ij k The light rays are represented as follows:

[0164] R ij k (l)=C ij +lij k N ij k

[0165] Where N ij k =D ij k / norm(D ij k ), norm(D ij k ) represents the magnitude of the vector, D ij k =[I ij k (1)-C ij (1),I ij k (2)-C ij (2),I ij k (3)-C ij (3)]. The possible intersection points of this ray with the cells on the fuselage are calculated below. First, a plane fitting is performed using the least squares method based on the point cloud within the cell, and the complete coordinates of the four vertices are calculated using the fitting results. Then, each cell is decomposed along the diagonal into two triangles, one in the upper left and one in the lower right. Let V0, V1, and V2 represent the three vertices, then the points on the triangular plane can be represented as:

[0166] T ij k (u,v)=(1-uv)V0+uV1+vV2,u≥0,v≥0,u+v≤1

[0167] If the light ray intersects the triangular plane, then there must be...

[0168] C ij +l ij k N ij k = (1-uv)V0+uV1+vV2

[0169] After sorting, we can obtain...

[0170]

[0171] Where E1 = V1 - V0, E2 = V2 - V0, T = C ij -V0. If {u≥0,v≥0,u+v≤1} is satisfied, then point T on the triangular plane is... ij k (u,v) is occluded. Remove all occluded points, count the covered and uncovered areas, and calculate the coverage rate.

[0172] Furthermore, the accuracy of the viewpoint generation and detection method for the fuselage skin of an unmanned vehicle in the disclosed embodiments of this invention can be further illustrated by the following experimental results:

[0173] In this experiment, the aircraft to be tested was a Boeing 737-300. The UGV was equipped with three Velodyne 16-line lidar sensors at heights of 0.6m, 1m, and 2m. Radar data was collected at nine observation points around the aircraft, resulting in 138,936 points after point cloud fusion and preprocessing. Camera selection was primarily determined by pixel accuracy, zoom range, and single-image coverage. Damage identification accuracy was higher than 1mm at the nominal shooting distance. Zooming to photograph the fuselage from the wingtip did not affect damage identification accuracy, and the horizontal dimension of a single image was no less than 1m at this accuracy. The Hikvision iDS-2DE7823IX-A(T5) PTZ camera met these requirements. The camera and related shooting parameters are as follows:

[0174] pan min =0°,pan max =360°,tilt min = -15°,tilt max =90°,

[0175] f min =5.9mm,f max =135.7mm,x p =3840,y p =2160,

[0176] L l =1m,L w =y p / x p L l m,e s =60°, d0=2m, f0=14.22mm,

[0177] σ l =0.3,σ w =0.3,h min =2.6m,h max =5.8m,d s =1.5m

[0178] Other parameters of the algorithm are

[0179] N e =50,z min =1m,z max =3.4m,y min =0m,y m1 =7m,

[0180] y m2 =19.5m,y max =24m,y c =1.6m,x O =0m,z O =2.32m, r=1.57m,

[0181] cf1=-0.04, cf2=0.46, cf3=0.11,

[0182] c l1 =-0.38,c l2 =1.63,c l3 = -0.06, Δtilt = 1°

[0183] The camera lens optical center height, pan, tilt angle, focal length, shooting tilt angle, and UGV yaw angle planned using the above algorithm are all within the constraints. Although the height limitation of the lifting device prevents some cells from being covered with a single tilt angle, at most two tilt angles are needed to achieve coverage. Based on the viewpoint generated by this invention, a trajectory planning method can be further used to obtain the spatiotemporal trajectory of each device state that achieves the shortest detection time. Regarding the aircraft fuselage skin coverage, shooting according to the planned viewpoint can cover 98.1% of the designated area, proving that the proposed algorithm can achieve high coverage. The remaining uncovered area is concentrated at the bottom of the tail, mainly due to the minimum height limitation of the lifting device. Due to the low height, this area can be detected manually by visual inspection or by holding a handheld device.

[0184] Figure 6 This is a structural block diagram illustrating a viewpoint generation and detection device for the fuselage skin of an unmanned vehicle, according to an exemplary embodiment. Figure 6 As shown, the device 600 includes:

[0185] The point cloud model acquisition module 610 acquires the fuselage point cloud model of the aircraft based on the pose information of the unmanned vehicle at each preset observation position and the fuselage detection points collected by the lidar on the unmanned vehicle.

[0186] Cell decomposition module 620, connected to point cloud acquisition module 610, decomposes the outline of the fuselage point cloud model projected on a preset plane into cells based on the fuselage point cloud model, the camera coverage of each unmanned vehicle, and the shooting overlap rate requirements.

[0187] The effective cell determination module 630 is connected to the cell decomposition module 620. It determines the number of point clouds contained in each decomposed cell, determines the cells with a number of point clouds greater than a preset threshold as effective cells, and obtains the normal vector and center point of the effective cell.

[0188] The initial pose determination module 640 is connected to the effective cell determination module 630. Based on the normal vector and center point of each effective cell, it calculates the initial pose information of the camera for each unmanned vehicle according to the shooting constraints of frontal view and equidistant distance.

[0189] The viewpoint optimization module 650 is connected to the initial pose determination module 640. Based on the minimum safe distance constraint and the height constraint of the lifting device, it optimizes the initial pose information and generates the optimized target viewpoint.

[0190] The coverage detection module 660, connected to the viewpoint optimization module 650, calculates the range of fuselage skin that the optimized target viewpoint can cover based on the viewpoint frustum culling strategy and the occlusion culling strategy, so as to detect the viewpoint coverage of the target viewpoint.

[0191] Figure 7 It is based on Figure 6 The diagram shown is a structural block diagram of a point cloud model acquisition module, such as... Figure 6 As shown, the point cloud model acquisition module 610 includes:

[0192] The unmanned vehicle pose acquisition unit 611 acquires the pose information of the unmanned vehicle after it reaches the preset observation position.

[0193] The detection point acquisition unit 612 is connected to the unmanned vehicle pose acquisition unit 611 and collects the fuselage detection points of the aircraft in the aircraft coordinate system through the lidar installed on the unmanned vehicle.

[0194] The point cloud model acquisition unit 613 is connected to the detection point acquisition unit 612. It converts the fuselage detection points in the aircraft coordinate system into fuselage detection points in the world coordinate system, and acquires the fuselage point cloud model composed of all fuselage detection points in the world coordinate system.

[0195] Figure 8 It is based on Figure 6 The diagram shown is a structural block diagram of an effective cell determination module, such as... Figure 8 As shown, the valid cell determination module 630 includes:

[0196] Point cloud quantity acquisition unit 631 determines the number of point clouds contained in each decomposed unit;

[0197] The valid cell determination unit 632 is connected to the point cloud quantity acquisition unit 631. If the number of point clouds contained in the cell is greater than or equal to a preset threshold N, then the cell is valid. e Define this cell as a valid cell U ij ;

[0198] Center point determination unit 633 is connected to valid cell determination unit 632, and the valid cell U is calculated. ij center point O ij ;

[0199] Normal vector determination unit 634 is connected to center point determination unit 633, and the effective cell U is calculated by principal component analysis. ij normal vector n ij .

[0200] In summary, this invention discloses a method and apparatus for generating and detecting viewpoints on fuselage skin based on an unmanned vehicle. The method includes: acquiring a fuselage point cloud model of the aircraft; decomposing the contour of the fuselage point cloud model into cells based on the fuselage point cloud model, camera coverage, and shooting overlap requirements; obtaining the normal vectors and center points of effective cells; calculating the initial pose information of the camera according to frontal and equidistant shooting constraints; generating an optimized target viewpoint based on minimum safe distance constraints and elevator height constraints; and detecting the viewpoint coverage of the target viewpoint based on a view frustum culling strategy and an occlusion culling strategy. This method, through automated data acquisition, cell decomposition, and viewpoint generation and optimization using an unmanned vehicle system, improves efficiency and safety, ensures coverage and data quality, and effectively solves the coverage problem of fuselage curved surfaces.

[0201] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0202] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0203] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for generating and detecting viewpoints on the fuselage skin of an unmanned vehicle, characterized in that, The method includes: Based on the pose information of the unmanned vehicle at each preset observation position and the fuselage detection points collected by the lidar on the unmanned vehicle, the fuselage point cloud model of the aircraft is obtained. Based on the fuselage point cloud model, the camera coverage of each unmanned vehicle, and the shooting overlap rate requirements, the outline of the fuselage point cloud model projected onto a preset plane is decomposed into cells. Determine the number of point clouds contained in each decomposed cell, identify cells with a number of point clouds greater than a preset threshold as valid cells, and obtain the normal vector and center point of the valid cells; Based on the normal vector and center point of each valid cell, the initial pose information of the camera for each unmanned vehicle is calculated according to the shooting constraints of frontal view and equidistant distance. Based on the minimum safe distance constraint and the height constraint of the lifting device, the initial pose information is optimized to generate an optimized target viewpoint; Based on the view frustum culling strategy and the occlusion culling strategy, the optimized target viewpoint's coverage area is calculated to detect the viewpoint coverage rate of the target viewpoint; wherein... The calculation of the initial camera pose information for each autonomous vehicle based on the normal vector and center point of each valid cell, according to the shooting constraints of frontal and equidistant views, includes: in, Indicates the position of the camera's optical center. This represents the distance between the center point of the cell and the optical center of the camera. Indicates the shooting distance of the nominal camera. Indicates the camera's focal length. Indicates the nominal camera focal length. This represents the centroid coordinates of the autonomous vehicle. This indicates the yaw angle of the autonomous vehicle. and Indicates the rotation angle of the gimbal. It is the tangent function. Represents the arctangent function. Indicates the height of the lifting boom. Indicates a constant bias, Representing the aircraft coordinate system Axis and Autonomous Vehicle Coordinate System The angle between the axes, if the aircraft coordinate system Rotate the axis counterclockwise to the autonomous vehicle coordinate system Axis It is a positive value. Representing the camera coordinate system Projection of the axis onto the horizontal plane and the pan-tilt coordinate system The angle between the axes, if the gimbal coordinate system Rotate the axis counterclockwise to the camera coordinate system Axis It is a positive value. Represented as camera coordinate system Axis and gimbal coordinate system The angle between the planes, if the camera coordinate system The axis is located in the gimbal coordinate system Above the plane It is a positive value. express The angle can be varied. express The angle can be varied. A valid cell is defined as a cell containing a number of point clouds greater than or equal to a preset threshold. , Valid cell The center point, Valid cell The normal vector.

2. The method for generating and detecting viewpoints of the fuselage skin of an unmanned vehicle according to claim 1, characterized in that, The step of obtaining the aircraft's fuselage point cloud model based on the pose information of the unmanned vehicle at each preset observation position and the fuselage detection points collected by the lidar on the unmanned vehicle includes: After the unmanned vehicle reaches the preset observation position, the pose information of the unmanned vehicle is acquired; The lidar installed on the unmanned vehicle collects the fuselage detection points of the aircraft in the aircraft coordinate system. Convert the fuselage detection points in the aircraft coordinate system to fuselage detection points in the world coordinate system to obtain a fuselage point cloud model composed of all fuselage detection points in the world coordinate system.

3. The method for generating and detecting viewpoints of the fuselage skin of an unmanned vehicle according to claim 2, characterized in that, The step of decomposing the outline of the fuselage point cloud model projected onto a preset plane into cells, based on the fuselage point cloud model, the camera coverage area of ​​each unmanned vehicle, and the required overlap rate, includes: Based on the fuselage point cloud model and the preset fuselage detection range, and using the least squares method, the fuselage detection points in the fuselage point cloud model are analyzed using a three-segment function expression of quadratic curve-straight line-quadratic curve. The first contour of the plane is represented as follows: ， in, ; The fuselage detection points in the fuselage point cloud model are approximated using a circular arc approximation function. The second contour of the plane is represented as follows: ; Based on the camera coverage area and shooting overlap requirements of each autonomous vehicle, determine separately The start and end points of all rows of the first contour of the plane, and The starting and ending points of all rows of the second contour of the plane are used to decompose the contour of the fuselage point cloud model projected onto the preset plane into cells.

4. The method for generating and detecting viewpoints of the fuselage skin of an unmanned vehicle according to claim 2, characterized in that, The process of determining the number of point clouds contained in each decomposed cell, identifying cells with a point cloud number greater than a preset threshold as valid cells, and obtaining the normal vector and center point of the valid cells includes: Determine the number of point clouds contained in each decomposed unit; If the number of point clouds contained in the cell is greater than or equal to a preset threshold The cell is determined as a valid cell. ; Calculate the valid cells center point ; The effective cells are calculated using principal component analysis. normal vector .

5. The method for generating and detecting viewpoints of the fuselage skin of an unmanned vehicle according to claim 1, characterized in that, The step of optimizing the initial pose information based on the minimum safe distance constraint and the height constraint of the lifting device to generate an optimized target viewpoint includes: Determine the minimum safe distance between unmanned vehicles and aircraft. ; With each fuselage detection point as the center and the minimum safe distance mentioned above... Draw circles with a radius, calculate the outer contour S of all circles, and determine the area outside the outer contour S as the safe zone where the unmanned vehicle can move freely. Determine whether the projection of each valid cell onto the horizontal plane is within the safe area; If not within the aforementioned safe zone, via Adjusting the position of the camera's optical center, and through... Adjust the camera's focal length; Determine whether the height constraint conditions of the lifting device are met at each viewpoint. ,in Indicates the minimum focal length. Indicates the maximum focal length. This indicates the minimum height of the camera's optical center under the adjustment of the lifting device. This indicates the maximum height of the camera's optical center under the adjustment of the lifting device; If the constraints of the lifting device are not met, and when season ; exist Based on Sampling is performed incrementally for each Angle calculation The angle can capture the skin area, and it determines whether the cell to be detected can be captured, until... Sampling will stop if either of the two conditions is met and the cell to be detected cannot be captured. make This indicates that the cell to be detected can be captured by camera. Angle set, calculation ,in Indicates the final choice Angle set, express The achievable coverage rate of the cells to be detected. The cardinality of a set. Indicates the maximum permissible shooting tilt angle; Obtain the optimized target viewpoint.

6. The method for generating and detecting viewpoints of the fuselage skin of an unmanned vehicle according to claim 1, characterized in that, The step of calculating the fuselage skin range that the optimized target viewpoint can cover based on the view frustum culling strategy and the occlusion culling strategy, in order to detect the viewpoint coverage of the target viewpoint, includes: Projecting the points on the fuselage skin onto the pixel coordinate system, if simultaneously satisfying... and , then it means It can be photographed in unobstructed conditions, among which, Represents horizontal resolution. Indicates the vertical resolution. A little bit on the fuselage skin , , This represents the transformation matrix between the camera coordinate system and the aircraft coordinate system. The intrinsic parameter matrix of the camera: ； Determine whether the camera's coverage area includes points not on the camera body skin based on the camera's pose information. ; If it is determined that points not on the fuselage skin are included, then occlusion has occurred and the occluded fuselage area is calculated. pass , indicating connection to the optical center and The light, among which, , Represents the magnitude of a vector. ; Calculate the light ray and the valid cell. The intersection points are used to perform plane fitting using the least squares method based on the point cloud within the valid cells. The complete coordinates of the four vertices are calculated using the fitting results. Each cell is decomposed into two triangles, the upper left and the lower right, along the diagonal. Let... , , Let the three vertices of a triangle represent the points on the triangular plane. ， If the light ray intersects the triangular plane, then Obtain the matrix form: in , , If satisfied This indicates the points on the triangular plane. Obscured; After removing all obscured points, determine the covered and uncovered areas of the fuselage skin and calculate the coverage rate.

7. A device for generating and detecting viewpoints of fuselage skin based on unmanned vehicles, characterized in that, The device includes: The point cloud model acquisition module acquires the fuselage point cloud model of the aircraft based on the pose information of the unmanned vehicle at each preset observation position and the fuselage detection points collected by the lidar on the unmanned vehicle. The cell decomposition module, connected to the point cloud acquisition module, decomposes the outline of the fuselage point cloud model projected onto a preset plane into cells based on the fuselage point cloud model, the camera coverage of each unmanned vehicle, and the shooting overlap rate requirements. The effective cell determination module is connected to the cell decomposition module. It determines the number of point clouds contained in each decomposed cell, determines the cells with a number of point clouds greater than a preset threshold as effective cells, and obtains the normal vector and center point of the effective cells. The initial pose determination module, connected to the valid cell determination module, calculates the initial pose information of the camera for each autonomous vehicle based on the normal vector and center point of each valid cell, according to the shooting constraints of frontal and equidistant viewing, including: in, Indicates the position of the camera's optical center. This represents the distance between the center point of the cell and the optical center of the camera. Indicates the shooting distance of the nominal camera. Indicates the camera's focal length. Indicates the nominal camera focal length. This represents the centroid coordinates of the autonomous vehicle. This indicates the yaw angle of the autonomous vehicle. and Indicates the rotation angle of the gimbal. It is the tangent function. Represents the arctangent function. Indicates the height of the lifting boom. Indicates a constant bias, Representing the aircraft coordinate system Axis and Autonomous Vehicle Coordinate System The angle between the axes, if the aircraft coordinate system Rotate the axis counterclockwise to the autonomous vehicle coordinate system Axis It is a positive value. Representing the camera coordinate system Projection of the axis onto the horizontal plane and the pan-tilt coordinate system The angle between the axes, if the gimbal coordinate system Rotate the axis counterclockwise to the camera coordinate system Axis It is a positive value. Represented as camera coordinate system Axis and gimbal coordinate system The angle between the planes, if the camera coordinate system The axis is located in the gimbal coordinate system Above the plane It is a positive value. express The angle can be varied. express The angle can vary within a certain range; The viewpoint optimization module, connected to the initial pose determination module, optimizes the initial pose information based on the minimum safe distance constraint and the height constraint of the lifting device to generate an optimized target viewpoint. The coverage detection module, connected to the viewpoint optimization module, calculates the range of fuselage skin that the optimized target viewpoint can cover based on the view frustum culling strategy and the occlusion culling strategy, so as to detect the viewpoint coverage of the target viewpoint.

8. The device for generating and detecting viewpoints of the fuselage skin of an unmanned vehicle according to claim 7, characterized in that, The point cloud model acquisition module includes: The autonomous vehicle pose acquisition unit acquires the pose information of the autonomous vehicle after it reaches the preset observation position. The detection point acquisition unit is connected to the unmanned vehicle pose acquisition unit and collects the fuselage detection points of the aircraft in the aircraft coordinate system through the lidar installed on the unmanned vehicle. The point cloud model acquisition unit is connected to the detection point acquisition unit. It converts the fuselage detection points in the aircraft coordinate system into fuselage detection points in the world coordinate system, and acquires a fuselage point cloud model composed of all fuselage detection points in the world coordinate system.

9. The device for generating and detecting viewpoints of the fuselage skin of an unmanned vehicle according to claim 7, characterized in that, The valid cell determination module includes: Point cloud quantity acquisition unit, which determines the number of point clouds contained in each decomposed unit; The valid cell determination unit is connected to the point cloud quantity acquisition unit. If the number of point clouds contained in the cell is greater than or equal to a preset threshold... The cell is determined as a valid cell. ; The center point determination unit is connected to the valid cell determination unit and calculates the valid cells. center point ; The normal vector determination unit, connected to the center point determination unit, calculates the effective cells using principal component analysis. normal vector .

Images