Unmanned aerial vehicle imaging modeling optimization method based on power system scene

Abstract

The invention relates to the technical field of three-dimensional modeling, in particular to an unmanned aerial vehicle imaging modeling optimization method based on a power system scene, and the method comprises the steps: improving the environment adaptability of a system through an optimization device and a data processing flow; the improved model is fusion of cross-dimension feature matching and registration methods, and multi-view combination is carried out on data acquisition; a centimeter-level precision requirement is obtained through training, and a three-dimensional building model with details is generated; an ASIFT algorithm is introduced to carry out image matching; key feature points in the point cloud data and the image are identified by combining a point cloud registration method based on ISS and 3DSC; data acquisition, image matching, point cloud registration and model generation are realized through full-process technical chain optimization; the method can help to improve the modeling precision and efficiency in the aspects of structure measurement and detail reduction of a complex building in a power system scene, and provides an end-to-end solution for industrial applications such as unmanned aerial vehicle spraying.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional modeling technology, and in particular to an optimization method for unmanned aerial vehicle (UAV) imaging modeling based on a power system scenario. Background Art

[0002] To ensure a comprehensive understanding of the drone's operating environment before actual operation, ensuring accurate positioning and efficient execution of the spraying task, a detailed analysis of the spray equipment and its surroundings is required to generate an accurate 3D model. This process involves comprehensive scanning and recording of the target equipment's structural characteristics, spatial location, and the distribution of surrounding obstacles. This environmental data is then transmitted to the spraying drone, providing it with a precise basis for navigation and operation.

[0003] However, traditional 3D modeling methods often rely on manual operations during the data collection process, making it difficult to efficiently cover complex working environments. This is especially true in scenes with high-density obstacles and complex structures, such as power equipment. Traditional methods have low data collection efficiency and are subject to blind spots and omissions. In addition, the data processing and modeling processes of traditional modeling methods are relatively lengthy, usually requiring multiple steps of splicing, optimization, and correction, and are unable to achieve real-time updates of the model, making it difficult to quickly respond and adjust when faced with dynamically changing scenes. For example, when wind speed, temperature changes, or equipment is operating, traditional modeling methods are unable to capture changes in environmental details in real time, thereby affecting the accuracy and safety of spraying operations.

[0004] In the existing technology, optimization strategies for drone imaging modeling are available. For example, patent publication number CN115860141A proposes an automated modeling optimization strategy. This strategy transforms the automated modeling process into a human-machine interactive and manually controllable process. Patent publication number CN119850856A optimizes point cloud distribution and geometric quality through weighted feature point selection and dynamic sparse point cloud adjustment, improving the terrain accuracy and detail restoration capabilities of the 3D model, and adapting to the modeling needs of complex terrain and key areas.

[0005] Although these methods have made adaptive improvements to 3D modeling registration and have achieved certain results in specific scenarios, they still have the following limitations: First, existing studies mostly optimize point cloud registration in isolation, and lack real-time analysis, decision-making, and data fusion in dynamic scenarios. For example, the data processing and modeling processes of traditional modeling methods are relatively lengthy, and usually require multiple steps of splicing, optimization, and correction, which cannot achieve real-time updates of the model, making it difficult to respond and adjust quickly when facing dynamically changing scenes. For example, when the wind force, temperature changes, or the equipment is running, the traditional modeling method cannot capture changes in environmental details in real time, thereby affecting the accuracy and safety of the spraying operation. Therefore, when facing complex targets, it is difficult for traditional 3D modeling models to achieve ideal results in terms of modeling timeliness and registration capabilities. Summary of the Invention

[0006] The main purpose of the present invention is to provide a UAV imaging modeling optimization method based on the power system scenario to effectively solve the above-mentioned problems mentioned in the background technology.

[0007] The technical solutions of the present invention are as follows:

[0008] First, a UAV imaging modeling optimization method based on the power system scenario is proposed, which includes the following steps:

[0009] S1. Oblique image data acquisition: collect modeling pictures and video data in the power system scene, plan the oblique photography route, use drones to perform aerial photography tasks, and pre-process the scene oblique image data;

[0010] S2. UAV image feature matching extracts the 3D spatial position, color information, normal vector, and intensity value data of a large number of points in 3D space to generate 3D point cloud data; an algorithm is added to use affine transformation to achieve perspective transformation of the image, obtain simulated images at various perspectives, and obtain the optimal match;

[0011] S3, accurate matching of point cloud data, using a new two-step registration scheme, through point cloud matching, filtering technology, point cloud coarse registration and point cloud fine registration to obtain accurate matching of point cloud data;

[0012] S4, 3D scene reconstruction: Apply texture mapping technology to accurately map the high-resolution texture of the original image to the point cloud surface, perform denoising, gridding, and resampling optimization processing, and ultimately generate a high-quality, complete 3D model.

[0013] A further improvement of the present invention is that the data acquisition in S1 uses oblique photography technology to shoot the target scene at vertical and oblique angles, thereby obtaining high-resolution images of the top and side of the object; oblique photography using a drone to shoot a scene can be represented by a pinhole camera imaging model, and the imaging process is to project a point M in three-dimensional space onto a corresponding point W in the image plane. The position of point W is the point where the line connecting the optical center C and the three-dimensional point M intersects the image plane R; this line is the optical axis; the focus is the intersection point W of the optical axis and R, and the distance from the optical center C to the image plane R is the focal length f; let M = (x, y, z, 1) T , W=(x′,y′,z′,1) T , according to the relationship between similar triangles, the formula is as follows:

[0014]

[0015] Further we get:

[0016]

[0017] Assume that the pixel coordinates of W are (u,v,1) T , α and β are scaling factors, and the offset of C′ is (u c ,ν c ,1) T , and then the pixel coordinate parameters u and v of W are obtained as:

[0018] u=αx′+u c ,ν=βx′+v c

[0019] After finishing, we can get:

[0020]

[0021] Assume that the coordinates of point M are (x,y,z,1) T , pixel coordinates are (u,v,1) T , and let f u =af,f v =βf, and then we get:

[0022]

[0023] In the above formula: m is the depth factor, f u and f v is the scale factor, γ is the tilt factor, (u c ,v c ) T is the offset of C′;

[0024] Let the camera's intrinsic parameter matrix K be:

[0025]

[0026] Assuming that the coordinate of the three-dimensional space point W in the real physical world is w, then:

[0027]

[0028] After finishing, we can get:

[0029] λm=KHw=Pw.

[0030] A further improvement of the present invention is that the preprocessing in S1 adopts an image restoration algorithm based on deep learning.

[0031] A further improvement of the present invention is that the algorithm in S2 is the ASIFT algorithm, which uses affine transformation to achieve image perspective transformation. By sampling the camera posture when the input image is shot, simulated images at various perspectives are obtained. These simulated image sets are used for large-scale matching, and the matching results are compared to select the best match. The ASIFT algorithm first deforms the image and uses the horizontal and vertical angles of the image to simulate various affine deformations. The affine transformation matrix is ​​decomposed into:

[0032]

[0033] Where: λ is the focal length of the camera, λ>0, Ψ is the rotation angle of the camera, the first eigenvalue of the diagonal matrix is ​​t, the second eigenvalue is 1, and is the camera viewing angle parameter,

[0034] ASIFT algorithm is used to calculate the camera viewing angle parameters and Continuously sample, sampling value t=1,α,α 2 ,…α n , selected n=5, sampling value longitude angle Wherein, b=72, k is an integer, to obtain a series of affine simulated images under different camera perspectives.

[0035] A further improvement of the present invention is that the two-step registration scheme in S3 is:

[0036] S31. Coarse registration: Coarse registration is performed based on a combination of ISS and 3DSC. The ISS algorithm extracts points with significant geometric features, while the 3DSC algorithm matches these feature points using descriptors and calculates a preliminary transformation matrix.

[0037] S32, fine registration: Based on the coarse registration, the ICP algorithm based on normal vector constraint is used to achieve fine registration of the two point cloud sets;

[0038] The ISS algorithm steps are:

[0039] S311, for each point P of the tower equipment point cloud set i Establish a local coordinate system and set the search radius r for it frame ;

[0040] S312, search the tower equipment point cloud data for each point within the set radius r frame The points around it are calculated and their weights are:

[0041] ω ij =1 / |p i -p j |,|p i -p j |<r frame

[0042] S313, calculate each point p i The covariance matrix of :

[0043]

[0044] S314, each point P i The covariance matrix cov(p i )'s eigenvalue {λ 1i ,λ 2i ,λ 3i}Carry out statistics and sort from small to large;

[0045] where r density is the radius of the sphere, p i As the center, it represents its weighted range, r frame Assume that the point cloud data has N points, and any point P is the search radius. i The coordinates are (x i ,y i ,z i ),i=0,1,…,N-1;

[0046] The 3DSC algorithm is:

[0047] There is a point cloud data P i , n is P i The normal direction is P iWith θ as the center of the sphere and n as the true north direction of the sphere, a local spherical space is established. According to the radius and two angles (azimuth and elevation), J+1, K+1 and L+1 segments are taken respectively, and a histogram data representation with J*K*L bins is obtained. The radius segmentation here uses a logarithmic operation, and the equation is as follows:

[0048]

[0049] Where: r min and r max , is the boundary of the spherical space radius;

[0050] On this basis, each bin is set with a corresponding weight. The weight formula is as follows:

[0051]

[0052] Where: V(j,k,l) ​​corresponds to the volume of the bin, pi corresponds to the local point density, p i is a point in the spherical space defined by P; through the above calculation, the number of weighted points in each bin is counted to obtain the histogram data;

[0053] The error function of the ICP algorithm is defined as:

[0054]

[0055] A further improvement of the present invention is that the three-dimensional modeling in S4 specifically includes the following steps:

[0056] S41: After point cloud matching is completed, multi-view point cloud fusion is performed;

[0057] S42. Apply texture mapping technology to accurately map the high-resolution texture of the original image to the point cloud surface;

[0058] S43, perform optimization processing such as denoising, gridding, and resampling to improve the structural accuracy and detail expression of the model;

[0059] S44. Generate a high-quality, complete three-dimensional model.

[0060] The technical effects of the present invention are as follows:

[0061] A UAV imaging modeling optimization method based on the power system scenario was constructed. In order to improve the pertinence of the sample data set, the present invention expanded the samples in the original professional data set. Oblique photography imaging was performed on the 10kV pole-mounted equipment and irregularly shaped buildings of the power system. The three-dimensional image of the modeled target in the power system scenario was captured, and its accurate three-dimensional point cloud data was obtained through processing. The registration algorithm was optimized, and the UAV image feature matching based on affine transformation was adopted. The image was deformed using ASIFT, and the camera posture of the input image was sampled when it was taken to obtain simulated images from various perspectives, and then large-scale matching was performed to obtain the optimal matching result; at the same time, during the dynamic flight of the UAV, its inertial navigation system (INS) was used for precise positioning, and the spatial position information of the target was obtained in real time. Combined with the global navigation satellite system (GNSS) and the inertial measurement unit (IMU), it is possible to accurately record the flight posture of the UAV (such as pitch angle, roll angle and yaw angle) and important data such as the coordinates of the image. Using deep learning-based image generation technology, the system predicts device models and scene data for occluded areas by learning and analyzing captured images. This solves the problem of data loss during the modeling process and improves modeling quality. To ensure accurate and rapid 3D modeling, this paper proposes a method that uses filtering algorithms (such as statistical filtering, radius filtering, or pass-through filtering) to extract valid scene point clouds and remove redundant and noisy data. Through point cloud matching and filtering techniques, the system successfully solves the problem of point cloud data alignment and fusion in complex scenes, generating a complete and accurate 3D model. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0063] Figure 1 This is a schematic diagram of the process of Example 1 of the present invention;

[0064] Figure 2 This is the flowchart of 3D reconstruction of UAV images by this method;

[0065] Figure 3 The relationship between the oblique photography imaging model and the similar triangle in the specific embodiment;

[0066] Figure 4 This is a schematic diagram of improving the UAV flight vision in this specific embodiment;

[0067] Figure 5 is the geometric model of the UAV affine camera in the present invention;

[0068] Figure 6 This is a schematic diagram of the image matching point cloud segmentation process of the present invention;

[0069] Figure 7This is a schematic diagram of removing outliers in the present invention;

[0070] Figure 8 This is a flow chart of a point cloud registration algorithm in a specific embodiment;

[0071] Figure 9 This is a schematic diagram of the ISS algorithm in a specific embodiment;

[0072] Figure 10 This is a three-dimensional modeling diagram of the 10kV pole-mounted equipment in the power system of the present invention. DETAILED DESCRIPTION

[0073] A UAV imaging modeling optimization method based on the power system scenario was constructed. In order to improve the pertinence of the sample data set, the present invention expanded the samples in the original professional data set. Oblique photography imaging was performed on the 10kV pole-mounted equipment and irregularly shaped buildings of the power system. The three-dimensional image of the modeled target in the power system scenario was captured, and its accurate three-dimensional point cloud data was obtained through processing. The registration algorithm was optimized, and the UAV image feature matching based on affine transformation was adopted. The image was deformed using ASIFT, and the camera posture of the input image was sampled when it was taken to obtain simulated images from various perspectives, and then large-scale matching was performed to obtain the optimal matching result; at the same time, during the dynamic flight of the UAV, its inertial navigation system (INS) was used for precise positioning, and the spatial position information of the target was obtained in real time. Combined with the global navigation satellite system (GNSS) and the inertial measurement unit (IMU), it is possible to accurately record the flight posture of the UAV (such as pitch angle, roll angle and yaw angle) and important data such as the coordinates of the image. Using deep learning-based image generation technology, the system predicts device models and scene data for occluded areas by learning and analyzing captured images. This solves the problem of data loss during the modeling process and improves modeling quality. To ensure accurate and rapid 3D modeling, this paper proposes a method that uses filtering algorithms (such as statistical filtering, radius filtering, or pass-through filtering) to extract valid scene point clouds and remove redundant and noisy data. Through point cloud matching and filtering techniques, the system successfully solves the problem of point cloud data alignment and fusion in complex scenes, generating a complete and accurate 3D model.

[0074] Example 1:

[0075] This embodiment proposes a UAV imaging modeling optimization method based on the power system scenario, such as Figures 1-10 As shown, the following specific steps are included:

[0076] S1. Oblique image data acquisition: collect modeling pictures and video data in the power system scene, plan the oblique photography route, use drones to perform aerial photography tasks, and pre-process the scene oblique image data;

[0077] S2. UAV image feature matching extracts the 3D spatial position, color information, normal vector, and intensity value data of a large number of points in 3D space to generate 3D point cloud data; an algorithm is added to use affine transformation to achieve perspective transformation of the image, obtain simulated images at various perspectives, and obtain the optimal match;

[0078] S3, accurate matching of point cloud data, using a new two-step registration scheme, through point cloud matching, filtering technology, point cloud coarse registration and point cloud fine registration to obtain accurate matching of point cloud data;

[0079] S4, 3D scene reconstruction: Apply texture mapping technology to accurately map the high-resolution texture of the original image to the point cloud surface, perform denoising, gridding, and resampling optimization processing, and ultimately generate a high-quality, complete 3D model.

[0080] In this embodiment, as shown in the attached Figure 3 As shown, the data acquisition in S1 uses oblique photography technology to shoot the target scene from multiple angles (vertical and oblique), thereby obtaining high-resolution images of the top and side of the object; oblique photography uses drones to shoot scenes, which can be represented by a pinhole camera imaging model. The imaging process is to project a point M in three-dimensional space onto the corresponding point W in the image plane. The position of point W is the point where the line connecting the optical center C and the three-dimensional point M intersects the image plane R. Since this line contains the optical center C and the point M in three-dimensional space, and intersects with the image plane R, it is called the optical axis. The focus is the intersection point W of the optical axis and R, and the distance from the optical center C to the image plane R is the focal length f. Let M = (x, y, z, 1) T , W=(x′,y′,z′,1) T , according to the relationship between similar triangles, the formula is as follows:

[0081]

[0082] Further we get:

[0083]

[0084] Let the pixel coordinates of W be (u,v,1) T , α and β are scaling factors, and the offset of C′ is (u c ,v c ,1) T , we can get the pixel coordinate parameters u and v of W as:

[0085] u=αx′+u c ,v=βx′+v c

[0086] After finishing, we can get:

[0087]

[0088] Let the coordinates of point M be (x,y,z,1) T , pixel coordinates are (u,ν,1) T , and let f u =af,f v =βf, we can get:

[0089]

[0090] Where: m is the depth factor, f u and f v is the scale factor, γ is the tilt factor, (u c ,v c ) T is the offset of C'.

[0091] Let the camera's intrinsic parameter matrix K be:

[0092]

[0093] Assume that the coordinate of the three-dimensional space point W in the real physical world is w, then:

[0094]

[0095] After finishing, we can get:

[0096] λm=KHw=Pw

[0097] The route plan and planning in S1 equip the data collection drone with high-resolution imaging equipment, including high-precision cameras and multispectral sensors, to obtain detailed surface features of power equipment and spectral information of the surrounding environment. The high-precision camera can capture the subtle textures and geometric features of the target scene, providing high-definition image data for 3D modeling. The multispectral sensor, by collecting spectral information in visible light, near-infrared, and other bands, provides multidimensional data support for power equipment status detection and defect identification. The drone is equipped with multiple cameras with different perspectives, including vertical and oblique cameras, at its geometric center, to simultaneously capture images from multiple vertical and oblique angles.

[0098] Furthermore, the drone uses its inertial navigation system (INS) for precise positioning, acquiring the target's spatial position information in real time. The INS, combined with the Global Navigation Satellite System (GNSS) and Inertial Measurement Unit (IMU), can accurately record important data such as the drone's flight attitude (such as pitch, roll, and yaw angles) and image coordinates.

[0099] As attached Figure 4As shown in the figure, after determining the aerial photography plan, the first step is to plan the drone's flight route. The drone's flight altitude and speed are set based on the complexity of the target scene and the modeling accuracy requirements. The choice of flight altitude must take into account both image resolution and coverage, while the flight speed must ensure image clarity and continuity. Following the pre-planned oblique photography route, the drone conducts multiple rounds of aerial photography from above and on the vertical plane to ensure that the images fully cover the target scene. To avoid data loss during model construction and ensure good overlap between adjacent images, both lateral overlap and heading overlap should be set to 60% during aerial photography. This high overlap design not only improves the accuracy of image matching, but also effectively reduces errors in the model splicing process, providing high-quality data support for subsequent 3D modeling.

[0100] Furthermore, the preprocessing in S1 uses a deep learning-based image restoration algorithm (such as the Generative Adversarial Network (GAN)) to automatically generate image content of missing areas by learning and reasoning about existing image data, ensuring data integrity. It also combines image registration and fusion technology to seamlessly stitch the restored images together to eliminate stitching artifacts. It also combines multi-source data fusion technology to fuse oblique photography data with LiDAR point cloud data.

[0101] Furthermore, as attached Figure 5 As shown, the algorithm in S2 is the ASIFT algorithm, which uses affine transformation to achieve perspective transformation of the image. By sampling the camera posture when the input image is taken, simulated images under various perspectives are obtained. These simulated image sets are used for large-scale matching, and the matching results are compared to select the best match.

[0102] The ASIFT algorithm first deforms the image and uses the horizontal and vertical angles of the image to simulate various affine deformations. The affine transformation matrix can be decomposed into:

[0103]

[0104] Where: λ is the focal length of the camera, λ>0, Ψ is the rotation angle of the camera, the first eigenvalue of the diagonal matrix is ​​t, the second eigenvalue is 1, φ and is the camera viewing angle parameter,

[0105] The affine distortion of an image mainly depends on two parameters, namely longitude φ and latitude θ. The ASIFT algorithm needs to ensure accurate accuracy of the tilt t and longitude φ, so that it is invariant to any affine transformation. The ASIFT algorithm continuously samples the camera view parameters and φ, with the sampling value t = 1, α, α 2 ,…α n , selected n=5, sampling value longitude angle Wherein, b=72, k is an integer, to obtain a series of affine simulated images under different camera perspectives.

[0106] Furthermore, as attached Figure 8 As shown in Figure 1, the two-step registration scheme in S3 is as follows: Step 1: Coarse Registration: This is achieved using a combination of ISS and 3DSC. The ISS algorithm extracts points with significant geometric features, while the 3DSC algorithm matches these feature points using descriptors to calculate a preliminary transformation matrix. Step 2: Fine Registration: Based on the coarse registration, the ICP algorithm, based on normal vector constraints, is used to achieve fine registration of the two point cloud sets. Normal vector constraints can effectively reduce local errors in point cloud matching and improve the stability and accuracy of the registration.

[0107] To address the issues of slow point cloud registration speed and low matching accuracy, a point cloud registration algorithm combining ISS and 3DSC was developed to achieve high efficiency and accuracy. The ISS algorithm makes the extracted point cloud more uniform, described by 3DSC features, and uses the SAC-IA algorithm to estimate the initial transformation matrix, followed by the ICP algorithm for fine registration.

[0108] As attached Figure 9 As shown, the ISS algorithm steps are:

[0109] 1) For each point P of the tower equipment point cloud i Establish a local coordinate system and set the search radius r for it frame .

[0110] 2) Find each point in the tower equipment point cloud data within the set radius r frame The points around it are calculated and their weights are:

[0111] ω ij =1 / |p i -p j |,|p i -p j |<r frame

[0112] 3) Calculate the covariance matrix of each point pi:

[0113]

[0114] 4) Each point P i The covariance matrix cov(p i )'s eigenvalue {λ 1i ,λ 2i ,λ 3i}Carry out statistics and sort from small to large;

[0115] where r densityis the radius of the sphere, p i As the center, it represents its weighted range, r frame is the search radius. Extract ISS features from the tower equipment point cloud. Assume that the point cloud data has N points, and any point P i The coordinates are (x i ,y i ,z i ),i=0,1,…,N-1.

[0116] The 3DSC algorithm is:

[0117] There is a point cloud data P i , n is P i Normal direction. i With n as the center of the sphere and n as the true north direction of the sphere, a local spherical space is established. According to the radius and two angles (azimuth and elevation), J+1, K+1, and L+1 segments are taken respectively, resulting in a histogram data representation with J*K*L bins. The radius segmentation here uses a logarithmic operation, and the equation is as follows:

[0118]

[0119] Where: r min and r max , is the boundary of the spherical space radius.

[0120] On this basis, each bin is set with a corresponding weight. The weight formula is as follows:

[0121]

[0122] Where: V(j,k,l) ​​corresponds to the volume of the bin, pi corresponds to the local point density, p i is a point in the spherical space defined by P. Through the above calculation, the number of weighted points in each bin is counted to obtain the histogram data.

[0123] The ICP algorithm is:

[0124] The most classic point cloud registration algorithm is based on the principle that for point p in point cloud P i , using Euclidean distance as the constraint condition, find the corresponding point q in the point cloud Q i , by calculating the corresponding point rotation and translation matrix R and translation matrix T, minimize the error function. The error function is defined as:

[0125]

[0126] Furthermore, the specific steps of step 4 3D modeling include the following:

[0127] S71: Point cloud matching is completed, and multi-view point cloud fusion is performed.

[0128] S72: Apply texture mapping technology to accurately map the high-resolution texture of the original image to the point cloud surface.

[0129] S73: Perform optimization processing such as denoising, meshing, and resampling to improve the structural accuracy and detail expression of the model

[0130] S74: Generate a high-quality, complete 3D model.

[0131] Example 2:

[0132] This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the above-mentioned drone imaging modeling optimization method based on the power system scenario by calling the computer program stored in the memory.

[0133] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) and one or more memories, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the UAV imaging modeling optimization method based on the power system scenario provided by the above method embodiment. The electronic device may also include other components for realizing the functions of the device. For example, the electronic device may also have components such as a wired or wireless network interface and an input and output interface to input and output data. This embodiment will not be described in detail here.

[0134] Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention may be implemented in the form of a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0135] Any combination of one or more computer-readable media may be employed. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0136] The present invention is described with reference to flowcharts and block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process or block in the flowcharts and block diagrams, as well as combinations of processes and blocks in the flowcharts or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts or block diagrams. Figure 1 A process or multiple processes and boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and boxes Figure 1 A step that specifies a function in one or more boxes.

[0138] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all protected by the present invention.

Claims

1. The UAV imaging modeling optimization method based on the power system scenario is characterized by: The specific steps include: S1. Oblique image data acquisition: collect modeling pictures and video data in the power system scene, plan the oblique photography route, use drones to perform aerial photography tasks, and pre-process the scene oblique image data; S2. UAV image feature matching extracts the 3D spatial position, color information, normal vector, and intensity value data of a large number of points in 3D space to generate 3D point cloud data; an algorithm is added to use affine transformation to achieve perspective transformation of the image, obtain simulated images at various perspectives, and obtain the optimal match; S3, accurate matching of point cloud data, using a new two-step registration scheme, through point cloud matching, filtering technology, point cloud coarse registration and point cloud fine registration to obtain accurate matching of point cloud data; S4, 3D scene reconstruction: Apply texture mapping technology to accurately map the high-resolution texture of the original image to the point cloud surface, perform denoising, gridding, and resampling optimization processing, and ultimately generate a high-quality, complete 3D model.

2. The UAV imaging modeling optimization method based on the power system scenario according to claim 1 is characterized by: The data acquisition in S1 uses oblique photography technology to shoot the target scene at vertical and oblique angles, thereby obtaining high-resolution images of the top and side of the object. Oblique photography uses a pinhole camera imaging model to represent the scene photography by drones. The imaging process is to project a point M in three-dimensional space onto the corresponding point W in the image plane. The position of point W is the point where the line connecting the optical center C and the three-dimensional point M intersects the image plane R. This line is the optical axis, the focus is the intersection point W of the optical axis and R, and the distance from the optical center C to the image plane R is the focal length f; M = (x, y, z, 1) T , W=(x′,y′,z′,1) T , according to the relationship between similar triangles, the formula is as follows: Further we get: Assume that the pixel coordinates of W are (u,v,1) T , α and β are scaling factors, and the offset of C′ is (u c ,v c ,1) T , the pixel coordinate parameters u and v of W are obtained as: u=αx′+u c ,v=βx′+v c After finishing, we can get: Assume that the coordinates of point M are (x,y,z,1) T , pixel coordinates are (u,v,1) T , and let f u =af,f v =βf, and then we get: In the above formula: m is the depth factor, f u and f v is the scale factor, γ is the tilt factor, (u c ,ν c ) T is the offset of C′; Let the camera's intrinsic parameter matrix K be: Assuming that the coordinate of the three-dimensional space point W in the real physical world is w, then: After finishing, we can get: λm=KHw=Pw.

3. The UAV imaging modeling optimization method based on the power system scenario according to claim 2 is characterized by: The preprocessing in S1 is to use an image restoration algorithm based on deep learning.

4. The UAV imaging modeling optimization method based on the power system scenario according to claim 3 is characterized by: The algorithm in S2 is the ASIFT algorithm, which uses affine transformation to achieve image perspective transformation. By sampling the camera posture when the input image is shot, simulated images at various perspectives are obtained. These simulated image sets are used for large-scale matching, and the matching results are compared to select the best match. The ASIFT algorithm first deforms the image and uses the horizontal and vertical angles of the image to simulate various affine deformations. The affine transformation matrix is ​​decomposed into: Where: λ is the focal length of the camera, λ>0, Ψ is the rotation angle of the camera, the first eigenvalue of the diagonal matrix is ​​t, the second eigenvalue is 1, and is the camera viewing angle parameter, ASIFT algorithm is used to calculate the camera viewing angle parameters and Continuously sample, sampling value t=1,α,α 2 ,…α n , selected n=5, sampling value longitude angle Wherein, b=72, k is an integer, to obtain a series of affine simulated images under different camera perspectives.

5. The UAV imaging modeling optimization method based on the power system scenario according to claim 4 is characterized in that: The two-step registration scheme in S3 is: S31. Coarse registration: Coarse registration is performed based on a combination of ISS and 3DSC. The ISS algorithm extracts points with significant geometric features, while the 3DSC algorithm matches these feature points using descriptors and calculates a preliminary transformation matrix. S32, fine registration: Based on the coarse registration, the ICP algorithm based on normal vector constraint is used to achieve fine registration of the two point cloud sets; The ISS algorithm steps are: S311, for each point P of the tower equipment point cloud set i Establish a local coordinate system and set the search radius r for it frame ; S312, search the tower equipment point cloud data for each point within the set radius r frame The points within the surrounding area are calculated, and their weights are: ω ij =1 / |p i -p j |,|p i -p j |<r frame S313. Calculate the covariance matrix of each point pi: S314, each point P i The covariance matrix cov(p i )'s eigenvalue {λ 1i ,λ 2i ,λ 3i }Carry out statistics and sort from small to large; where r density is the radius of the sphere, p i As the center, it represents its weighted range, r frame Assume that the point cloud data has N points, and any point P is the search radius. i The coordinates are (x i ,y i ,z i ),i=0,1,…,N-1; The 3DSC algorithm is: There is a point cloud data P i , n is P i The normal direction is P i With θ as the center of the sphere and n as the true north direction of the sphere, a local spherical space is established. According to the radius and two angles (azimuth and elevation), J+1, K+1 and L+1 segments are taken respectively, and a histogram data representation with J*K*L bins is obtained. The radius segmentation here uses a logarithmic operation, and the equation is as follows: Where: r min and r max , is the boundary of the spherical space radius; On this basis, each bin is set with a corresponding weight. The weight formula is as follows: Where: V(j,k,l) ​​corresponds to the volume of the bin, pi corresponds to the local point density, p i is a point in the spherical space defined by P; through the above calculation, the number of weighted points in each bin is counted to obtain the histogram data; The error function of the ICP algorithm is defined as:

6. The UAV imaging modeling optimization method based on the power system scenario according to claim 5 is characterized by: The three-dimensional modeling in S4 specifically includes the following steps: S41: After point cloud matching is completed, multi-view point cloud fusion is performed; S42. Apply texture mapping technology to accurately map the high-resolution texture of the original image to the point cloud surface; S43, perform optimization processing such as denoising, gridding, and resampling to improve the structural accuracy and detail expression of the model; S44. Generate a high-quality, complete three-dimensional model.

Images