A Method and System for 3D Real-Scene Modeling of New Energy Power Stations Based on Unmanned Aerial Vehicles

Abstract

This invention discloses a method and system for 3D real-scene modeling of new energy power stations based on unmanned aerial vehicles (UAVs), belonging to the field of UAV 3D modeling technology. It constructs an initial point cloud field based on lidar point cloud and visible light image data of the new energy power station collected by the UAV; it constructs three virtual semi-axes representing the lidar channel, visible light channel, and position correction vector, generating a virtual differential allocator; it configures a virtual differential allocator for each voxel unit in the initial point cloud field, and generates multiple updated point cloud representative points based on the virtual differential allocator; and it constructs a 3D real-scene model of the new energy power station based on the multiple updated point cloud representative points. This invention achieves an adaptive balance between the acquisition and fusion stages by constructing a virtual differential allocator and an offline mapping table, effectively overcoming the problem of decreased modeling accuracy caused by fluctuations in the quality of multi-source data in complex new energy power station environments.

Description

Technical Field

[0001] This invention relates to the field of UAV 3D modeling technology, specifically to a method and system for 3D real-scene modeling of new energy power stations based on UAVs. Background Technology

[0002] In the current field of 3D real-scene modeling of new energy power stations, existing technologies generally include: motion recovery structure and multi-view stereo matching methods based on visible light images, and direct ranging and point cloud reconstruction methods based on lidar. However, existing technologies have certain limitations. While the former can generate high-density point clouds and realistic color models in textured areas, feature matching is prone to failure in weakly textured or repetitive textured areas such as photovoltaic panel surfaces and tower coatings, leading to point cloud voids or geometric distortions. The latter, although providing accurate spatial coordinate information unaffected by lighting and texture conditions, has point cloud density limited by the scan line interval, resulting in insufficient ability to depict millimeter-level defects such as microcracks in blades and insulator damage. Existing integration strategies generally include using a fixed-weight weighted average after point cloud registration, or based on distance... Inverse distance weighted interpolation is used, but this type of fusion method assumes that the data quality of different sensors at different spatial locations remains constant, ignoring the dynamic influence of factors such as illumination angle, surface reflection characteristics, flight altitude and speed on lidar echo intensity and image texture gradient in actual operation; when lidar generates multipath noise due to large incident angle or specular reflection, low-quality point clouds still participate in fusion with preset weights, causing geometric reference shift; when visible light images suffer from texture gradient degradation due to backlight or overexposure, mismatched points are also included in the fusion, causing local texture blurring or ghosting; Therefore, developing a method for 3D real-scene modeling of new energy power stations based on UAVs is of great significance. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for three-dimensional real-scene modeling of new energy power stations based on unmanned aerial vehicles (UAVs) to address the shortcomings in the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for three-dimensional real-scene modeling of new energy power stations based on unmanned aerial vehicles (UAVs), comprising: An initial point cloud field is constructed based on lidar point cloud and visible light image data collected by drones from new energy power stations; Construct three virtual half-axis representing the lidar channel, visible light channel, and position correction vector, and generate a virtual differential allocator; Configure a virtual differential allocator for each voxel unit in the initial point cloud field, and generate multiple updated point cloud representative points based on the virtual differential allocator; A 3D real-world model of a new energy power station is constructed based on multiple updated point cloud representative points.

[0005] In a preferred embodiment, the step of constructing an initial point cloud field based on lidar point cloud and visible light image data collected by UAV from new energy power stations is as follows: An offline mapping table of drone flight parameters was established based on test flights of the drone. The drone's shooting path is generated based on an offline mapping table. The shooting path includes the flight path, shooting angle, and shooting density. Point cloud data of new energy power stations are collected based on the shooting path. The point cloud data includes lidar point cloud and dense point cloud of images. Among them, the dense point cloud of the image is generated based on visible light image data. The visible light image data is input into the structure-of-motion motion algorithm to generate a sparse point cloud, and the camera pose of each visible light image is calculated to generate the dense point cloud of the image. The three-dimensional space of the new energy power station is divided into a three-dimensional voxel grid. The lidar point cloud and the dense point cloud of the image are spatially aligned and inserted into the corresponding voxels to generate the initial point cloud field.

[0006] In a preferred embodiment, the step of constructing three virtual half-axes representing the lidar channel, the visible light channel, and the position correction vector, and generating the virtual differential allocator, is as follows: The left half-axis is defined as the LiDAR channel, and the left half-axis is used to correlate the local quality score of the LiDAR channel. The local quality score of the lidar channel is generated by multiplying the density score of the lidar point cloud, the inverse score of the plane fitting residual, and the edge intensity score, and then normalizing them. The right half-axis is defined as the visible light channel, and the right half-axis is used to correlate the local quality score of the visible light channel. The visible light channel local quality score is generated by multiplying the texture gradient score of the dense point cloud of the image, the multi-view matching confidence score, and the reciprocal score of the reprojection error, and then normalizing them. The central input axis is defined as the position correction vector. The central input axis is used to correlate the position correction vector, which is the balanced output of the local quality score of the lidar channel and the local quality score of the visible light channel. Define the resistance ratio between the left and right half-shafts. The resistance ratio is the ratio of the resistance value of the left half-shaft to the resistance value of the right half-shaft. The drag value of the left half-axis is 1 minus the local quality score of the lidar channel, and the drag value of the right half-axis is equal to 1 minus the local quality score of the visible light channel. Define an adaptive fusion rule based on the drag ratio. The adaptive fusion rule is to use an optimization function to make the drag ratio meet a preset value.

[0007] In a preferred embodiment, the step of establishing an offline mapping table of UAV flight parameters based on the UAV's test flights is as follows: Perform test flights and record the test flight parameters during the test flight process. The test flight parameters include flight speed parameters, shooting distance parameters, exposure parameters, and illumination angle parameters. Based on the test flight parameters, lidar channel, and visible light channel, the local data quality score of the lidar channel and the local data quality score of the visible light channel are obtained for each set of test flight parameters. Record the mapping relationship between each set of test flight parameters and the corresponding local quality scores of the lidar channel and the visible light channel, and generate an offline mapping table; The channel quality difference value for each set of test flight parameters is calculated based on the offline mapping table. The channel quality difference value is the difference between the local data quality score of the lidar channel and the local data quality score of the visible light channel. A preset difference threshold is set, and test flight parameters with channel quality difference values ​​below the difference threshold are used as balanced flight parameters. The balanced feature vector of the balanced flight parameters is then extracted. The balanced feature vector includes flight speed feature components, shooting distance feature components, exposure parameter feature components, and illumination angle feature components; Test flight parameters with channel quality differences exceeding the difference threshold are marked as unbalanced parameter groups, and unbalanced feature vectors of unbalanced parameter groups are extracted. Calculate the offset of each component of the imbalance parameter from the mean of each component of the equilibrium eigenvector; Based on the offset of each imbalance parameter component, the values ​​of each characteristic component in the balance parameter group are adjusted in the direction of offsetting the offset to generate an optimized compensation value for the imbalance parameter group. Optimized flight parameters are generated based on the optimized compensation values, and the drone's shooting path is generated based on the balanced flight parameters and the optimized flight parameters.

[0008] In a preferred embodiment, the step of configuring a virtual differential allocator for each voxel unit in the initial point cloud field, and generating multiple updated point cloud representative points based on the virtual differential allocator, is as follows: For each voxel, the geometric median of the lidar point cloud and the geometric median of the dense point cloud of the image are calculated respectively. The average of the spatial coordinates of the two geometric medians is taken as the initial representative point of the point cloud of the voxel. Based on the virtual differential allocator, the local quality score of the lidar channel and the local quality score of the visible light channel within the current voxel unit are calculated respectively; Simultaneously, the drag ratio is calculated, and a position correction vector is generated based on the optimized function to balance the drag ratio. When the local quality score of the lidar channel or the local quality score of the visible light channel is unbalanced, the virtual differential lock is activated to lock the corresponding channel. The position correction vector is superimposed onto the initial point cloud representative points to generate the updated point cloud representative.

[0009] In a preferred embodiment, the step of activating a virtual differential lock to lock the corresponding channel when the local quality score of the lidar channel or the local quality score of the visible light channel is unbalanced is as follows: When the local quality score of the lidar channel or the local quality score of the visible light channel is continuously lower than the preset score threshold in a preset number of voxel units; Then, the virtual differential lock is activated, the quality score of the corresponding half-axis that is continuously below the scoring threshold is forcibly set to zero, the half-axis is locked, and the position is corrected based on the other half-axis to generate a position correction vector.

[0010] In a preferred embodiment, the step of constructing a three-dimensional real-scene model of the new energy power station based on multiple updated point cloud representative points is as follows: All updated point cloud representative points are merged according to their respective spatial coordinates to generate a fused point cloud; A Poisson surface reconstruction algorithm is used, with fused point cloud as input, to construct a triangular mesh model; The original visible light image is projected onto a triangular mesh model, and weighted fusion is performed based on the visibility and incident angle of each viewpoint to generate a three-dimensional real-scene model of the new energy power station.

[0011] This invention also provides a UAV-based 3D real-scene modeling system for new energy power stations, including: Data acquisition module: Based on the acquisition of lidar point cloud and visible light image data of new energy power stations by UAV, an initial point cloud field is constructed; Differential Allocator Module: Connected to the data acquisition module, it constructs three virtual half-axes representing the lidar channel, visible light channel, and position correction vector, and generates a virtual differential allocator; Point cloud update module: Connected to the differential allocator module, it configures a virtual differential allocator for each voxel unit in the initial point cloud field, and generates multiple representative points for updating the point cloud based on the virtual differential allocator; Model building module: Connected to the point cloud update module, it builds a 3D real-world model of the new energy power station based on multiple updated point cloud representative points.

[0012] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. This invention constructs three virtual half-axis representing the LiDAR channel, visible light channel, and position correction vector, generates a virtual differential allocator, and optimizes the UAV shooting path based on an offline mapping table, achieving adaptive balance in both acquisition and fusion stages. In the acquisition stage, an offline mapping table is established using test flights to record the local quality scores of the two channels under different flight parameters, calculate the channel quality difference, filter and extract feature vectors for balanced flight parameters, and optimize and compensate for imbalanced parameter groups to generate a shooting path that balances the quality of the two channels. This strategy simulates the principle of a differential automatically distributing power based on drag, automatically increasing waypoint density and adjusting the gimbal angle in complex areas, avoiding the low data quality on one side caused by traditional fixed routes. In the fusion stage, a virtual differential allocator is configured for each voxel unit, calculating the local quality scores and drag ratios of the left and right half-axis, generating a position correction vector using an optimization function, and activating a virtual differential lock to lock inferior channels when the channel quality continuously falls below a threshold. This two-stage differential mechanism enables the system to adapt to changes in terrain and lighting, significantly improving modeling robustness and reducing manual parameter adjustments. 2. This invention achieves efficient fusion of LiDAR point clouds and dense image point clouds by embedding a virtual differential allocator into the voxel-level point cloud update process and using the geometric median as the initial representative point. Within each voxel unit, the geometric median of both point clouds is calculated, and their average is used as the initial representative point, effectively avoiding the influence of outliers from a single sensor. During iterative updates, the left and right half-axis correction vectors are taken as the median of the vectors pointing from the current representative point to their respective point cloud spaces, effectively resisting noise within each channel. Adaptive weight allocation based on drag ratio ensures the sum of the weights of the two channels is always equal to 1, and the optimization function is sensitive to changes in drag ratio, allowing the dominant channel to quickly gain dominance. The virtual differential lock cuts off the contribution of inferior channels in extreme failure scenarios, preventing erroneous information from contaminating the model. This method fully integrates geometric accuracy and texture details, demonstrating excellent performance in modeling minute defects such as wind turbine blade cracks and photovoltaic panel hot spots, reducing data redundancy and increasing edge point cloud density. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1 This is a flowchart of the method of the present invention.

[0015] Figure 2 This is a system block diagram of the present invention.

[0016] Figure 3 This is a logic block diagram of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1, please refer to Figure 1 and Figure 3 As shown in this embodiment, the method for 3D real-scene modeling of new energy power stations based on unmanned aerial vehicles includes: S1. Construct an initial point cloud field based on lidar point cloud and visible light image data collected by UAVs from new energy power stations; S2. Construct three virtual half-axes representing the lidar channel, visible light channel, and position correction vector, and generate a virtual differential allocator; S3. Configure a virtual differential allocator for each voxel unit in the initial point cloud field, and generate multiple updated point cloud representative points based on the virtual differential allocator; S4. Construct a 3D real-world model of the new energy power station based on multiple updated point cloud representative points.

[0019] As described in steps S1-S4 above, the existing technologies in the field of 3D real-scene modeling of new energy power stations generally include: motion recovery structure and multi-view stereo matching method based on visible light images and direct ranging and point cloud reconstruction method based on lidar. However, existing technologies have certain limitations. While the former can generate high-density point clouds and realistic color models in textured areas, feature matching is prone to failure in weakly textured or repetitive textured areas such as photovoltaic panel surfaces and tower coatings, leading to point cloud voids or geometric distortions. The latter, although providing accurate spatial coordinate information unaffected by lighting and texture conditions, has point cloud density limited by the scan line interval, resulting in insufficient ability to depict millimeter-level defects such as microcracks in blades and insulator damage. Existing integration strategies generally include using a fixed-weight weighted average after point cloud registration, or based on distance... Inverse distance weighted interpolation is used, but this type of fusion method assumes that the data quality of different sensors at different spatial locations remains constant, ignoring the dynamic influence of factors such as illumination angle, surface reflection characteristics, flight altitude and speed on lidar echo intensity and image texture gradient in actual operation; when lidar generates multipath noise due to large incident angle or specular reflection, low-quality point clouds still participate in fusion with preset weights, causing geometric reference shift; when visible light images suffer from texture gradient degradation due to backlight or overexposure, mismatched points are also included in the fusion, causing local texture blurring or ghosting; This invention constructs three virtual half-axis representing the lidar channel, visible light channel, and position correction vector, generates a virtual differential allocator, and optimizes the UAV shooting path based on an offline mapping table, achieving adaptive balance in both acquisition and fusion stages. In the acquisition stage, an offline mapping table is established using test flights, recording the local quality scores of the two channels under different flight parameters, calculating the channel quality difference, selecting balanced flight parameters and extracting feature vectors, and optimizing and compensating for imbalanced parameter groups to generate a shooting path that balances the quality of the two channels. This strategy simulates the principle of a differential automatically distributing power based on drag, automatically increasing waypoint density and adjusting the gimbal angle in complex areas, avoiding the low data quality on one side caused by traditional fixed routes. In the fusion stage, a virtual differential allocator is configured for each voxel unit, calculating the local quality scores and drag ratios of the left and right half-axis, generating a position correction vector using an optimization function, and activating a virtual differential lock to lock inferior channels when the channel quality continuously falls below a threshold. This two-stage differential mechanism enables the system to adapt to changes in terrain and lighting, significantly improving modeling robustness and reducing manual parameter adjustments. By embedding a virtual differential allocator into the voxel-level point cloud update process and using the geometric median as the initial representative point, efficient fusion of LiDAR point cloud and dense image point cloud is achieved. Within each voxel unit, the geometric median of both point clouds is calculated, and their average is used as the initial representative point, effectively avoiding the influence of outliers from a single sensor. During iterative updates, the left and right half-axis correction vectors are taken as the median of the vectors pointing from the current representative point to their respective point cloud spaces, effectively resisting noise within each channel. Adaptive weight allocation based on drag ratio ensures the sum of the weights of the two channels is always equal to 1, and the optimization function is sensitive to changes in drag ratio, allowing the dominant channel to quickly gain dominance. The virtual differential lock cuts off the contribution of inferior channels in extreme failure scenarios, preventing erroneous information from contaminating the model. This method fully integrates geometric accuracy and texture details, demonstrating excellent performance in modeling minute defects such as wind turbine blade cracks and photovoltaic panel hot spots, reducing data redundancy and increasing edge point cloud density.

[0020] In one embodiment, step S1, which constructs an initial point cloud field based on lidar point cloud and visible light image data collected by a UAV from a new energy power station, includes: S11. Based on the test flight of the UAV, establish an offline mapping table of UAV flight parameters; S12. Generate the drone's shooting path based on the offline mapping table. The shooting path includes the flight path, shooting angle, and shooting density. S13. Collect point cloud data of new energy power stations based on the shooting path. The point cloud data includes lidar point cloud and dense point cloud of images. S14. Among them, the dense point cloud of the image is generated based on the visible light image data. The visible light image data is input into the motion recovery structure algorithm to generate a sparse point cloud, and the camera pose of each visible light image is calculated to generate the dense point cloud of the image. S15. Divide the three-dimensional space of the new energy power station into a three-dimensional voxel grid, spatially align the lidar point cloud and the dense point cloud of the image, and insert them into the corresponding voxels to generate the initial point cloud field. As described in steps S11-S15 above, a test flight is performed in a typical area of ​​the new energy power station. During the flight, multiple sets of different flight speed parameters, shooting distance parameters, exposure parameters, and illumination angle parameters are recorded. Simultaneously, lidar point clouds and visible light images are acquired. The local quality score of the lidar channel and the local quality score of the visible light channel corresponding to each set of parameters are calculated through post-processing. The local quality score of the lidar channel is obtained by multiplying the density score, the inverse score of the plane fitting residual, and the edge intensity score of the lidar point cloud acquired under that set of parameters and then normalizing the result. The local quality score of the visible light channel is obtained by normalizing the results of the previous steps. The texture gradient score, multi-view matching confidence score, and reciprocal score of the dense point cloud generated from the visible light image are multiplied and normalized. Each set of flight parameters is associated with and stored in an offline mapping table with the corresponding local quality scores of the two channels. Based on the offline mapping table, the channel quality difference value for each set of test flight parameters is calculated. This channel quality difference value is the difference between the local data quality score of the lidar channel and the local data quality score of the visible light channel. A preset difference threshold is set, and test flight parameters with channel quality difference values ​​below the threshold are selected as balanced flight parameters. The balanced feature vector of the balanced flight parameters is then extracted. The balanced feature vector includes flight speed feature components, shooting distance feature components, exposure parameter feature components, and illumination angle feature components. Test flight parameters with channel quality differences exceeding a difference threshold are marked as imbalanced parameter groups, and imbalanced feature vectors are extracted from these groups. The offset of each imbalanced parameter component from the mean of each component in the balanced feature vector is calculated. Based on the offset of each imbalanced parameter component, the values ​​of each feature component in the balanced parameter group are adjusted in the direction of offset cancellation, generating optimized compensation values ​​for that imbalanced parameter group. Optimized flight parameters are generated based on the optimized compensation values, and the balanced flight parameters and optimized flight parameters are then compared. The parameters generate the shooting path of the UAV, which includes the flight path, shooting angle, and shooting density. The UAV is controlled to fly according to the generated shooting path to collect the lidar point cloud and visible light images of the new energy power station. The collected visible light image data is input into the structure-of-motion motion algorithm to generate a sparse point cloud and solve the camera pose of each visible light image. Based on the camera pose, a multi-view stereo matching algorithm is used to generate a dense point cloud. The three-dimensional space of the new energy power station is divided into a three-dimensional voxel mesh. The lidar point cloud and the dense point cloud are spatially aligned and inserted into the corresponding voxels to generate the initial point cloud field.

[0021] In one embodiment, step S2, which constructs three virtual half-axes representing the lidar channel, the visible light channel, and the position correction vector to generate the virtual differential allocator, includes: S21. Define the left half-axis as the LiDAR channel. The left half-axis is used to correlate the local quality score of the LiDAR channel. S22. The local quality score of the lidar channel is generated by multiplying the density score, the inverse score of the plane fitting residual, and the edge intensity score of the lidar point cloud and then normalizing them. S23. Define the right half-axis as the visible light channel. The right half-axis is used to correlate the local quality score of the visible light channel. S24. The local quality score of the visible light channel is generated by multiplying the texture gradient score of the dense point cloud of the image, the multi-view matching confidence score, and the reciprocal score of the reprojection error and then normalizing them. S25. Define the central input axis as the position correction vector. The central input axis is used to correlate the position correction vector. The position correction vector is the balanced output of the local quality score of the lidar channel and the local quality score of the visible light channel. S26. Define the resistance ratio between the left and right half-shafts. The resistance ratio is the ratio of the resistance value of the left half-shaft to the resistance value of the right half-shaft. S27. The drag value of the left half-axis is 1 minus the local quality score of the lidar channel, and the drag value of the right half-axis is equal to 1 minus the local quality score of the visible light channel. S28. Define an adaptive fusion rule based on the drag ratio. The adaptive fusion rule is to use an optimization function to make the drag ratio meet a preset ratio value. As described in steps S21-S28 above, the left half-axis is defined as the LiDAR channel. This left half-axis is used to associate the local quality score of the LiDAR channel within the current voxel unit with the LiDAR point cloud in subsequent iterations. This score reflects the reliability of the LiDAR data in the voxel unit. The steps are as follows: First, the density score is obtained by calculating the ratio of the number of data points in the LiDAR point cloud within the current voxel unit to the total number of points in the LiDAR point cloud and the dense point cloud of the image within the voxel unit. Then, all LiDAR point cloud data points within the voxel unit are fitted to a spatial plane. The distance from each LiDAR point cloud data point to the spatial plane is calculated and the average value is obtained. The reciprocal of this average value is taken to obtain the reciprocal score of the plane fitting residual. Then, the LiDAR data points within the voxel unit are... Radar point cloud data points are divided into multiple sub-blocks according to their spatial location. The normal vector of each sub-block is calculated, and the standard deviation of the angle between the normal vectors of all sub-blocks is calculated to obtain the edge strength score. Finally, the density score, the inverse score of the plane fitting residual, and the edge strength score are multiplied and normalized to ensure that the result falls within the range of 0 to 1, outputting the local quality score of the LiDAR channel. The right half-axis is defined as the visible light channel, used to associate the local quality score of the visible light channel of the dense point cloud image within the current voxel unit. This score reflects the reliability of the visible light image data in the voxel unit. The acquisition steps are as follows: extract the original visible light image region corresponding to each dense point cloud data point within the current voxel unit, calculate the gradient magnitude of all pixels within this region, and calculate the average value to obtain the local quality score. Texture gradient scoring is performed by counting the number of images in which each dense point cloud data point participates in multi-view matching and the consistency of feature points of that data point across all participating images. The two are multiplied to obtain the matching confidence score for each data point. The average of all matching confidence scores is then taken as the multi-view matching confidence score. Each dense point cloud data point within the voxel unit is projected back to its corresponding multiple images, and the pixel distance between the projected point and the actual feature point is calculated and averaged. The reciprocal of this average is then taken to obtain the reciprocal reprojection error score. Finally, the texture gradient score, multi-view matching confidence score, and reciprocal reprojection error score are multiplied and normalized to output the local quality score for the visible light channel. The central input axis is defined as the position correction vector. The position correction vector is used to characterize the direction and distance that the current voxel unit representative point should move. Its physical meaning is the output result after the local quality score of the lidar channel and the local quality score of the visible light channel are balanced by the virtual differential distributor. The drag ratio between the left half axis and the right half axis is defined. The specific calculation method is as follows: first, the drag value of the left half axis is equal to 1 minus the local quality score of the lidar channel, and the drag value of the right half axis is equal to 1 minus the local quality score of the visible light channel. Then, the drag ratio is obtained by dividing the drag value of the left half axis by the drag value of the right half axis. This drag ratio quantifies the relative data quality difference between the left and right channels in the current voxel unit. A drag ratio greater than 1 indicates that the lidar channel has a large drag, i.e., poor quality, and a drag ratio less than 1 indicates that the visible light channel has a large drag.The adaptive fusion rule based on drag ratio employs an optimization function to ensure the drag ratio meets a preset value. This is achieved by setting a target drag ratio, typically 1 (meaning equal drag in both channels), and mapping the current drag ratio to left and right half-axis weights using an S-curve function. The sum of these weights is always equal to 1. When the drag ratio deviates from the preset value, the optimization function automatically adjusts the weight distribution of the two channels to bring the drag ratio closer to the preset value, achieving adaptive fusion output of the position correction vector. Furthermore, specific drag ratios can be set for specific regions, including but not limited to the wind turbine blade edge region, photovoltaic module junction box region, transmission line insulator string region, highly reflective photovoltaic panel surface region, and wind turbine tower surface region under backlight conditions. Within these specific regions, the preset ratio is adjusted to deviate from 1 to strengthen the dominant role of a particular channel.

[0022] In one embodiment, step S11, which establishes an offline mapping table of UAV flight parameters based on the UAV's test flights, includes: S111. Perform a test flight and record the test flight parameters during the test flight. The test flight parameters include flight speed parameters, shooting distance parameters, exposure parameters, and illumination angle parameters. S112. Based on the test flight parameters, lidar channel, and visible light channel, obtain the local data quality score of the lidar channel and the local data quality score of the visible light channel corresponding to each set of test flight parameters. S113. Record the mapping relationship between each set of test flight parameters and the corresponding local quality score of the lidar channel and the local quality score of the visible light channel, and generate an offline mapping table; S114. Calculate the channel quality difference value for each set of test flight parameters based on the offline mapping table. The channel quality difference value is the difference between the local data quality score of the lidar channel and the local data quality score of the visible light channel. S115. Set a preset difference threshold, retrieve test flight parameters whose channel quality difference value is lower than the difference threshold as balanced flight parameters, and extract the balanced feature vector of the balanced flight parameters; S116. The balanced feature vector includes flight speed feature components, shooting distance feature components, exposure parameter feature components, and illumination angle feature components; S117. Mark the test flight parameters whose channel quality difference value is higher than the difference threshold as the unbalanced parameter group, and extract the unbalanced feature vector of the unbalanced parameter group. S118. Calculate the offset of each component of each imbalance parameter from the mean of each component of the balance eigenvector; S119. Based on the offset of each unbalanced parameter component, adjust the values ​​of each characteristic component in the balance parameter group in the direction of offsetting the offset to generate an optimized compensation value for the unbalanced parameter group. S1110. Generate optimized flight parameters based on optimized compensation values, and generate the drone's shooting path based on balanced flight parameters and optimized flight parameters; As described in steps S111-S1110 above, the UAV is controlled to fly in a typical area of ​​the new energy power station according to multiple preset sets of different flight speed parameters, shooting distance parameters, exposure parameters, and illumination angle parameters. During the flight, LiDAR point cloud and visible light images are collected simultaneously, and the flight speed value, shooting distance value, exposure value, and the angle between the sunlight and the shooting direction corresponding to each set of parameters are recorded in real time as illumination angle parameters. The local data quality score of the LiDAR channel and the local data quality score of the visible light channel corresponding to each set of test flight parameters are obtained. The flight speed parameter, shooting distance parameter, exposure parameter, and illumination angle parameter of each set of test flight parameters are used as keys to calculate the LiDAR... Local quality scores for both the LiDAR channel and the visible light channel are stored as key-value pairs in the database, forming an offline mapping table. For each record in the offline mapping table, the local quality score for the LiDAR channel is subtracted from the local quality score for the visible light channel, and the absolute value is taken as the channel quality difference value for that set of test flight parameters. A preset difference threshold is set, and all records in the offline mapping table are iterated to select test flight parameters whose channel quality difference value is lower than the threshold. These parameters are marked as balanced flight parameters. Then, for each set of balanced flight parameters, the flight speed parameter, shooting distance parameter, exposure parameter, and illumination angle parameter are used as four feature components to form a balanced feature vector, which is then used to determine the channel quality difference value. Test flight parameters with difference values ​​exceeding the difference threshold are marked as unbalanced parameter groups. Unbalanced feature vectors are extracted from these groups. The test flight parameters with channel quality difference values ​​exceeding the difference threshold in the offline mapping table are traversed, and the four components of each parameter group are extracted to form unbalanced feature vectors. All unbalanced feature vectors form the unbalanced parameter group. Next, the offset of each component of each unbalanced parameter from the mean of each component of the balanced feature vector is calculated. The steps are as follows: First, the arithmetic mean of each component of all balanced feature vectors is calculated to obtain the mean balanced flight speed, mean balanced shooting distance, mean balanced exposure parameters, and mean balanced illumination angle. Then, for each unbalanced feature vector, its flight speed feature component is subtracted from the mean balanced flight speed. To obtain the flight speed offset, the shooting distance offset is obtained by subtracting the average balanced shooting distance from its shooting distance feature component; the exposure parameter offset is obtained by subtracting the average balanced exposure parameter from its exposure parameter feature component; and the illumination angle offset is obtained by subtracting the average balanced illumination angle from its illumination angle feature component. For each parameter in each imbalance parameter group, the values ​​of each feature component in the balanced flight parameter group are adjusted in the direction of offset cancellation based on the sign and magnitude of the offset of its four components. That is, if the flight speed offset of a certain imbalance parameter is positive, the flight speed component in the optimized compensation value is negative, and the adjustment range is proportional to the absolute value of the offset. Based on this, four optimized compensation value components are generated for that imbalance parameter group.The balanced flight parameters are superimposed with the corresponding optimized compensation values ​​to obtain the optimized flight parameters for the imbalanced parameter group. Then, all optimized flight parameters are merged with the balanced flight parameters and arranged in spatial order. Corresponding flight speed, shooting distance, exposure parameters, and illumination angle are assigned to each spatial location of the new energy power station. These parameters are then converted into a sequence of waypoints along the flight path. Each waypoint is labeled with its flight speed, shooting distance, gimbal shooting angle, and shooting density, generating the complete shooting path of the UAV.

[0023] In one embodiment, step S3, which configures a virtual differential allocator for each voxel unit in the initial point cloud field and generates multiple updated point cloud representative points based on the virtual differential allocator, includes: S31. For each voxel unit, calculate the geometric median of the lidar point cloud and the geometric median of the dense point cloud of the image within that voxel unit, and take the average of the spatial coordinates of the two geometric median points as the initial point cloud representative point of that voxel unit. S32. Based on the virtual differential allocator, calculate the local quality score of the lidar channel and the local quality score of the visible light channel in the current voxel unit respectively; S33. Simultaneously, calculate the drag ratio, balance the drag ratio based on the optimization function, and generate a position correction vector; S34. When the local quality score of the lidar channel or the local quality score of the visible light channel is unbalanced, the virtual differential lock is activated to lock the corresponding channel. S35. Superimpose the position correction vector onto the initial point cloud representative points to generate an updated point cloud representative; As described in steps S31-S35 above, within each voxel unit, the geometric median of the LiDAR point cloud and the geometric median of the dense point cloud of the image are first calculated, and the average of the spatial coordinates of the two geometric median points is taken as the initial point cloud representative point of the voxel unit. Then, based on the virtual differential allocator, the local quality score of the LiDAR channel and the local quality score of the visible light channel of the current voxel unit are calculated, and the drag value of the left half-axis and the drag value of the right half-axis are calculated based on these two scores, thereby obtaining the drag ratio. Using the adaptive fusion rule in the virtual differential allocator, the drag ratio is mapped to the left half-axis weight and the right half-axis weight through the optimization function. At the same time, the left half-axis correction vector and the right half-axis correction vector are calculated. The two are the median of the spatial vector from the current initial point cloud representative point to their respective point cloud data points. The product of the left half-axis correction vector and the left half-axis weight is added to the product of the right half-axis correction vector and the right half-axis weight to obtain the position correction vector. During the iteration process, when the local quality score of the lidar channel or the local quality score of the visible light channel is lower than the preset threshold in multiple consecutive voxel units, the channel is determined to be unbalanced. The virtual differential lock is activated, and the half-axis weight corresponding to the unbalanced channel is forcibly set to zero, relying entirely on another channel for position correction. Finally, the position correction vector is superimposed on the initial point cloud representative point to generate the updated point cloud representative point. The above process is repeated for all voxel units to obtain all updated point cloud representative points.

[0024] In one embodiment, step S34, which involves activating a virtual differential lock to lock the corresponding channel when the local quality score of the lidar channel or the local quality score of the visible light channel is unbalanced, includes: S341. When the local quality score of the lidar channel or the local quality score of the visible light channel is continuously lower than the preset score threshold in a preset number of voxel units; S342, then activate the virtual differential lock, force the quality score of the corresponding half-shaft that is continuously below the scoring threshold to be set to zero, lock the half-shaft, perform position correction based on the other half-shaft, and generate a position correction vector. As described in steps S341-S342 above, during the iterative update process, the system maintains a sliding window counter. This counter tracks the number of times the local quality score of the same channel in the current voxel unit and its adjacent consecutive voxel units is lower than a preset score threshold. The preset number can be set according to the actual scenario, for example, three voxel units. When the local quality score of the LiDAR channel is lower than the preset score threshold in three consecutive voxel units, or when the local quality score of the visible light channel is lower than the preset score threshold in three consecutive voxel units, an imbalance judgment condition is triggered. Once the imbalance judgment condition is triggered, the system immediately activates the virtual differential lock, forcibly setting the quality score of the half-axis corresponding to the imbalanced channel to zero, and simultaneously forcibly setting the weight of the half-axis to zero. The locking of this half-axis prevents it from having any impact on subsequent position correction calculations. At this point, the position correction vector is entirely contributed by the other half-axis. That is, if the LiDAR channel is locked, the position correction vector is directly taken as the right half-axis correction vector multiplied by the right half-axis weight, and the right half-axis weight is automatically adjusted to one. Conversely, if the visible light channel is locked, the position correction vector is directly taken as the left half-axis correction vector multiplied by the left half-axis weight, and the left half-axis weight is automatically adjusted to one. Then, the position correction vector is superimposed on the current representative point according to the standard procedure to complete the position correction. The locking state will continue until the local quality score of the locked channel in multiple consecutive voxel units recovers to above the preset score threshold. At this time, the system automatically releases the virtual differential lock and restores the normal working mode in which both channels jointly participate in position correction.

[0025] In one embodiment, step S4, which involves constructing a 3D reality model of a new energy power station based on multiple updated point cloud representative points, includes: S41. Merge all updated point cloud representative points according to their respective spatial coordinates to generate a fused point cloud; S42. Using the Poisson surface reconstruction algorithm, a triangular mesh model is constructed with the fused point cloud as input. S43. Project the original visible light image onto the triangular mesh model, and perform weighted fusion based on the visibility and incident angle of each viewpoint to generate a three-dimensional real-scene model of the new energy power station. As described in steps S41-S43 above, when generating a 3D real-world model of a new energy power station, all updated point cloud representative points are first merged according to their respective spatial coordinate positions. The steps are as follows: traverse each voxel unit, extract the 3D spatial coordinates of the updated point cloud representative points of that voxel unit, and combine all the extracted spatial coordinate points to form a complete fused point cloud. This fused point cloud contains both the geometric accuracy information of the lidar point cloud and the texture detail information of the dense point cloud of the image. Then, the Poisson surface reconstruction algorithm is used to construct a triangular mesh model with the fused point cloud as input. First, the normal vector direction of each point in the fused point cloud is estimated. The matching relationship between the gradient field of the indicator function and the normal vector field of the point cloud is constructed by solving the Poisson equation. Finally, a continuous triangular mesh model is generated by the isosurface extraction algorithm. This triangular mesh model accurately represents the three-dimensional geometric shape of facilities such as wind turbine blades, photovoltaic panels, and transmission towers in a new energy power station. Finally, the original visible light images are projected onto the triangular mesh model for texture mapping. Specifically, the camera pose parameters of each original visible light image are obtained. For each triangular facet in the triangular mesh model, the set of visible images of that facet in all original visible light images is determined. The image with the smallest angle between the normal vector of the facet and the camera optical axis and the largest projected area is selected as the best texture source. At the same time, the differences in illumination and resolution between different images are considered. Based on the visibility and incident angle of each viewpoint, a weighted fusion is performed. The texture pixels in multiple images are mixed according to weights and mapped onto the surface of the corresponding triangular facet, finally generating a three-dimensional real-scene model of the new energy power station.

[0026] Example 2, please refer to Figure 2 As shown in this embodiment, the UAV-based 3D real-scene modeling system for new energy power stations includes: Data acquisition module: Based on the acquisition of lidar point cloud and visible light image data of new energy power stations by UAV, an initial point cloud field is constructed; Differential Allocator Module: Connected to the data acquisition module, it constructs three virtual half-axes representing the lidar channel, visible light channel, and position correction vector, and generates a virtual differential allocator; Point cloud update module: Connected to the differential allocator module, it configures a virtual differential allocator for each voxel unit in the initial point cloud field, and generates multiple representative points for updating the point cloud based on the virtual differential allocator; Model building module: Connected to the point cloud update module, it builds a 3D real-world model of the new energy power station based on multiple updated point cloud representative points.

[0027] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for three-dimensional real scene modeling of a new energy station based on a UAV, characterized in that, include: An initial point cloud field is constructed based on lidar point cloud and visible light image data collected by drones from new energy power stations; Construct three virtual half-axis representing the lidar channel, visible light channel, and position correction vector, and generate a virtual differential allocator; Configure a virtual differential allocator for each voxel unit in the initial point cloud field, and generate multiple updated point cloud representative points based on the virtual differential allocator; A 3D real-world model of a new energy power station is constructed based on multiple updated point cloud representative points. 2.The unmanned aerial vehicle based new energy station three-dimensional real scene modeling method according to claim 1, characterized in that, The steps for constructing the initial point cloud field based on lidar point cloud and visible light image data collected by UAVs from new energy power stations are as follows: An offline mapping table of drone flight parameters was established based on test flights of the drone. The drone's shooting path is generated based on an offline mapping table. The shooting path includes the flight path, shooting angle, and shooting density. Point cloud data of new energy power stations are collected based on the shooting path. The point cloud data includes lidar point cloud and dense point cloud of images. Among them, the dense point cloud of the image is generated based on visible light image data. The visible light image data is input into the structure-of-motion motion algorithm to generate a sparse point cloud, and the camera pose of each visible light image is calculated to generate the dense point cloud of the image. The three-dimensional space of the new energy power station is divided into a three-dimensional voxel grid. The lidar point cloud and the dense point cloud of the image are spatially aligned and inserted into the corresponding voxels to generate the initial point cloud field. 3.The unmanned aerial vehicle based new energy station three-dimensional real scene modeling method according to claim 1, characterized in that, The steps for constructing three virtual half-axes representing the lidar channel, visible light channel, and position correction vector, and generating the virtual differential allocator, are as follows: The left half-axis is defined as the LiDAR channel, and the left half-axis is used to correlate the local quality score of the LiDAR channel. The local quality score of the lidar channel is generated by multiplying the density score of the lidar point cloud, the inverse score of the plane fitting residual, and the edge intensity score, and then normalizing them. The right half-axis is defined as the visible light channel, and the right half-axis is used to correlate the local quality score of the visible light channel. The visible light channel local quality score is generated by multiplying the texture gradient score of the dense point cloud of the image, the multi-view matching confidence score, and the reciprocal score of the reprojection error, and then normalizing them. The central input axis is defined as the position correction vector. The central input axis is used to correlate the position correction vector, which is the balanced output of the local quality score of the lidar channel and the local quality score of the visible light channel. Define the resistance ratio between the left and right half-shafts. The resistance ratio is the ratio of the resistance value of the left half-shaft to the resistance value of the right half-shaft. The drag value of the left half-axis is 1 minus the local quality score of the lidar channel, and the drag value of the right half-axis is equal to 1 minus the local quality score of the visible light channel. Define an adaptive fusion rule based on the drag ratio. The adaptive fusion rule is to use an optimization function to make the drag ratio meet a preset value.

4. The method for three-dimensional real-scene modeling of new energy power stations based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, The steps for establishing an offline mapping table of UAV flight parameters based on UAV test flights are as follows: Perform test flights and record the test flight parameters during the test flight process. The test flight parameters include flight speed parameters, shooting distance parameters, exposure parameters, and illumination angle parameters. Based on the test flight parameters, lidar channel, and visible light channel, the local data quality score of the lidar channel and the local data quality score of the visible light channel are obtained for each set of test flight parameters. Record the mapping relationship between each set of test flight parameters and the corresponding local quality scores of the lidar channel and the visible light channel, and generate an offline mapping table; The channel quality difference value for each set of test flight parameters is calculated based on the offline mapping table. The channel quality difference value is the difference between the local data quality score of the lidar channel and the local data quality score of the visible light channel. A preset difference threshold is set, and test flight parameters with channel quality difference values ​​below the difference threshold are used as balanced flight parameters. The balanced feature vector of the balanced flight parameters is then extracted. The balanced feature vector includes flight speed feature components, shooting distance feature components, exposure parameter feature components, and illumination angle feature components; Test flight parameters with channel quality differences exceeding the difference threshold are marked as unbalanced parameter groups, and unbalanced feature vectors of unbalanced parameter groups are extracted. Calculate the offset of each component of the imbalance parameter from the mean of each component of the equilibrium eigenvector; Based on the offset of each imbalance parameter component, the values ​​of each characteristic component in the balance parameter group are adjusted in the direction of offsetting the offset to generate an optimized compensation value for the imbalance parameter group. Optimized flight parameters are generated based on the optimized compensation values, and the drone's shooting path is generated based on the balanced flight parameters and the optimized flight parameters.

5. The method for three-dimensional real-scene modeling of new energy power stations based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The steps of configuring a virtual differential allocator for each voxel unit in the initial point cloud field, and generating multiple updated point cloud representative points based on the virtual differential allocator, are as follows: For each voxel, the geometric median of the lidar point cloud and the geometric median of the dense point cloud of the image are calculated respectively. The average of the spatial coordinates of the two geometric medians is taken as the initial representative point of the point cloud of the voxel. Based on the virtual differential allocator, the local quality score of the lidar channel and the local quality score of the visible light channel within the current voxel unit are calculated respectively; Simultaneously, the drag ratio is calculated, and a position correction vector is generated based on the optimized function to balance the drag ratio. When the local quality score of the lidar channel or the local quality score of the visible light channel is unbalanced, the virtual differential lock is activated to lock the corresponding channel. The position correction vector is superimposed onto the initial point cloud representative points to generate the updated point cloud representative.

6. The method for three-dimensional real-scene modeling of new energy power stations based on unmanned aerial vehicles (UAVs) according to claim 5, characterized in that, When the local quality score of the lidar channel or the local quality score of the visible light channel is unbalanced, the virtual differential lock is activated to lock the corresponding channel. The steps are as follows: When the local quality score of the lidar channel or the local quality score of the visible light channel is continuously lower than the preset score threshold in a preset number of voxel units; Then, the virtual differential lock is activated, the quality score of the corresponding half-axis that is continuously below the scoring threshold is forcibly set to zero, the half-axis is locked, and the position is corrected based on the other half-axis to generate a position correction vector.

7. The method for three-dimensional real-scene modeling of new energy power stations based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The steps for constructing a 3D reality model of a new energy power station based on multiple updated point cloud representative points are as follows: All updated point cloud representative points are merged according to their respective spatial coordinates to generate a fused point cloud; A Poisson surface reconstruction algorithm is used, with fused point cloud as input, to construct a triangular mesh model; The original visible light image is projected onto a triangular mesh model, and weighted fusion is performed based on the visibility and incident angle of each viewpoint to generate a three-dimensional real-scene model of the new energy power station.

8. A UAV-based 3D real-scene modeling system for new energy power stations, used to implement the UAV-based 3D real-scene modeling method for new energy power stations as described in any one of claims 1-7, characterized in that, include: Data acquisition module: Based on the acquisition of lidar point cloud and visible light image data of new energy power stations by UAV, an initial point cloud field is constructed; Differential Allocator Module: Connected to the data acquisition module, it constructs three virtual half-axes representing the lidar channel, visible light channel, and position correction vector, and generates a virtual differential allocator; Point cloud update module: Connected to the differential allocator module, it configures a virtual differential allocator for each voxel unit in the initial point cloud field, and generates multiple representative points for updating the point cloud based on the virtual differential allocator; Model building module: Connected to the point cloud update module, it builds a 3D real-world model of the new energy power station based on multiple updated point cloud representative points.

Images