Cloud-edge coordinated point cloud model lightweight rendering processing method and system

Abstract

The invention relates to the technical field of lightweight rendering, in particular to a cloud-edge coordinated point cloud model lightweight rendering processing method and system. The method comprises the following steps: performing point cloud coordination prior processing according to electric power facility point cloud data to generate coordination prior point cloud data; a point cloud rendering model is constructed, point cloud curved surface coefficient calculation and curved surface reconstruction processing are carried out, and a point cloud curved surface reconstruction rendering model is generated; obtaining image view angle data, performing rendering model pose alignment processing and model texture enhancement mapping processing, and generating an enhanced texture rendering model; performing rendering coordination lightweight deployment and lightweight real-time rendering processing on the enhanced texture rendering model to generate lightweight real-time rendering data; and obtaining user behavior data, performing point cloud data rendering feedback analysis and cloud edge coordination caching in combination with the lightweight real-time rendering data, and feeding back to the terminal. According to the method, rendering of point cloud power equipment data curved surface reconstruction can be lightened, and redundant transmission of cloud edge rendering data is reduced and coordinated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of lightweight rendering, in particular to a cloud-edge coordinated point cloud model lightweight rendering processing method and system. BACKGROUND

[0002] In the field of digital operation and maintenance of power facilities, efficient rendering and lightweight processing of point cloud models are key to real-time monitoring and fault diagnosis.

[0003] In the existing related technology, due to the characteristics of large data volume and complex structure of point cloud data, the point cloud data rendering processing method adopts global downsampling or mesh simplification strategy, and often faces the problem of insufficient surface reconstruction accuracy in the rendering process, without fully considering the curvature change and spatial level relationship of point cloud power equipment, which leads to distortion of point cloud power equipment in lightweight rendering of complex curved surfaces (such as arc surface structure of power equipment), affecting the authenticity of model rendering. Secondly, the cloud-edge coordination efficiency is low, and the existing scheme lacks dynamic optimization of rendering computing power and transmission granularity, making it difficult to quickly and real-time coordinate and allocate cloud-edge resources according to user behavior, which easily leads to redundant transmission of cloud-edge coordination data after lightweight rendering of point cloud power equipment data.

[0004] In view of the above related technologies, there are problems of insufficient surface reconstruction accuracy in lightweight processing of point cloud power equipment data, and easy redundant transmission of cloud-edge coordination after lightweight rendering of point cloud power equipment data. SUMMARY

[0005] Therefore, the present application provides a cloud-edge coordinated point cloud model lightweight rendering processing method to solve at least one of the above technical problems.

[0006] To achieve the above purpose, a cloud-edge coordinated point cloud model lightweight rendering processing method comprises the following steps:

[0007] Step S1: Obtain power facility point cloud data, and perform point cloud coordination prior processing according to the power facility point cloud data to generate coordinated prior point cloud data;

[0008] Step S2: Construct a point cloud rendering model according to the coordinated prior point cloud data; calculate point cloud surface coefficients according to the coordinated prior point cloud data to generate point cloud surface coefficients; perform surface reconstruction processing on the point cloud rendering model based on the point cloud surface coefficients to generate a point cloud surface reconstruction rendering model;

[0009] Step S3: Obtain image perspective data corresponding to the power facility, and perform rendering model pose alignment processing based on the image perspective data, the point cloud rendering model and the point cloud surface reconstruction rendering model to generate a fusion pose alignment model; perform model texture enhancement mapping processing according to the fusion pose alignment model to generate an enhanced texture rendering model;

[0010] Step S4: rendering and coordinating the enhanced texture rendering model based on the coordinated prior point cloud data to generate a lightweight point cloud rendering model;

[0011] Step S5: performing lightweight real-time rendering processing on the coordinated prior point cloud data based on the lightweight point cloud rendering model to generate lightweight real-time rendering data; obtaining user behavior data; performing point cloud data rendering feedback analysis according to the user behavior data to generate rendering feedback data; performing cloud-edge rendering coordination transmission granularity optimization according to the rendering feedback data to generate cloud-edge rendering coordination transmission granularity data; and performing cloud-edge coordination caching on the lightweight real-time rendering data based on the cloud-edge rendering coordination transmission granularity data and feeding back to the terminal.

[0012] Further, step S1 includes the following steps:

[0013] Step S11: obtaining power facility point cloud data;

[0014] Step S12: performing clustering processing on the power facility point cloud data based on a preset power point cloud facility semantic clustering algorithm to generate clustered point cloud data;

[0015] Step S13: performing time sequence node verification alignment processing on the clustered point cloud data to generate time sequence verification point cloud data;

[0016] Step S14: performing regional point cloud overlap analysis according to the time sequence verification point cloud data to generate regional overlap point cloud data;

[0017] Step S15: performing redundant power point cloud facility filtering processing on the clustered point cloud data based on the regional overlap point cloud data to generate filtered power point cloud facility data;

[0018] Step S16: performing point cloud coordination prior processing based on the time sequence verification point cloud data and the filtered power point cloud facility data to generate coordinated prior point cloud data.

[0019] Further, step S2 includes the following steps:

[0020] Step S21: performing point cloud data layering processing according to the coordinated prior point cloud data to generate layered point cloud prior data;

[0021] Step S22: performing semantic structure segmentation vector matrix calculation on the layered point cloud prior data to generate structure segmentation vector point cloud data;

[0022] Step S23: performing point cloud gridding processing on the layered point cloud prior data based on the structure segmentation vector point cloud data to generate layered grid point cloud data;

[0023] Step S24: constructing a point cloud rendering model based on the layered point cloud prior data and the layered grid point cloud data;

[0024] Step S25: Perform point cloud surface coefficient calculation on the structure segmentation vector point cloud data according to the structure segmentation vector point cloud data, and generate point cloud surface coefficient;

[0025] Step S26: Construct a point cloud surface reconstruction rendering model based on the point cloud surface coefficient and the point cloud rendering model.

[0026] Further, step S25 includes the following steps:

[0027] Perform associated vector point cloud layer curvature analysis on the structure segmentation vector point cloud data, and generate curvature change vector data;

[0028] Perform point cloud spatial level aggregation processing according to the curvature change vector data, and generate aggregated curvature point cloud layer data;

[0029] Perform three-dimensional surface segmentation processing according to the aggregated curvature point cloud layer data, and generate surface segmentation point cloud layer data;

[0030] Perform surface fitting coefficient calculation according to the surface segmentation point cloud layer data, and generate point cloud surface coefficient.

[0031] Further, step S26 includes the following steps:

[0032] Perform angle window sliding simulation on the point cloud rendering model based on the point cloud surface coefficient, and generate point cloud curvature sliding window data;

[0033] Perform spatial gradient change analysis according to the point cloud curvature sliding window data, and generate curvature spatial gradient change data;

[0034] Perform local geometric warping distortion detection according to the curvature spatial gradient change data, and generate spatial surface warping data;

[0035] Perform warping layered detail smoothing correction processing on the spatial surface warping data based on the point cloud surface coefficient, and generate surface smoothing correction data;

[0036] Perform surface reconstruction processing on the point cloud rendering model based on the surface smoothing correction data, and generate a point cloud surface reconstruction rendering model.

[0037] Further, step S3 includes the following steps:

[0038] Step S31: Obtain image view angle data corresponding to the power facility;

[0039] Step S32: Perform initial pose standard processing according to the image view angle data and the point cloud rendering model, and generate initial pose alignment data;

[0040] Step S33: Joint rendering model pose alignment processing is performed on the initial pose alignment data and the point cloud surface reconstruction rendering model to generate a fusion pose alignment model;

[0041] Step S34: Multi-frame rendering point cloud space reconstruction alignment fitting processing is performed on the fusion pose alignment model to generate a multi-frame rendering space alignment point cloud model;

[0042] Step S35: Image texture mapping region analysis is performed based on the multi-frame rendering space alignment point cloud model to generate image texture mapping region data;

[0043] Step S36: Model texture enhancement mapping processing is performed on the multi-frame rendering space alignment point cloud model based on the image texture mapping region data to generate an enhanced texture rendering model.

[0044] Further, step S36 includes the following steps:

[0045] Texture feature detail extraction is performed based on the image texture mapping region data and the image view angle data to generate texture feature detail data;

[0046] Image texture space mapping enhancement processing is performed on the texture feature detail data to generate enhanced texture mapping data;

[0047] Model texture enhancement mapping processing is performed on the multi-frame rendering space alignment point cloud model based on the enhanced texture mapping data to generate an enhanced texture rendering model.

[0048] Further, step S4 includes the following steps:

[0049] Step S41: Coordination cloud edge analysis is performed according to the coordination prior point cloud data to generate coordination cloud edge data;

[0050] Step S42: Rendering computing power thread estimation calculation is performed according to the coordination cloud edge data to generate rendering computing power thread data;

[0051] Step S43: Rendering coordination lightweight deployment is performed based on the rendering computing power thread data and the enhanced texture rendering model to generate a lightweight point cloud rendering model.

[0052] Further, step S5 includes the following steps:

[0053] Step S51: Rendering interaction progressive level analysis is performed on the coordination prior point cloud data to generate rendering interaction progressive level data;

[0054] Step S52: Rendering resource allocation calculation is performed on the coordination prior point cloud data according to the rendering interaction progressive level data to generate rendering resource data;

[0055] Step S53: based on the rendering interaction progressive level data and the estimated rendering resource data, point cloud data progressive rendering scheduling is performed to generate point cloud progressive rendering scheduling data;

[0056] Step S54: based on the lightweight point cloud rendering model, the point cloud progressive rendering scheduling data is subjected to lightweight real-time rendering processing to generate lightweight real-time rendering data;

[0057] Step S55: user behavior data is acquired;

[0058] Step S56: according to the user behavior data, point cloud data rendering feedback analysis is performed to generate rendering feedback data;

[0059] Step S57: according to the rendering feedback data, cloud-edge rendering coordination transmission granularity optimization is performed to generate cloud-edge rendering coordination transmission granularity data;

[0060] Step S58: based on the cloud-edge rendering coordination transmission granularity data, the lightweight real-time rendering data is subjected to cloud-edge coordination caching and feedback to the terminal.

[0061] Further, the application also provides a cloud-edge coordinated point cloud model lightweight rendering processing system for executing the cloud-edge coordinated point cloud model lightweight rendering processing method as described above, and the cloud-edge coordinated point cloud model lightweight rendering processing system comprises:

[0062] A point cloud data coordination preprocessing module is configured to acquire power facility point cloud data and perform point cloud coordination prior processing according to the power facility point cloud data to generate coordinated prior point cloud data.

[0063] A rendering model construction module is configured to construct a point cloud rendering model according to the coordinated prior point cloud data, perform point cloud surface coefficient calculation according to the coordinated prior point cloud data to generate point cloud surface coefficients, and perform surface reconstruction processing on the point cloud rendering model based on the point cloud surface coefficients to generate a point cloud surface reconstruction rendering model.

[0064] A rendering model enhancement module is configured to acquire image perspective data corresponding to the power facility, perform rendering model pose alignment processing based on the image perspective data, the point cloud rendering model and the point cloud surface reconstruction rendering model to generate a fusion pose alignment model, and perform model texture enhancement mapping processing according to the fusion pose alignment model to generate an enhanced texture rendering model.

[0065] A rendering model deployment module is configured to perform rendering coordination lightweight deployment on the enhanced texture rendering model based on the coordinated prior point cloud data to generate a lightweight point cloud rendering model.

[0066] The lightweight rendering and transmission module performs lightweight real-time rendering processing on the coordinated prior point cloud data based on a lightweight point cloud rendering model to generate lightweight real-time rendering data; acquires user behavior data; performs point cloud data rendering feedback analysis according to the user behavior data to generate rendering feedback data; performs cloud-edge rendering coordination transmission granularity optimization according to the rendering feedback data to generate cloud-edge rendering coordination transmission granularity data; and performs cloud-edge coordination caching on the lightweight real-time rendering data based on the cloud-edge rendering coordination transmission granularity data and feeds back to the terminal.

[0067] The beneficial effects of the present application are:

[0068] 1、The cloud edge coordination point cloud model lightweight rendering processing method provided by the application has the beneficial effects that, compared with the prior art, the power facility point cloud data is acquired, and point cloud coordination prior processing is performed according to the power facility point cloud data, the core advantage of this step is that the noise points in the power equipment point cloud data can be effectively removed, and these interference data can be avoided to mislead the subsequent lightweight rendering model construction and analysis. At the same time, the redundant data is simplified, the data amount is reduced, the calculation burden and storage cost are reduced on the premise of not losing key information. In addition, the prior processing can also adjust the spatial distribution of the data based on the spatial structure and physical characteristics of the power facility, so that the point cloud data is more uniform and orderly coordinated, and the point cloud rendering model in the subsequent step can more accurately reflect the actual form of the power facility, and provide more reliable data support in the application scenarios of power facility inspection, fault diagnosis and maintenance planning, which helps to improve the intelligent and accurate level of power system operation and maintenance. The point cloud rendering model is constructed according to the coordinated prior point cloud data, the processed point cloud data can be presented in the form of a visual three-dimensional model, and an intuitive power facility spatial form display is provided for users. The point cloud surface coefficient is calculated according to the coordinated prior point cloud data, the point cloud surface coefficient is calculated, the geometric characteristics and curvature change of the surface of the power facility can be analyzed in depth, and important basis is provided for surface reconstruction. The point cloud surface reconstruction rendering model is generated by performing surface reconstruction processing on the point cloud rendering model based on the point cloud surface coefficient, which can accurately model and optimize the complex surface form of the power facility, especially the parts with special structure or surface. It is helpful for power engineers and maintenance personnel to more accurately observe and analyze the structural characteristics of the power facility, effectively improve the work efficiency and decision accuracy in the process of power facility design, construction and operation. The image perspective data corresponding to the power facility is acquired, and the rendering model pose alignment processing is performed based on the image perspective data, the point cloud rendering model and the point cloud surface reconstruction rendering model, and the fusion pose alignment model is generated. The point cloud rendering model and the point cloud surface reconstruction rendering model are combined to perform rendering model pose alignment processing, which can accurately match and fuse different sources and types of data in space. The image perspective data contains rich texture and color information, and the point cloud model focuses on spatial geometric structure. Through pose alignment, the two are accurately corresponding in three-dimensional space, and the generated fusion pose alignment model has accurate geometric form and real texture details. The enhanced texture rendering model is generated by performing model texture enhancement mapping processing according to the fusion pose alignment model, which further optimizes and enhances the texture information, and can highlight the subtle features of the power facility surface. In the application scenarios of remote monitoring and virtual inspection of power facilities, maintenance personnel can more clearly observe the equipment state through the enhanced texture rendering model, timely discover potential faults and abnormalities, reduce the workload and cost of on-site inspection, and improve the efficiency and safety of power system operation and maintenance.Based on the coordinated prior point cloud data, the enhanced texture rendering model is rendered in a coordinated and lightweight manner to generate a lightweight point cloud rendering model. While ensuring that the model's key information and visualization effects are basically unaffected, the model's data volume is significantly reduced. The generated lightweight point cloud rendering model reduces the requirements for network bandwidth and terminal device performance, enabling the model to run smoothly on a variety of devices, improving the applicability and scalability of the rendering model. Whether it is on-site viewing of power facilities on mobile devices or centralized management and analysis of large-scale power facility models in the cloud, the lightweight point cloud rendering model can effectively improve the operating efficiency of the system, reduce resource consumption, and provide more convenient and efficient technical support for the digital management and operation and maintenance of power facilities. Based on the lightweight point cloud rendering model, lightweight real-time rendering processing of coordinated prior point cloud data can quickly generate high-quality real-time rendering images to meet users' needs for dynamic visualization of power facilities. By capturing user behavior data and analyzing point cloud rendering feedback, we can gain a deeper understanding of user behavior habits and focus points when operating and observing rendered models. Based on this point cloud rendering feedback, we optimize the granularity of cloud-edge rendering coordination transmission. This allows us to rationally adjust the data transmission strategy during cloud-edge collaborative rendering, improve transmission accuracy and frequency, and ensure that relevant details are presented clearly and smoothly. Based on cloud-edge rendering coordination transmission granularity data, we perform cloud-edge coordinated caching of lightweight real-time rendering data and feedback it to the terminal, effectively balancing the utilization of cloud-edge computing resources and improving both the transmission efficiency of rendering data and the terminal display quality.

[0069] 2. The cloud-edge coordinated point cloud model lightweight rendering processing method system proposed in the present invention is composed of a point cloud data coordination preprocessing module, a rendering model construction module, a rendering model enhancement module, a rendering model deployment module and a lightweight rendering and transmission module. It can realize any cloud-edge coordinated point cloud model lightweight rendering processing method described in the present invention, and is used to combine the cloud-edge coordinated point cloud model lightweight rendering processing method between computer programs running on each module. The internal structure of the system cooperates with each other, which can greatly reduce repetitive work and manpower investment, and can quickly and effectively provide more accurate and efficient rendering of lightweight point cloud power equipment data, and reduce the redundant transmission of cloud-edge coordinated data after lightweight rendering of point cloud power equipment data. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 Schematic diagram of the steps of the cloud-edge coordinated point cloud model lightweight rendering processing method of the present invention;

[0071] Figure 2 for Figure 1 Detailed implementation steps of step S1 in FIG.

[0072] Figure 3 for Figure 1The detailed implementation procedure of step S2 is shown in a flowchart.

[0073] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0074] The technical method of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0075] In addition, the accompanying drawings are only schematic illustrations of the present application, and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated description thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0076] It should be understood that although the terms "first", "second" and the like can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the exemplary embodiments, a first element can be referred to as a second element, and similarly a second element can be referred to as a first element. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0077] To achieve the above-mentioned purpose, please refer to Figures 1 to 3 The present application provides a cloud edge coordination point cloud model lightweight rendering processing method, comprising the following steps:

[0078] Step S1: Obtain power facility point cloud data, and perform point cloud coordination prior processing according to the power facility point cloud data to generate coordinated prior point cloud data;

[0079] Step S2: Construct a point cloud rendering model according to the coordinated prior point cloud data; calculate point cloud surface coefficients according to the coordinated prior point cloud data to generate point cloud surface coefficients; and perform surface reconstruction processing on the point cloud rendering model based on the point cloud surface coefficients to generate a point cloud surface reconstruction rendering model;

[0080] Step S3: Obtain image view angle data corresponding to the power facility, and perform rendering model pose alignment processing based on the image view angle data, the point cloud rendering model, and the point cloud surface reconstruction rendering model to generate a fusion pose alignment model; perform model texture enhancement mapping processing according to the fusion pose alignment model to generate an enhanced texture rendering model;

[0081] Step S4: Perform rendering coordination lightweight deployment on the enhanced texture rendering model based on the coordinated prior point cloud data to generate a lightweight point cloud rendering model;

[0082] Step S5: Perform lightweight real-time rendering processing on the coordinated prior point cloud data based on the lightweight point cloud rendering model to generate lightweight real-time rendering data; obtain user behavior data; perform point cloud data rendering feedback analysis according to the user behavior data to generate rendering feedback data; perform cloud edge rendering coordination transmission granularity optimization according to the rendering feedback data to generate cloud edge rendering coordination transmission granularity data; perform cloud edge coordination caching on the lightweight real-time rendering data based on the cloud edge rendering coordination transmission granularity data and feed back to the terminal.

[0083] In the embodiment of the application, please refer to Figure 1 The cloud edge coordinated point cloud model lightweight rendering processing method includes the following steps:

[0084] Step S1: Obtain power facility point cloud data, and perform point cloud coordination prior processing according to the power facility point cloud data to generate coordinated prior point cloud data;

[0085] In an embodiment of the present invention, a three-dimensional laser scanner is used to perform an all-round scan of the power facilities at a scanning speed of 1 million points per second to obtain raw point cloud data. During the scanning process, the scanner samples at an angular resolution of 0.008 degrees in the horizontal direction and 0.016 degrees in the vertical direction to ensure the spatial density of the data. The raw point cloud data contains spatial coordinates (x, y, z) and reflection intensity I information. The raw point cloud data is denoised using a statistical filtering algorithm. The neighborhood radius r of each point is set to 0.2m, and the average distance μ and standard deviation σ of the points in the neighborhood are calculated. If the average distance d from a point to a point in the neighborhood satisfies d>μ+3σ, the point is determined to be a noise point and deleted. The voxel grid method is used to simplify redundant data, dividing the point cloud space into a cubic grid with a side length of 0.1m. Only the point at the center of gravity is retained as a representative in each grid, so that the data volume is compressed. Based on the design drawings of the power facilities, a spatial structure constraint model is constructed. For power towers, we divide them into vertical zones at 1-meter intervals based on their height and crossarm distribution. If the point cloud density in a particular zone is lower than the average, we use an interpolation algorithm to supplement the point cloud data. This ultimately generates coordinated prior point cloud data, which reduces the data size while ensuring the integrity of key structural information.

[0086] Step S2: constructing a point cloud rendering model based on the coordinated prior point cloud data; calculating the point cloud surface coefficients based on the coordinated prior point cloud data to generate the point cloud surface coefficients; performing surface reconstruction processing on the point cloud rendering model based on the point cloud surface coefficients to generate a point cloud surface reconstructed rendering model;

[0087] In the embodiment of the present invention, an octree structure is used to organize the coordinated prior point cloud data. The root node covers the entire point cloud space and is recursively divided into 8 child nodes. When the number of points in a node is less than 100 or the node side length is less than 0.05m, the division is stopped and a point cloud rendering model is constructed. When calculating the point cloud surface coefficient, the moving least squares method (MLS) is used. For each point p i , within its radius (r = 0.3m), by minimizing (where f(p j ) is the fitting function, z j For point p j The z coordinate of N(p i ) is point p i neighborhood), fitting the local quadratic surface z=ax 2 +bxy+cy 2dx+ey+f, and further calculate the Gaussian curvature K and the mean curvature H of the point as the surface coefficients. Based on the calculated surface coefficients, the surface reconstruction is performed on the area with large curvature change (such as the bending part of the insulator string and the edge of the transformer cooling fin). The Poisson surface reconstruction algorithm is adopted, the reconstruction depth is set to 8, the point cloud data is converted into an implicit function, a high-precision triangular mesh model is generated, and compared with the original point cloud rendering model, the surface details are improved.

[0088] Step S3: Obtain the image view angle data corresponding to the power facility, and perform rendering model pose alignment processing based on the image view angle data, the point cloud rendering model and the point cloud surface reconstruction rendering model to generate a fusion pose alignment model; perform model texture enhancement mapping processing on the fusion pose alignment model to generate an enhanced texture rendering model;

[0089] In the embodiment of the present application, a drone equipped with a 2000 million pixel camera is used to take pictures around the power facility at a flight height of 50 m and a flight speed of 15 m / s, image data with a resolution of 4000x3000 is obtained, the shooting interval is 2 seconds, and a total of 200 images are obtained. The scale-invariant feature transform (SIFT) algorithm is used to extract feature points in the images, and an average of 1000 feature points are extracted from each image. At the same time, the feature points are extracted from the point cloud rendering model and the point cloud surface reconstruction rendering model by the voxel grid method with a grid length of 0.2 m, and about 5000 feature points are extracted from each model. The random sample consensus (RANSAC) algorithm is used to calculate the rotation matrix R and the translation vector T between the image and the point cloud model by matching the spatial position relationship of the feature point pairs, to realize the pose alignment of the rendering model, and the error is controlled within 0.05 m. The model texture enhancement mapping adopts the physically based rendering (PBR) technology. After the image is subjected to light correction and color balance processing, it is mapped onto the fusion pose alignment model. For the equipment nameplate area, the texture resolution is increased from 100x50 to 200x100 by using the bilinear interpolation algorithm. For the insulator surface, the local histogram equalization algorithm is used to enhance the crack contrast, and an enhanced texture rendering model is generated, and the texture clarity is improved.

[0090] Step S4: Perform rendering coordination and lightweight deployment on the enhanced texture rendering model based on the coordinated prior point cloud data to generate a lightweight point cloud rendering model;

[0091] In the embodiment of the present application, the edge folding simplification algorithm is used to perform lightweight processing on the enhanced texture rendering model. The cost C of folding each edge is calculated edge , the formula is (wherein f is a set of faces adjacent to the edge, ΔA fThe area change amount of the folding edge rear face f is preferentially folded to minimize the folding cost. The original number of model faces is reserved by setting a simplification threshold, and the number of model faces is reduced after processing. Combined with the level of detail (LOD) technology, the model is divided into three levels: LOD0 is the original simplified 320,000 face model, LOD1 is the model further simplified to 160,000 faces, and LOD2 is the 8,000 face model. According to the screen resolution and computing performance of the terminal device, when the device resolution is lower than 1920*1080 and the GPU computing power is lower than 3TFLOPS, the LOD2 model is automatically switched to generate a lightweight point cloud rendering model, so that the original data amount is compressed.

[0092] Step S5: based on the lightweight point cloud rendering model, the coordinated prior point cloud data is processed for lightweight real-time rendering, to generate lightweight real-time rendering data; user behavior data is obtained; point cloud data rendering feedback analysis is performed according to the user behavior data, to generate rendering feedback data; cloud edge rendering coordination transmission granularity optimization is performed according to the rendering feedback data, to generate cloud edge rendering coordination transmission granularity data; and cloud edge coordination caching is performed on the lightweight real-time rendering data based on the cloud edge rendering coordination transmission granularity data, and the data is fed back to the terminal.

[0093] In the embodiment of the application, the WebGL technology is used to perform real-time rendering on the lightweight point cloud rendering model at a frame rate of 60 frames / second. For the coordinated prior point cloud data, the octree view frustum culling algorithm is adopted to render only the point cloud data within the camera view frustum range, thereby reducing invalid rendering calculation. Through the event listener embedded in the rendering interface, the user's mouse click, drag, zoom and other behavior data are obtained in real time. For example, the screen coordinates (x model ,y model ) of the mouse click position are recorded, which are converted into model space coordinates (x model ,y model ,z model ) through a projection matrix, and the number of clicks and the dwell time of the user in different areas are counted. Based on the user behavior data, the density-based clustering algorithm (DBSCAN) is adopted to identify the user's key attention area by setting the neighborhood radius r=0.5m and the minimum number of points to 10. For the key area, the compression ratio of the cloud edge transmission data is reduced from 10:1 to 5:1 to improve the transmission accuracy, and for the non-key area, the compression ratio is increased to 15:1. The Redis cache database is used to establish a bidirectional caching mechanism in the cloud and the edge, and the cache strategy is dynamically adjusted according to the transmission granularity data, so as to reduce the terminal rendering delay and realize efficient cloud edge coordination rendering.

[0094] Further, step S1 includes the following steps:

[0095] Step S11: obtaining power facility point cloud data;

[0096] Step S12: clustering the power facility point cloud data based on a preset power point cloud facility semantic clustering algorithm to generate clustered point cloud data;

[0097] Step S13: performing time series node verification and alignment processing on the clustered point cloud data to generate time series verification point cloud data;

[0098] Step S14: performing regional point cloud overlap analysis based on the time series verification point cloud data to generate regional overlapping point cloud data;

[0099] Step S15: performing redundant power point cloud facility filtering processing on the clustered point cloud data based on the regional overlapping point cloud data to generate filtered power point cloud facility data;

[0100] Step S16: performing point cloud coordination priori processing based on the time series verification point cloud data and filtering out the power point cloud facility data to generate coordinated priori point cloud data.

[0101] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Detailed step flow diagram of step S1 in FIG. 1 , in this embodiment, step S1 includes the following steps:

[0102] Step S11: Acquire power facility point cloud data;

[0103] In an embodiment of the present invention, a Leica P50 three-dimensional laser scanner is used to collect data on power facilities. The scanner scanning speed is set to 1 million points / second, the horizontal angular resolution is set to 0.008 degrees, and the vertical angular resolution is set to 0.016 degrees. The scanning operation is performed in a spiral path at a distance of 25 meters from the transmission line tower and 20 meters outside the main transformer cluster of the substation. During scanning, the scanner records the three-dimensional spatial coordinates (x, y, z), reflection intensity I and timestamp t of the scanning time of each point in real time. For example, a 220kV substation is scanned, and a single scan obtains a 15GB raw data file containing approximately 1.5 billion point cloud data points, which completely covers the point cloud information of power facilities such as transformers, circuit breakers and insulator strings.

[0104] Step S12: clustering the power facility point cloud data based on a preset power point cloud facility semantic clustering algorithm to generate clustered point cloud data;

[0105] In the embodiment of the present invention, the power facility point cloud data is clustered based on the improved Mean-Shift algorithm. The bandwidth parameter h is set to 0.12m. For each point cloud data point p i (x i ,y i ,z i ), with p iFor the center, h is the radius of the spherical neighborhood, the centroid C of all points in the neighborhood is calculated i , the formula is Where K is the kernel function, here the Gaussian kernel function is adopted ||p i ―p j || represents the Euclidean distance between p i and the point p j in the neighborhood. Move the point p i to the centroid C i position, repeat this process until the centroid position converges. The points in the same centroid neighborhood after convergence are classified into the same cluster. In the point cloud data processing of a certain power transmission line, the point cloud data of the tower, insulator and conductor and other facilities are accurately divided into different clusters by the algorithm, and finally 1.5 billion point cloud data points are divided into 60 clusters, each cluster corresponds to a specific power facility component, and the preliminary structured classification of the point cloud data is realized.

[0106] Step S13: performing time sequence node checking alignment processing on the clustered point cloud data to generate time sequence checking point cloud data;

[0107] In the embodiment of the present application, when performing time sequence node checking alignment processing on the clustered point cloud data, the point cloud data in each cluster is arranged in time sequence according to time sequence. The iterative closest point (ICP) algorithm is used to calculate the alignment relationship of adjacent time sequence point cloud data. For two time sequence point cloud sets A and B, first find the closest point in B for each point of A to construct a corresponding point pair. Then, by minimizing the mean square error between the corresponding point pairs (where a i ∈A, b i ∈B, n is the number of corresponding point pairs), the rotation matrix R and the translation vector T are calculated to align A with B through the transformation A' = RA + T. In the point cloud data processing of a certain power facility monthly inspection, the same tower point cloud data collected at different times is processed, the misaligned point cloud data caused by the difference in scanning position and angle is accurately aligned through 10 times of ICP algorithm, the time sequence checking point cloud data is generated, and the consistency of the facility point cloud data in the time dimension is ensured.

[0108] Step S14: performing regional point cloud overlap analysis according to the time sequence checking point cloud data to generate regional overlap point cloud data;

[0109] In the embodiment of the present application, the hierarchical oriented bounding box (OBB) algorithm is used for regional point cloud overlap analysis according to the time sequence checking point cloud data. First, the directional bounding box of the point cloud data of each cluster is constructed to determine the center coordinates O, size vector E and rotation matrix R of the bounding box. For any two clusters c i and cj whether the two OBBs overlap or not. Specifically, the projection intervals of the two OBBs on 15 axes (including 3 main axes of each OBB, 9 axes obtained by the cross product of each pair of main axes, and 3 axes obtained by the cross product of the main axes of the two OBBs) are calculated. If there is overlap in the projection intervals on all the 15 axes, the overlap volume V overlap is calculated. The OBB algorithm is used to accurately identify the overlapping region caused by the overlapping scanning ranges of two adjacent groups of disconnectors in a substation. When the overlap volume exceeds 15% of the volume of a single cluster OBB, it is determined that there is an area overlap, and the area overlap point cloud data is generated, which provides a basis for subsequent redundant data processing.

[0110] Step S15: performing redundant power point cloud facility filtering processing on the clustered point cloud data based on the area overlap point cloud data to generate filtered power point cloud facility data.

[0111] In the embodiment of the present application, the redundant power point cloud facility filtering processing is performed on the clustered point cloud data based on the area overlap point cloud data. For the clusters with overlap, the local density p of the point cloud data in the overlapping region is calculated, and the formula is where N is the number of point clouds in the overlapping region, and V is the volume of the overlapping region. The density threshold p threshold = 800m 3 is set. If the density of a cluster in the overlapping region is greater than p threshold , the point cloud data of the cluster in the overlapping region is retained, otherwise it is deleted. In the processing of multiple scanning data of a power transmission line, for the overlapping region between towers caused by scanning connection, the density is calculated and compared with the threshold, which effectively removes the redundant point cloud data caused by repeated scanning, reduces the overall data volume by about 18%, and ensures the complete retention of point cloud information of key structures of the towers, thereby generating filtered power point cloud facility data.

[0112] Step S16: performing point cloud coordination prior processing based on the time sequence check point cloud data and the filtered power point cloud facility data to generate coordinated prior point cloud data.

[0113] In the embodiment of the present application, the point cloud coordination prior processing is performed based on the time sequence check point cloud data and the filtered power point cloud facility data. First, the median filter algorithm is used for denoising. For each point p iIn a spherical neighborhood with a radius r=0.18 m, the coordinate values of the points are sorted by size, and the middle value is taken as the new coordinate of the point, so as to eliminate isolated noise points. Then, the voxel grid downsampling method is used for data reduction, and the point cloud space is divided into a cubic grid with a side length of 0.08 m, and only the point closest to the center of each grid is retained. Finally, based on the three-dimensional model constructed based on the power facility design drawing, the point cloud in the sparse area is supplemented. By using the moving least square method (MLS), for the to-be-supplemented point q, the local surface is fitted by minimizing (where w j (q) is a weight function, z j is the z coordinate of the point (p j in the neighborhood, and f(p j is a fitting function) fitting the local surface, and the coordinates of the supplemented point are calculated. After the above processing, the original 15GB point cloud data is finally generated into 5GB coordinated prior point cloud data, and the data quality is significantly improved, providing a high-quality data basis for subsequent point cloud rendering and analysis.

[0114] Further, step S2 includes the following steps:

[0115] Step S21: performing point cloud data layering processing according to the coordinated prior point cloud data to generate layered point cloud prior data;

[0116] Step S22: performing semantic structure segmentation vector matrix calculation on the layered point cloud prior data to generate structure segmentation vector point cloud data;

[0117] Step S23: performing point cloud gridding processing on the layered point cloud prior data based on the structure segmentation vector point cloud data to generate layered grid point cloud data;

[0118] Step S24: constructing a point cloud rendering model based on the layered point cloud prior data and the layered grid point cloud data;

[0119] Step S25: performing point cloud surface coefficient calculation according to the structure segmentation vector point cloud data to generate point cloud surface coefficients;

[0120] Step S26: constructing a point cloud surface reconstruction rendering model based on the point cloud surface coefficients and the point cloud rendering model.

[0121] As an embodiment of the present application, referring to FIG. 2, which is a detailed step flowchart of step S2 in the embodiment, step S2 in the embodiment includes the following steps: Figure 3 Figure 1 Step S21: performing point cloud data layering processing according to the coordinated prior point cloud data to generate layered point cloud prior data;

[0122] Step S21: performing point cloud data layering processing according to the coordinated prior point cloud data to generate layered point cloud prior data;

[0123] ​In the embodiment of the present invention, point cloud layering processing is performed based on the coordinated prior point cloud data. First, the elevation value h of each point is calculated. i =z i ―min(z)(where z i For point p i The z coordinate of the point cloud data, min(z) is the minimum z coordinate value in the point cloud data, and z is the height position of each point on the power facility relative to a certain reference plane. According to the structural characteristics of the power facility, the elevation stratification threshold is set, such as the elevation value of the ground layer does not exceed 0.5m, the elevation value of the foundation layer is between 0.5m-3m, the elevation value of the equipment layer is between 3m-15m, and the elevation value of the overhead layer is not less than 15m. For the ground layer, the statistical outlier removal algorithm (SOR) is used, and the number of neighborhood points k=20 and the standard deviation multiple t=2.5 are set to remove outliers. The base layer is filtered by voxel grid, and the voxel size is set to 0.05m to retain geometric features. The equipment layer uses radius filtering with a radius r=0.1m and a point threshold n=10 to retain the point cloud of key equipment. The overhead layer is filtered by calculating the curvature C of the point i (Using the PCA method with a neighborhood radius of r = 0.3m), points with a curvature greater than 0.05 are identified as conductor points. After layering, the original coordinated prior point cloud data (5GB) is divided into a ground layer (0.8GB), a foundation layer (1.2GB), an equipment layer (2.5GB), and an overhead layer (0.5GB). This generates layered point cloud prior data, achieving a hierarchical spatial representation of power facilities.

[0124] Step S22: performing semantic structure segmentation vector matrix calculation on the layered point cloud prior data to generate structure segmentation vector point cloud data;

[0125] In the embodiment of the present invention, semantic structure segmentation vector matrix calculation is performed on the layered point cloud prior data. The PointNet++ network architecture is used, the sampling rate is set to 0.01, and 1024 key points are extracted. For each key point p j , construct local feature descriptors within its neighborhood radius r = 0.5m. Calculate the normal vector of the point cloud (PCA method is used, the number of neighborhood points k = 30), and the covariance matrix is ​​constructed (in is the neighborhood center point). j Perform eigenvalue decomposition to obtain eigenvalues ​​λ1≥λ2≥λ3 and eigenvectors Defining Structural Eigenvectors: Linearity Flatness Scattering Anisotropy Feature entropy These features are combined with the coordinates (x, y, z) of the point, the normal vector into a 10-dimensional feature vector The point cloud is classified into 12 semantic categories such as tower (weight matrix W tower ), insulator W insulator ), and conductor W wire ) by a Softmax classifier, generating structure segmentation vector point cloud data.

[0126] Step S23: Point cloud gridding processing is performed on the hierarchical point cloud prior data based on the structure segmentation vector point cloud data, generating hierarchical grid point cloud data;

[0127] In the embodiment of the present application, the hierarchical grid processing is performed on the hierarchical point cloud prior data based on the structure segmentation vector point cloud data. The Ball Pivoting Algorithm (BPA) algorithm is adopted, the initial ball radius r0=0.15m, the maximum ball radius r max =0.3m, the minimum ball radius r min =0.08m, and the normal deviation threshold θ max =45 degrees are set. For the device layer point cloud, the normal vector is first redirected to ensure that all normal vectors are consistent. In the iteration process of the BPA algorithm, when the ball contacts three points, an initial triangle is formed. Then the ball rolls along the edges of the triangle to find new points to form new triangles until it cannot continue to expand. For the conductor point cloud, a cylindrical fitting method is adopted. For each segment of the conductor point cloud, the center point and the direction vector (obtained by PCA) are calculated. With as the center and as the axis, a cylindrical surface with a radius of r=0.05m is constructed, and the conductor points are projected onto the cylindrical surface to generate a grid. In the finally generated hierarchical grid point cloud data, the average edge length of the device layer grid is 0.08m, the average circumference of the conductor grid is 0.314m, and the overall grid quality evaluation index (such as triangle aspect ratio) reaches the industry standard requirement.

[0128] Step S24: Construct a point cloud rendering model based on the hierarchical point cloud prior data and the hierarchical grid point cloud data;

[0129] In the embodiment of the present application, the point cloud rendering model is constructed based on the hierarchical point cloud prior data and the hierarchical grid point cloud data. The Octree space partitioning method is adopted, the root node covers the entire point cloud space, and recursive partitioning is performed until the number of points contained in the leaf node is less than 500 or the node edge length is less than 0.1m. For each leaf node, the center point coordinate O i of its bounding box, the size vector E i =[l i ,wi ,h i ] and color value C i (Obtained by calculating the color average of all points in the leaf node). Construct an LOD (Level of Detail) model and set three levels of detail: LOD0 (complete point cloud data), LOD1 (retain 1 for every 10 points), and LOD2 (retain 1 for every 100 points). Fuse the layered grid point cloud data with the Octree structure. For the grid model, store the index of the triangle patch that intersects with the node in each Octree node. During rendering, the LOD level is dynamically selected according to the viewpoint distance: LOD0 is displayed when the viewpoint distance is less than 10m, LOD1 is displayed when the distance is between 10m-50m, and LOD2 is displayed when the distance is greater than 50m. Through the above processing, the constructed point cloud rendering model maintains visual accuracy while improving rendering efficiency and reducing memory usage.

[0130] Step S25: Calculating point cloud surface coefficients based on the structure segmentation vector point cloud data to generate point cloud surface coefficients;

[0131] In the embodiment of the present invention, the point cloud surface coefficients are calculated based on the structure segmentation vector point cloud data. The moving least squares method (MLS) is used, and the fitting radius r = 0.2m and the polynomial order k = 2 are set. For each point p i , construct a local coordinate system in its neighborhood and fit the quadratic surface z=f(x,y)=a0+a1x+a2y+a3x 2 +a4xy+a5y 2 . Calculate the first fundamental form coefficients of the surface F=f x f y , and the second fundamental form coefficients The surface coefficient is calculated as follows: Gaussian curvature Mean curvature principal curvature Shape Index Surface change rate For the insulator surface point cloud, the calculated Gaussian curvature range is [-0.5, 0.5] m ―2 , the average curvature range is [―0.2,0.2]m ―2 , providing an accurate geometric feature description for subsequent surface reconstruction.

[0132] Step S26: constructing a point cloud surface reconstruction rendering model based on the point cloud surface coefficients and the point cloud rendering model.

[0133] In the embodiment of the present invention, a point cloud surface reconstruction rendering model is constructed based on the point cloud surface coefficient and the point cloud rendering model. The implicit surface reconstruction method is adopted to define the Signed Distance Function (SDF) d(p), and the Poisson equation is used to calculate the point cloud surface reconstruction rendering model. Solve (where is the normal vector field). Set the solution depth to 8, construct an octree mesh, and the mesh resolution is 0.05m. At each mesh node, the SDF value is calculated by trilinear interpolation. Use the Marching Cubes algorithm to extract the isosurface (the threshold is set to 0) and generate a triangular mesh model. For high curvature areas (such as insulator edges and conductor clamps), an adaptive subdivision strategy is adopted. When the local curvature |K|>0.1m-2, the mesh resolution is increased to 0.02m to ensure that details are retained. During the reconstruction process, the point cloud surface coefficient is used to optimize the normal vector so that the direction of the normal vector is consistent with the change of the surface curvature. Compared with the original point cloud rendering model, the final constructed point cloud surface reconstructed rendering model has improved surface details and reduced geometric accuracy errors, maintaining visual quality while meeting real-time rendering requirements.

[0134] Furthermore, step S25 includes the following steps:

[0135] Perform curvature analysis of associated vector point cloud layers based on the structure segmentation vector point cloud data to generate curvature change vector data;

[0136] In the embodiment of the present invention, the curvature analysis of the associated vector point cloud layer is performed based on the structure segmentation vector point cloud data. The local principal component analysis (PCA) method is used to calculate the curvature of each point. For any point p in the structure segmentation vector point cloud data, i (x i ,y i ,z i ), a spherical neighborhood with a radius of r = 0.15m is established with it as the center, and all points p in the neighborhood are collected j The set N(p i ). Calculate the neighborhood point set N(p i )'s covariance matrix in is the centroid of the neighborhood point set. Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalues ​​λ1≥λ2≥λ3 and the corresponding eigenvectors Click p i (Gaussian curvature of Mean curvature The Gaussian curvature K i , mean curvature H iThe point coordinates, normal vectors, and structure segmentation vectors are combined into new multi-dimensional vectors to generate curvature change vector data. Taking point cloud data of a substation insulator string as an example, the curvature characteristics of each point are calculated by this method, and the bending characteristics of the insulator surface are completely described, thereby providing accurate geometric information for subsequent processing.

[0137] Preferably, the point cloud spatial hierarchical aggregation processing is performed according to the curvature change vector data to generate aggregated curvature point cloud layer data.

[0138] In the embodiment of the present application, the point cloud spatial hierarchical aggregation processing is performed according to the curvature change vector data. The density-based spatial clustering method DBSCAN is adopted, the neighborhood radius ∈ is set to 0.2 m, and the minimum point quantity MinPts is set to 15. The point clouds in the curvature change vector data are clustered according to the spatial positions to form different point cloud clusters. For each point cloud cluster, the average curvature K and the average Gaussian curvature K where n is the number of points in the cluster. The point cloud clusters are hierarchically divided according to the average curvature sizes: low curvature layer (|K and |K ˉ |<0.02), medium curvature layer (0.02≤|K or 0.02≤|K ˉ |0.1), and high curvature layer (|K or |K ˉ |≥0.1). The point cloud clusters in the same layer are merged to generate aggregated curvature point cloud layer data. For example, when processing point cloud data of a power line tower, the smooth main body part of the tower is divided into the low curvature layer, and the connection parts, bolts, and other detailed parts of the tower are divided into the high curvature layer by the hierarchical aggregation processing, thereby realizing effective hierarchical management of the point cloud data.

[0139] Preferably, the three-dimensional surface segmentation processing is performed according to the aggregated curvature point cloud layer data to generate surface segmentation point cloud layer data.

[0140] In the embodiment of the present application, the three-dimensional surface segmentation processing is performed according to the aggregated curvature point cloud layer data. The region growing algorithm is adopted, and for each aggregated curvature point cloud layer, the point with the most representative average curvature is selected as the seed point p seed. Set the growth conditions: the angle θ between the normal vector of the neighborhood point and the seed point is less than 30 degrees, the curvature difference ΔH between the neighborhood point and the seed point is less than 0.03 and ΔK is less than 0.01. Starting from the seed point, search for points that meet the growth conditions within the neighborhood with a radius of r = 0.1m, and add these points to the current surface area. Repeat this process and continuously expand the surface area until there are no points that meet the conditions to be added. When the growth of a surface area is completed, a new seed point is selected from the remaining point cloud to continue the growth of the next surface area. Through this method, the aggregated curvature point cloud layer data is segmented into multiple independent surface areas to generate surface segmentation point cloud layer data. Taking the point cloud data processing of the transformer heat sink as an example, the different surface parts of the heat sink are successfully segmented accurately, and each segmented surface area has relatively consistent geometric features.

[0141] Preferably, surface fitting coefficients are calculated based on the surface segmentation point cloud layer data to generate point cloud surface coefficients.

[0142] In the embodiment of the present invention, the surface fitting coefficient is calculated based on the surface segmentation point cloud layer data. For each segmented surface area, the moving least squares (MLS) method is used to perform surface fitting. Let the point set in the surface area be P = {p1, p2, ..., p m}, for any point q on the surface to be fitted, a local coordinate system is constructed within its neighborhood with a radius of r = 0.12m. By minimizing the weighted square error To determine the coefficients of the fitting function f(x,y), where w(q―p i ) is the weight function, using Gaussian weight σ=0.05m. The fitting function f(x,y) adopts the quadratic polynomial form f(x,y)=a0+a1x+a2y+a3x 2 +a4xy+a5y 2 By solving the linear equations to obtain the coefficients a0, a1, ..., a5, we can calculate the first basic form coefficients of the surface F=f x f y , and the second fundamental form coefficients Calculate the Gaussian curvature of the surface based on these basic form coefficients Mean curvature principal curvature These parameters together constitute the point cloud surface coefficients. For example, when processing the surface of a power equipment casing, the point cloud surface coefficients obtained through precise surface fitting coefficient calculation can accurately reflect the geometric form of the equipment casing, providing reliable parameter support for the subsequent point cloud surface reconstruction and rendering model construction.

[0143] Furthermore, step S26 includes the following steps:

[0144] Perform angle window sliding simulation on the point cloud rendering model based on the point cloud surface coefficient to generate point cloud curvature sliding window data;

[0145] In the embodiment of the present invention, the angle window sliding simulation is performed on the point cloud rendering model based on the point cloud surface coefficient. i A cubic sliding window of 0.2 m × 0.2 m × 0.2 m was constructed with a 15-degree rotation step. For the point cloud data within the window, the Gaussian curvature K and mean curvature H of each point were extracted based on the calculated point cloud surface coefficients. In a point cloud rendering model of a substation insulator string, starting from the insulator base, the sliding window was moved along the insulator axis in 0.1-meter increments, while simultaneously rotating the window around the axis in 15-degree steps. At each window position, the Gaussian curvature and mean curvature data of the points within the window were recorded as a data set. After the window traversed the entire insulator string point cloud data, all data sets were combined according to the window movement and rotation sequence to generate the point cloud curvature sliding window data. This data contains curvature information at different positions and angles of the insulator string. For example, at the edge of the insulator shed, the sliding window captures the dramatic change in Gaussian curvature in the local area, providing rich curvature details for subsequent analysis.

[0146] Preferably, a spatial gradient change analysis is performed based on the point cloud curvature sliding window data to generate curvature spatial gradient change data;

[0147] In the embodiment of the present invention, spatial gradient change analysis is performed based on the point cloud curvature sliding window data. For each group of adjacent windows (adjacent along the moving direction or rotation direction) in the point cloud curvature sliding window data, the Gaussian curvature and the average curvature are calculated.

[0148] Taking Gaussian curvature as an example, let the adjacent window W i and W i+1 The average Gaussian curvatures in and The coordinates of the center points of the two windows are (x i ,y i ,z i ) and (x i+1 ,y i+1 ,z i+1 ), then the gradient of Gaussian curvature in space is G K The calculation formula is: Similarly, the spatial gradient of the mean curvature G is calculated HIn the process of processing the point cloud curvature sliding window data of the cross arm of the power transmission line tower, the Gaussian curvature spatial gradient value from the cross arm connection to the main part is calculated by the above formula, which gradually decreases from 2.5 m ―2 / m to 0.3 m ―2 / m, and the average curvature spatial gradient value decreases from 1.2 m ―2 / m to 0.1 m ―2 / m. The curvature spatial gradient values of all adjacent windows are integrated to generate curvature spatial gradient change data, which clearly shows the curvature change trend of the point cloud model in space.

[0149] Preferably, the local geometric warping distortion detection is performed according to the curvature spatial gradient change data to generate spatial surface warping data;

[0150] In the embodiment of the present application, the local geometric warping distortion detection is performed according to the curvature spatial gradient change data. The Gaussian curvature spatial gradient threshold T K = 1.5 m ―2 / m and the average curvature spatial gradient threshold T H = 1.5 m ―2 / m are set. For each gradient value in the curvature spatial gradient change data, if the Gaussian curvature spatial gradient G K is greater than T K or the average curvature spatial gradient G H is greater than T H , it is determined that there is local geometric warping distortion in the region. In the detection of the point cloud model of the transformer fin, it is found that the Gaussian curvature spatial gradient at the fin corrugation connection reaches 3.2 m ―2 / m, which exceeds the set threshold, and it is determined that this is a warping distortion region. The point cloud data coordinates of all detected warping distortion regions, the corresponding curvature spatial gradient values and the distortion type (determined based on the exceeding of the Gaussian curvature or the average curvature) are recorded to generate spatial surface warping data. Through the data, the positions of the geometric distortion in the point cloud model of the power facility, such as the deformation of the fin and the local distortion of the tower, can be accurately located.

[0151] Preferably, the warping layered detail smoothing correction processing is performed on the spatial surface warping data based on the point cloud surface coefficient to generate surface smoothing correction data;

[0152] In the embodiment of the present application, the warping layered detail smoothing correction processing is performed on the spatial surface warping data based on the point cloud surface coefficient. According to the distortion regions recorded in the spatial surface warping data, the warping degree (measured by the size of the curvature spatial gradient value) is layered. The mild warping layer is set as 1.5 m ―2 / m < G K ≤ 3 m ―2 / m or 0.8 m ―2 / m < GH ≤1.6m ―2 / m, moderate warping layer 3m ―2 / m < G K ≤5m ―2 / m or 1.6m ―2 / m < G H ≤2.5m ―2 / m and severe warping layer G K >5m ―2 / m or G H >2.5m ―2 / m. For the light warping layer, a weighted average method is used for smoothing correction. Let the coordinates of a point p in the warping area be (x, y, z), and the point set in its neighborhood be N(p), and different weights are given to the neighborhood points according to the average curvature H in the point cloud surface coefficient. j j j j j j The new point coordinates are calculated by weighted average: For the moderate and severe warping layers, the local surface is refitted in combination with the moving least square method (MLS) with the point cloud surface coefficient as the constraint condition. In the processing of a warping area of a power equipment shell, the average curvature spatial gradient of the light warping area is reduced from 2.2m / m to 0.9m / m after layered smoothing correction, and the modified point cloud data is integrated to generate surface smoothing correction data. ―2 ―2

[0153] Preferably, the point cloud rendering model is subjected to surface reconstruction processing based on the surface smoothing correction data to generate a point cloud surface reconstruction rendering model.

[0154] In the embodiment of the application, the point cloud rendering model is subjected to surface reconstruction processing based on the surface smoothing correction data. Triangular mesh subdivision and optimization algorithms are used. For each point in the surface smoothing correction data, its triangular face sheet in the triangular mesh structure of the point cloud rendering model is located. If the point is in the warping correction area, the triangular face sheet containing the point is subdivided. When subdividing, a new vertex is inserted at the midpoint of the three sides of the triangle to divide the original triangle into four small triangles, and the connection relationship and normal vector of the vertex are updated. Taking the point cloud model reconstruction of the connection of a tower insulator as an example, after three times of triangular mesh subdivision of the area with warping correction, the mesh is further optimized by using the Laplace smoothing algorithm. Let the neighborhood vertex set of a vertex v be N(v), and the new coordinates of the vertex v are calculated by the following formula: ​​​​​​​​Wherein, a = 0.3 is a smoothing factor. After subdivision and smoothing, the surface of the point cloud rendering model at the connection of the insulator is smoother and more natural, the geometric accuracy error of the overall model is reduced from 0.05m to 0.02m, and finally the point cloud surface reconstruction rendering model is generated, which provides a high-precision three-dimensional model for the visualization display and subsequent analysis of the power facility.

[0155] Further, the step S3 comprises the following steps:

[0156] Step S31: acquiring image perspective data corresponding to the power facility;

[0157] In the embodiment of the present application, the DJI longitude and latitude M300RTK unmanned aerial vehicle with a 2400 million pixel full-frame sensor is used to acquire the image perspective data corresponding to the power facility. The flight height of the unmanned aerial vehicle is set to 30 meters above the highest point of the power facility, and the unmanned aerial vehicle flies along the periphery of the power facility in a spiral around at a speed of 10 meters / second, and the spiral radius of the flight trajectory is increased by 5 meters as a step. During the flight, the camera is triggered to take pictures at a frequency of once every 2 seconds, the camera lens focal length is fixed at 24mm, the aperture is set to f / 8, and the ISO value is set to 100, so as to ensure that the image is clear and the exposure is uniform. When collecting image data of a certain 500kV substation, the unmanned aerial vehicle flies around the substation for 3 circles, and a total of 320 image data with a resolution of 6000x4000 are acquired, each image is attached with accurate geographic coordinates (longitude, latitude, elevation) obtained by the RTK positioning system and shooting posture information (heading angle, pitch angle, roll angle) recorded by the IMU inertial measurement unit. The image data is stored in DNG format, which retains complete original image information and provides a high-quality data source for subsequent processing.

[0158] Step S32: performing initial pose standard processing according to the image perspective data and the point cloud rendering model to generate initial pose alignment data;

[0159] In the embodiment of the present application, the initial pose standard processing is performed according to the image perspective data and the point cloud rendering model. First, the scale invariant feature transform (SIFT) algorithm is used to extract feature points from the image perspective data and the point cloud rendering model respectively. For the image, extreme points are detected on different levels of the Gaussian pyramid, and through key point positioning and direction assignment, an average of 1500 feature points are extracted from each image. For the point cloud rendering model, a voxel grid method is used, and the point cloud space is divided into grids with a grid length of 0.1 meters. The point with the maximum curvature in each grid is selected as a feature point, and a total of 8000 feature points are extracted. The feature point pairs of the image and the point cloud rendering model are matched through the fast nearest neighbor search algorithm (FLANN), and after at least 100 matching point pairs are obtained, the random sample consensus algorithm (RANSAC) is used to remove the mis-matching point pairs. The rotation matrix R and the translation vector T are calculated according to the matching point pairs, so that the point cloud rendering model is aligned with the image perspective data, and the alignment data of the initial pose is generated by minimizing the re-projection error solved, where u i is the pixel coordinate of the feature point in the image, p i is the three-dimensional coordinate of the corresponding feature point in the point cloud rendering model, π is the projection function, and n is the number of matching point pairs. After 100 iterations, the re-projection error is reduced, and the initial pose alignment data is generated, realizing the preliminary spatial registration of the point cloud and the image.

[0160] Step S33: performing joint rendering model pose alignment processing based on the initial pose alignment data and the point cloud surface reconstruction rendering model to generate a fusion pose alignment model;

[0161] In the embodiment of the present application, the joint rendering model pose alignment processing is performed based on the initial pose alignment data and the point cloud surface reconstruction rendering model. After the point cloud surface reconstruction rendering model is spatially transformed according to the initial pose alignment data, the iterative closest point (ICP) algorithm is used to further optimize the pose. For each point p cloud in the point cloud surface reconstruction rendering model, the closest point p image in the three-dimensional space corresponding to the image perspective data is found, and the rotation matrix R ICP and the translation vector T ICP are calculated by minimizing the mean square error (where m is the number of corresponding point pairs). In the processing of the model and image data of the transmission line tower, the maximum number of iterations of the ICP algorithm is set to 50 times, and the convergence threshold is set to 0.01 meters. After iterative calculation, the alignment error of the tower model and the image in the horizontal direction is reduced from 0.3 meters to 0.05 meters, and the vertical direction error is reduced from 0.25 meters to 0.03 meters, generating a fusion pose alignment model. The model realizes high-precision alignment of the point cloud surface reconstruction rendering model and the image perspective data in the three-dimensional space, laying a foundation for subsequent fusion rendering.

[0162] Step S34: performing multi-frame rendering point cloud spatiotemporal reconstruction alignment fitting processing on the fusion pose alignment model to generate a multi-frame rendering spatiotemporal alignment point cloud model;

[0163] In the embodiment of the present application, the fusion pose alignment model is subjected to multi-frame rendering point cloud spatiotemporal reconstruction alignment fitting processing. The acquired multi-frame image perspective data is arranged in the order of shooting time, and for the fusion pose alignment model corresponding to each frame of image, the optical flow method is used to calculate the point cloud motion vector between adjacent frames. For each point p in the point cloud, the two-dimensional pixel displacement (u, v) is calculated in the adjacent two frames of images by the pyramid LK optical flow algorithm, and then the displacement vector Δp in the three-dimensional space is calculated according to the camera intrinsic matrix K and the extrinsic matrix (rotation matrix R and translation vector T). The displacement vector calculated is used to fit the motion trajectory of each point in the time dimension by using the Bezier curve fitting algorithm. Assuming that the three-dimensional coordinate sequence of a point in n frames of images is {p1, p2, …, p n}, a cubic Bezier curve B(t) = (1-t) 3 p1+3(1-t) 2 tp2+3(1-t)t 2 p3+t 3 p n The curve parameters are adjusted by the least square method to minimize the error between the curve and the actual motion trajectory of the point. After processing, a multi-frame rendering spatiotemporal alignment point cloud model is generated, which accurately restores the spatial morphological changes of the power facility at different time points, and reduces the trajectory fitting error in the time dimension.

[0164] Step S35: performing image texture mapping region analysis based on the multi-frame rendering spatiotemporal alignment point cloud model to generate image texture mapping region data;

[0165] In the embodiment of the present application, the image texture mapping region analysis is performed based on the multi-frame rendering spatiotemporal alignment point cloud model. First, the point cloud model is projected onto each frame of image plane to calculate the projection coordinates (u, v) of each point in the image, and the projection formula is where (x, y, z) is the three-dimensional coordinates in the point cloud model. For the projected image region, a segmentation algorithm based on color and gradient is used to divide the texture mapping region. The Lab color space distance and the gradient amplitude of each pixel point are calculated. The color distance threshold T Lab = 15 and the gradient threshold T G = 20 are set, and the color distance is less than T Lab and the gradient amplitude is less than T GThe pixels of the same texture region are divided. In the analysis of the transformer image texture mapping region of the transformer substation, the transformer shell, the heat sink, the nameplate and other regions are accurately segmented, and 12 texture mapping regions are divided. The boundary coordinates, average color value, texture feature descriptor and other information of each region are recorded to generate image texture mapping region data, which provides a basis for accurate texture mapping.

[0166] Step S36: Based on the image texture mapping region data, the multi-frame rendering space-time aligned point cloud model is processed by model texture enhancement mapping to generate an enhanced texture rendering model.

[0167] In the embodiment of the application, the multi-frame rendering space-time aligned point cloud model is processed by model texture enhancement mapping based on the image texture mapping region data. The physical-based rendering (PBR) technology is used. For each texture mapping region, different reflectivity, roughness and refractive index parameters are set according to its material properties (metal, insulating material, glass, etc.). For example, for the metal part of the tower, the reflectivity is set to 0.8 and the roughness is set to 0.2, and for the insulator ceramic part, the reflectivity is set to 0.3 and the roughness is set to 0.4. The image texture is mapped to the surface of the point cloud model using a bilinear interpolation algorithm. For the points at the boundary of the texture mapping region, weighted fusion is performed according to the distance d of the point to the boundary to avoid the sawtooth generated by texture splicing. The color values of the texture regions on both sides of the boundary are C1 and C2, respectively, and the color value after fusion is where d1 and d2 are the distances of the point to the two boundaries. For key regions (such as device nameplates and bolt connections), a super-resolution reconstruction algorithm is used to increase the texture resolution from the original 100x100 to 300x300. After processing, an enhanced texture rendering model is generated, the texture definition of the model surface is improved, the reflection and refraction effects conform to the physical laws, and the appearance details and material properties of the power facilities can be truly restored in the three-dimensional visualization display.

[0168] Further, step S36 includes the following steps:

[0169] Based on the image texture mapping region data and the image view angle data, texture feature detail extraction is performed to generate texture feature detail data;

[0170] In the embodiment of the present application, the texture feature details are extracted based on the image texture mapping region data and the image view angle data. A local binary pattern (LBP) algorithm is used to process the image in each texture mapping region. The image is divided into 8x8 pixel blocks. For each pixel block center pixel point p(x, y), 8 neighborhood pixel points are selected on the circumference with the center pixel point p(x, y) as the center and a radius r = 1. The gray scale values of the neighborhood pixel points and the center pixel point are compared in turn in the clockwise direction. If the gray scale value of the neighborhood pixel point is greater than or equal to the gray scale value of the center pixel point, it is recorded as 1, otherwise as 0. An 8-bit binary number is obtained, which is converted into a decimal number as the LBP feature value of the pixel block. When the transformer nameplate region of the transformer substation is processed, the edge and stroke thickness of the characters on the nameplate are extracted by the LBP algorithm. Each 8x8 pixel block corresponds to an LBP feature value, forming a feature matrix. At the same time, a histogram of oriented gradients (HOG) algorithm is used to divide each texture mapping region into 4x4 cell units. The gradient histogram of 8 directions is calculated in each cell unit. The gradient amplitude and direction of each pixel point in different directions are counted, forming a 32-dimensional HOG feature vector. The LBP feature matrix and the HOG feature vector are combined to generate the texture feature detail data. For example, in the texture feature extraction on the surface of the insulator, the concave-convex texture and the stain trace on the surface are completely captured by the combination of the two algorithms, providing a rich feature basis for subsequent texture enhancement.

[0171] Preferably, the texture feature detail data is subjected to image texture space mapping enhancement processing to generate enhanced texture mapping data.

[0172] In the embodiment of the present application, the texture feature detail data is subjected to image texture space mapping enhancement processing. A multi-scale Gaussian pyramid algorithm is used to construct the image of each texture mapping region into a Gaussian pyramid structure containing 5 layers. The bottom layer is the original image, and the upper layer image is obtained by Gaussian filtering (Gaussian kernel size of 5x5, standard deviation σ = 1.0) and down-sampling (sampling factor of 2) on the lower layer image. For the LBP feature matrix in the texture feature detail data, feature matching is performed on each layer of the Gaussian pyramid. With a certain feature point as the center, a search radius of 3 pixels is used to find the point with the closest LBP feature value as the matching point in the adjacent layer image. The spatial displacement vector between the matching points is calculated. For the HOG feature vector, it is mapped to the image of different scales by a bilinear interpolation algorithm to enhance the spatial continuity of the feature. The calculated displacement vector and the feature mapping relationship are used to enhance the texture image by a non-local mean (NLM) filtering algorithm. For each pixel point p in the texture image, the similarity weight w(p, q) between the pixel point p and other pixel points q in a search window with the pixel point p as the center and a size of 7x7 is calculated, and the formula is as follows: wherein, I N(p) and Z(p) are respectively the pixel gray value vectors in the 3x3 neighborhood centered at p and q, Z(p) is a normalization constant, h=10 is the gray similarity control parameter, a=2 is the spatial distance control parameter, and sigma=1.5 is the standard deviation of the Gaussian kernel. The enhanced gray value of the pixel point p is calculated by weighted average to generate the enhanced texture mapping data. When processing the rust texture on the surface of the power transmission tower, the color contrast of the rust is improved and the texture details are more clear and prominent through the processing.

[0173] Preferably, the model texture enhancement mapping processing is performed on the multi-frame rendered spatiotemporally aligned point cloud model based on the enhanced texture mapping data to generate an enhanced texture rendering model.

[0174] In the embodiment of the present application, the model texture enhancement mapping processing is performed on the multi-frame rendered spatiotemporally aligned point cloud model based on the enhanced texture mapping data. The enhanced texture mapping data is accurately attached to the surface of the point cloud model by using the physical-based rendering (PBR) technology combined with the texture coordinate mapping method. For each triangular patch in the point cloud model, the corresponding texture coordinates (u, v) are calculated according to the three-dimensional coordinates of its vertices by projection transformation and texture coordinate mapping formula: (where (x, y, z) is the three-dimensional coordinates of the vertices, K is the camera intrinsic matrix, and R and T are the extrinsic matrices) to assign the pixel values in the enhanced texture mapping data to the triangular patch. For power facility components of different materials, the PBR parameters are set according to their physical properties. For example, for the metal components of the tower, the base color is set to RGB(128, 128, 128), the metalness is set to 1.0, the roughness is set to 0.3, and the reflectance is set to 0.8; for the ceramic components of the insulator, the base color is set to RGB(255, 255, 255), the metalness is set to 0.0, the roughness is set to 0.4, and the reflectance is set to 0.3. By using these parameters, the reflection, refraction, and scattering effects of light on the model surface are calculated by the physical-based lighting model. When processing complex curved surfaces of power facilities (such as the insulator shed), the triangular patches are further subdivided by using the adaptive subdivision algorithm to ensure the accuracy of the texture mapping. After processing, the enhanced texture rendering model is generated, and the degree of detail restoration of the texture on the model surface is improved. Under different lighting conditions, the material texture and surface characteristics of the power facility are presented more realistically, providing high-quality three-dimensional model support for the visualization monitoring and analysis of power facilities.

[0175] Further, the step S4 comprises the following steps:

[0176] Step S41: performing coordinated cloud edge analysis according to the coordinated prior point cloud data to generate coordinated cloud edge data;

[0177] In the embodiment of the present application, the cloud-edge analysis is performed according to the coordinated prior point cloud data. A cloud-edge computing resource evaluation model is constructed, the cloud resource includes a 16-core CPU (main frequency 2.8 GHz), an NVIDIA A100 GPU (40 GB display memory) and a 1000 Mbps network bandwidth, and the edge device is configured as an 8-core CPU (main frequency 2.4 GHz), an NVIDIA T4 GPU (16 GB display memory) and a 100 Mbps network bandwidth. The coordinated prior point cloud data is divided into 100 data blocks according to the spatial region, and each data block contains an average of 500,000 point cloud data points. For each data block, the data size S i (unit: MB), the geometric complexity index C i and the transmission time T trans . The geometric complexity index C i is obtained by calculating the average curvature change rate of the point cloud in the data block and the product of the neighborhood point density ρ, that is, The transmission time , where B is the network bandwidth (1000 Mbps for the cloud and 100 Mbps for the edge). Taking the point cloud data block of the main transformer area of a substation as an example, the data size

[0178] S = 80 MB, the average curvature change rate , the neighborhood point density ρ = 600, and the geometric complexity index C = 0.12 x 600 = 72. The transmission time in the cloud is , and the transmission time in the edge is By comparing the processing cost (processing cost = transmission time + estimated calculation time, and the estimated calculation time is obtained according to the relationship between the geometric complexity index and the device computing power) of each data block in the cloud and the edge, the processing position of the data block is determined, and the coordinated cloud-edge data is generated. As described above, the data block of the transformer area is determined to be processed by the cloud due to the high geometric complexity and large data size.

[0179] Step S42: performing rendering computing power thread estimation calculation according to the coordinated cloud-edge data to generate rendering computing power thread data;

[0180] In the embodiment of the present application, the rendering computing power thread estimation calculation is performed according to the coordinated cloud-edge data. A rendering computing power demand evaluation formula is established, and the number of rendering computing power threads N is determined by the data processing amount V, the rendering precision requirement P and the device computing power coefficient F, that is, The data processing amount V is calculated according to the total number of data blocks allocated to the corresponding device (cloud or edge) in the coordinated cloud-edge data. For example, the total number of point cloud data allocated to the cloud is 30 million, and V = 30000000. The rendering precision requirement P is represented by the level of detail (LOD) of point cloud rendering. LOD0 corresponds to a precision coefficient of 1.0, LOD1 corresponds to 0.6, and LOD2 corresponds to 0.3. Here, LOD1 is taken, i.e. P = 0.6. The device computing power coefficient F is obtained by benchmark testing. The average time for a cloud device to process 1 million point cloud data points is 0.2s, and the corresponding computing power coefficient is The average time for an edge device to process 1 million point cloud data points is 0.5s, and the corresponding computing power coefficient is Taking 30 million point cloud data points allocated to the cloud as an example, the rendering computing thread number is calculated as follows: Rounded up to 4 threads, 10 million point cloud data points allocated to the edge threads. The rendering computing thread numbers of the cloud and the edge obtained by calculation are integrated to generate rendering computing thread data.

[0181] Step S43: Based on the rendering computing thread data and the enhanced texture rendering model, rendering coordination lightweight deployment is performed to generate a lightweight point cloud rendering model.

[0182] In the embodiments of the present application, rendering coordination lightweight deployment is performed based on the rendering computing thread data and the enhanced texture rendering model. The progressive mesh simplification algorithm is used to process the enhanced texture rendering model, and the simplification target is set to 40% of the original model face number. The folding cost E of each triangle edge in the model is calculated fold , and the formula is where F is the set of faces adjacent to the edge, ΔA fThe area change amount of the folded edge rear f is obtained. Taking the enhanced texture rendering model of a certain power transmission line tower as an example, the original model contains 1.2 million triangular facets, and the edges are folded in turn according to the edge folding cost from small to large. The model topology and texture mapping relationship are updated each time the model is folded. When the number of model surfaces is reduced to 0.48 million, the simplification is stopped. The lightweight model is thread allocated in combination with the rendering computing thread data. The model is divided into sub-models equal to the number of rendering computing threads. If 4 threads are allocated in the cloud, the lightweight tower model is divided into 4 sub-models, each containing 120,000 triangular facets. During rendering, each thread independently processes a sub-model, and multi-thread parallel computing is used to accelerate the rendering process. At the same time, a texture compression algorithm is used to convert the model texture from RGBA8888 format to ETC2 format, with a compression ratio of 4:1, and the texture data volume is reduced from the original 200MB to 50MB. After the above processing, a lightweight point cloud rendering model is generated, which improves the model rendering efficiency while ensuring that the loss of model visualization effect is within a certain range, meeting the performance requirements of cloud-edge collaborative rendering.

[0183] Further, step S5 comprises the following steps:

[0184] Step S51: rendering interaction progressive level analysis is performed on the coordinated prior point cloud data to generate rendering interaction progressive level data;

[0185] In the embodiment of the present application, rendering interaction progressive level analysis is performed on the coordinated prior point cloud data. The point cloud data space topology structure is constructed, and the octree algorithm is used to recursively divide the coordinated prior point cloud data space into 8 subspaces. When the number of point clouds in the subspace is less than 100 or the edge length is less than 0.05 m, the division is stopped, and an octree structure with a depth of 8 is formed. The rendering interaction level coefficient L is defined i from the bottom layer (leaf node) to the top layer (root node), L i is 1, 2, 4, 8, 16, 32, 64, and 128 in turn, and the coefficient represents the importance and detail level of the data in the rendering interaction. Taking the coordinated prior point cloud data of a certain 220kV substation as an example, after octree division, the subspaces where key devices such as transformers and circuit breakers are located are at a lower level L i with a smaller value), and the point cloud data contained is more detailed, while the subspaces where secondary areas such as substation sites are located are at a higher level L i with a larger value). The geometric complexity C sub of each subspace is calculated, and the formula is where ΔK p is the curvature change amount of the point p in the subspace sub, N subThe number of point clouds in the subspace is a point cloud quantity. According to the geometric complexity and the level coefficient, rendering interaction progressive level data is generated, and the priority and detail presentation degree of point cloud data in different regions in the rendering interaction are determined.

[0186] Step S52: rendering resource allocation calculation is performed on the coordinated prior point cloud data according to the rendering interaction progressive level data, and rendering resource data is generated;

[0187] In the embodiment of the present application, rendering resource allocation calculation is performed on the coordinated prior point cloud data according to the rendering interaction progressive level data. A rendering resource allocation model is established, and the rendering resources include CPU computing resources (core number), GPU computing resources (video memory occupation) and network bandwidth resources. The resource allocation weights W cpu 、W gpu 、W net are respectively 0.3, 0.5 and 0.2. For each point cloud data subspace, its rendering resource requirement R sub is calculated, and the formula is R sub =W cpu ×C cpu +W gpu ×C gpu +W net ×C net . Wherein, C cpu is calculated according to the number of point clouds in the subspace and the processing complexity, and 0.1 CPU core is occupied for processing 1000 point cloud data, C gpu is determined according to the geometric complexity and the texture complexity of the subspace, and 10MB of video memory is occupied for each increase of 1 unit of geometric complexity, and 5MB of video memory is occupied for each increase of 1 unit of texture complexity, C net is calculated according to the data volume and the transmission priority of the subspace, and 0.1Mbps bandwidth is occupied for each 1MB of data volume at high priority, and 0.02Mbps bandwidth is occupied for each 1MB of data volume at low priority.

[0188] Step S53: point cloud data progressive rendering scheduling is performed based on the rendering interaction progressive level data and the estimated rendering resource data, and point cloud progressive rendering scheduling data is generated;

[0189] In the embodiment of the present application, point cloud data progressive rendering scheduling is performed based on the rendering interaction progressive level data and the rendering resource data. A time slicing rendering algorithm is adopted, and the rendering time slice T slice = 16.67ms (corresponding to 60fps frame rate) is set. According to the rendering interaction level coefficient L i and the rendering resource requirement R sub , the rendering order and the rendering time t subFor the subspaces with low hierarchy coefficient (high importance of details) and low resource demand, rendering is performed preferentially and a longer rendering duration is allocated, and the formula is For the subspaces with high hierarchy coefficient (low importance of details) and high resource demand, rendering is performed later and a shorter duration is allocated. The rendering order and duration information of all subspaces are integrated to generate point cloud progressive rendering scheduling data, so as to realize ordered and efficient rendering of point cloud data.

[0190] Step S54: performing lightweight real-time rendering processing on the point cloud progressive rendering scheduling data based on the lightweight point cloud rendering model to generate lightweight real-time rendering data;

[0191] In the embodiment of the present application, lightweight real-time rendering processing is performed on the point cloud progressive rendering scheduling data based on the lightweight point cloud rendering model. Real-time rendering is realized in a browser environment by using a WebGL graphics library. According to the point cloud progressive rendering scheduling data, in each rendering time slice, the submodel data in the lightweight point cloud rendering model is called in sequence for rendering. For each submodel, a view frustum culling algorithm is used to calculate the intersection of the submodel bounding box and the camera view frustum, and only the submodel in the view frustum is rendered. Let the minimum coordinates of the submodel bounding box be (x max ,y max ,z max ), and the maximum coordinates be (x max ,y max ,z max ), the equations of the near plane, far plane, left plane, right plane, upper plane and lower plane of the camera view frustum be F near , F far , F left , F right , F top , F bottom , if at least one of the 8 vertices of the submodel bounding box satisfies the conditions of all plane equations (for example, for the near plane F near , the vertex coordinates are substituted into the equation and the result is greater than 0), it is determined that the submodel is in the view frustum and is rendered, otherwise it is skipped. In the rendering process, the level of detail (LOD) setting of the lightweight point cloud rendering model is combined, and the LOD level is dynamically switched according to the distance between the camera and the submodel. When the distance is less than 10 m, LOD0 (highest detail) is used, when the distance is between 10-50 m, LOD1 is used, and when the distance is greater than 50 m, LOD2 is used. Through the above processing, lightweight real-time rendering data is generated, which realizes real-time and efficient rendering of power facility point cloud data while ensuring rendering quality.

[0192] Step S55: obtaining user behavior data;

[0193] In the embodiments of the present application, user behavior data is obtained through an event listener embedded in the rendering interface. For mouse operation, the screen coordinates (x screen ,y screen ,) of the mouse click position are recorded, and they are converted into three-dimensional space coordinates (x world ,y world ,z world ) through a projection matrix, while recording the time t click at the time of clicking. For mouse dragging operation, the displacement (Δx, Δy) of mouse movement and the time sequence during movement are recorded in real time. For mouse wheel zooming operation, the direction (up or down) and the scale value of the wheel rolling are recorded. For keyboard operation, the key press and release events are recorded, including the key name (such as W, A, S, D, etc. directional keys, Ctrl, Shift, etc. function keys) and the key duration.

[0194] Step S56: rendering feedback analysis of point cloud data is performed according to the user behavior data, and rendering feedback data is generated;

[0195] In the embodiments of the present application, rendering feedback analysis of point cloud data is performed according to the user behavior data. A user behavior analysis model is constructed, and click heat value H click , dragging influence factor F drag , and zoom adjustment coefficient C zoom are defined. The click heat value H click is calculated according to the number of clicks N click of the user in a certain area, and the formula is wherein is the total number of clicks of all areas. The dragging influence factor F drag is calculated according to the displacement and duration of mouse dragging, wherein t drag is the duration of dragging. The zoom adjustment coefficient C zoom is determined according to the scale value and direction of the wheel rolling, and C zoom = 1 for upward rolling of one scale, and C zoom = 0.8 for downward rolling of one scale. By integrating these parameters, the attention of the user to the tower area and the operation intention are analyzed, and rendering feedback data is generated, such as determining that the rendering precision of the area needs to be improved or the rendering resource allocation needs to be increased.

[0196] Step S57: cloud-edge rendering coordination transmission granularity optimization is performed according to the rendering feedback data, and cloud-edge rendering coordination transmission granularity data is generated;

[0197] In the embodiments of the present application, cloud-edge rendering coordination transmission granularity optimization is performed according to the rendering feedback data. A transmission granularity evaluation formula is established, and the transmission granularity G is determined by the user attention A, the data volume S, and the network condition N, and the formula is wherein the user attention A is calculated according to the click heat value H in the rendering feedback data click , the drag influence factor F drag , the zoom adjustment coefficient C zoom , the weighted calculation, A = 0.4 x H click + 0.3 x F drag + 0.3 x C zoom , wherein S max is a set maximum data volume threshold, taking 100 MB, and the network condition N is calculated according to the real-time network bandwidth and delay, with the network bandwidth corresponding to N = 1 every 100 Mbps, and the delay corresponding to N decreasing by 0.1 every 100 ms.

[0198] Step S58: cloud-edge rendering coordination transmission granularity data is used to perform cloud-edge coordination caching on the lightweight real-time rendering data and feedback to the terminal.

[0199] In the embodiment of the present application, cloud-edge rendering coordination transmission granularity data is used to perform cloud-edge coordination caching on the lightweight real-time rendering data and feedback to the terminal. A cloud-edge collaborative caching architecture is constructed, the cloud end uses a Redis caching database, the caching capacity is set to 10 GB, the edge end uses a local solid state disk (SSD) cache, and the capacity is 2 GB. According to the cloud-edge rendering coordination transmission granularity data, the data caching strategy is determined. For data with high transmission granularity (high attention, small data volume, and good network), the data is cached simultaneously at the cloud end and the edge end, and the caching time is set to 30 minutes. For data with low transmission granularity, the data is cached only at the edge end, and the caching time is set to 10 minutes. The least recently used (LRU) algorithm is used to manage the cache space, and when the cache space is insufficient, the least recently used data is deleted. When the lightweight real-time rendering data is fed back to the terminal, the data transmission mode is dynamically adjusted according to the network condition and the computing capability of the terminal device. If the terminal network bandwidth is lower than 50 Mbps, the data is preferentially obtained from the edge end cache, and if the bandwidth is higher than 100 Mbps, the latest data is obtained from the cloud end. Taking the user viewing the power facility model as an example, through cloud-edge coordination caching and intelligent transmission, the average transmission delay of the rendering data is reduced, and a smooth terminal display effect is realized.

[0200] Further, the present application also provides a cloud-edge coordinated point cloud model lightweight rendering processing system for executing the cloud-edge coordinated point cloud model lightweight rendering processing method as described above, which comprises:

[0201] a point cloud data coordination preprocessing module, configured to acquire power facility point cloud data, and perform point cloud coordination prior processing according to the power facility point cloud data to generate coordinated prior point cloud data;

[0202] The rendering model construction module is configured to construct a point cloud rendering model according to the coordinated prior point cloud data, perform point cloud surface coefficient calculation according to the coordinated prior point cloud data, and generate point cloud surface coefficients; and perform surface reconstruction processing on the point cloud rendering model based on the point cloud surface coefficients, to generate a point cloud surface reconstruction rendering model.

[0203] The rendering model enhancement module is configured to obtain image perspective data corresponding to the power facility, perform rendering model pose alignment processing based on the image perspective data, the point cloud rendering model, and the point cloud surface reconstruction rendering model, to generate a fusion pose alignment model; and perform model texture enhancement mapping processing according to the fusion pose alignment model, to generate an enhanced texture rendering model.

[0204] The rendering model deployment module is configured to perform rendering coordination lightweight deployment on the enhanced texture rendering model based on the coordinated prior point cloud data, to generate a lightweight point cloud rendering model.

[0205] The lightweight rendering and transmission module is configured to perform lightweight real-time rendering processing on the coordinated prior point cloud data based on the lightweight point cloud rendering model, to generate lightweight real-time rendering data; obtain user behavior data; perform point cloud data rendering feedback analysis according to the user behavior data, to generate rendering feedback data; perform cloud-edge rendering coordination transmission granularity optimization according to the rendering feedback data, to generate cloud-edge rendering coordination transmission granularity data; and perform cloud-edge coordination caching on the lightweight real-time rendering data based on the cloud-edge rendering coordination transmission granularity data, and feed back to a terminal.

[0206] Therefore, from any viewpoint, the embodiments should be considered as being exemplary and non-limiting, the scope of the application being defined by the appended claims and not by the above description, and all changes coming within the meaning and range of equivalency of the claims are therefore encompassed by the application.

[0207] The above description is merely that of a specific implementation of the present application, enabling a person skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not to be limited to the embodiments shown herein but is to accord with the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cloud-edge coordinated point cloud model lightweight rendering processing method, characterized in that: The following steps are involved: Step S1: acquiring power facility point cloud data, and performing point cloud coordination prior processing on the power facility point cloud data to generate coordinated prior point cloud data; Step S2: constructing a point cloud rendering model based on the coordinated prior point cloud data; calculating the point cloud surface coefficients based on the coordinated prior point cloud data to generate the point cloud surface coefficients; performing surface reconstruction processing on the point cloud rendering model based on the point cloud surface coefficients to generate a point cloud surface reconstructed rendering model; Step S3: Obtain image view data corresponding to the power facility, and perform rendering model pose alignment processing based on the image view data, the point cloud rendering model, and the point cloud surface reconstruction rendering model to generate a fused pose alignment model; Perform model texture enhancement mapping processing based on the fused pose alignment model to generate an enhanced texture rendering model; Step S4: performing rendering coordination and lightweight deployment on the enhanced texture rendering model based on the coordinated prior point cloud data to generate a lightweight point cloud rendering model; Step S5: performing lightweight real-time rendering processing on the coordinated prior point cloud data based on the lightweight point cloud rendering model to generate lightweight real-time rendering data; obtaining user behavior data; performing point cloud data rendering feedback analysis based on the user behavior data to generate rendering feedback data; Based on the rendering feedback data, the cloud-edge rendering coordination transmission granularity is optimized to generate cloud-edge rendering coordination transmission granularity data; based on the cloud-edge rendering coordination transmission granularity data, the lightweight real-time rendering data is cloud-edge coordinated and cached and fed back to the terminal.

2. The cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 1 is characterized in that: Step S1 includes the following steps: Step S11: Acquire power facility point cloud data; Step S12: clustering the power facility point cloud data based on a preset power point cloud facility semantic clustering algorithm to generate clustered point cloud data; Step S13: performing time series node verification and alignment processing on the clustered point cloud data to generate time series verification point cloud data; Step S14: performing regional point cloud overlap analysis based on the time series verification point cloud data to generate regional overlapping point cloud data; Step S15: performing redundant power point cloud facility filtering processing on the clustered point cloud data based on the regional overlapping point cloud data to generate filtered power point cloud facility data; Step S16: performing point cloud coordination priori processing based on the time series verification point cloud data and filtering out the power point cloud facility data to generate coordinated priori point cloud data.

3. The cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 1 is characterized in that: Step S2 includes the following steps: Step S21: performing layered processing on the point cloud data according to the coordinated prior point cloud data to generate layered point cloud prior data; Step S22: performing semantic structure segmentation vector matrix calculation on the layered point cloud prior data to generate structure segmentation vector point cloud data; Step S23: performing point cloud meshing processing on the layered point cloud prior data based on the structure segmentation vector point cloud data to generate layered mesh point cloud data; Step S24: constructing a point cloud rendering model based on the layered point cloud prior data and the layered grid point cloud data; Step S25: Calculating point cloud surface coefficients based on the structure segmentation vector point cloud data to generate point cloud surface coefficients; Step S26: constructing a point cloud surface reconstruction rendering model based on the point cloud surface coefficients and the point cloud rendering model.

4. The cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 3 is characterized in that: Step S25 includes the following steps: Perform curvature analysis of associated vector point cloud layers based on the structure segmentation vector point cloud data to generate curvature change vector data; Perform point cloud spatial hierarchical aggregation processing based on curvature change vector data to generate aggregated curvature point cloud layer data; Perform three-dimensional surface segmentation processing based on the aggregated curvature point cloud layer data to generate surface segmentation point cloud layer data; The surface fitting coefficients are calculated based on the surface segmentation point cloud layer data to generate the point cloud surface coefficients.

5. The cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 3 is characterized in that: Step S26 includes the following steps: Perform angle window sliding simulation on the point cloud rendering model based on the point cloud surface coefficient to generate point cloud curvature sliding window data; Perform spatial gradient change analysis based on point cloud curvature sliding window data to generate curvature spatial gradient change data; Perform local geometric warping distortion detection based on the curvature spatial gradient change data to generate spatial surface warping data; Perform warping layered detail smoothing correction processing on the spatial surface warping data based on the point cloud surface coefficient to generate surface smoothing correction data; The point cloud rendering model is reconstructed based on the surface smoothing correction data to generate a point cloud surface reconstructed rendering model.

6. The cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 1 is characterized in that: Step S3 includes the following steps: Step S31: Acquire image viewing angle data corresponding to the power facility; Step S32: performing initial pose standard processing according to the image view angle data and the point cloud rendering model to generate initial pose alignment data; Step S33: performing joint rendering model pose alignment processing based on the initial pose alignment data and the point cloud surface reconstructed rendering model to generate a fused pose alignment model; Step S34: performing a multi-frame rendering point cloud spatiotemporal reconstruction alignment fitting process on the fused pose alignment model to generate a multi-frame rendering spatiotemporal alignment point cloud model; Step S35: performing image texture mapping area analysis based on the multi-frame rendering spatiotemporal alignment point cloud model to generate image texture mapping area data; Step S36: performing model texture enhancement mapping processing on the multi-frame rendered spatiotemporally aligned point cloud model based on the image texture mapping area data to generate an enhanced texture rendering model.

7. The cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 6 is characterized in that: Step S36 includes the following steps: Extract texture feature details based on image texture mapping area data and image viewing angle data to generate texture feature detail data; Performing image texture space mapping enhancement processing on texture feature detail data to generate enhanced texture mapping data; Based on the enhanced texture mapping data, the multi-frame rendering spatiotemporal aligned point cloud model is subjected to model texture enhancement mapping processing to generate an enhanced texture rendering model.

8. The cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing coordinated cloud-edge analysis based on the coordinated prior point cloud data to generate coordinated cloud-edge data; Step S42: performing rendering computing power thread estimation calculation based on the coordinated cloud-edge data to generate rendering computing power thread data; Step S43: Perform rendering coordination and lightweight deployment based on the rendering computing power thread data and the enhanced texture rendering model to generate a lightweight point cloud rendering model.

9. The cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 1, characterized in that: Step S5 includes the following steps: Step S51: performing rendering interaction progressive hierarchical analysis on the coordinated prior point cloud data to generate rendering interaction progressive hierarchical data; Step S52: performing rendering resource allocation calculation on the coordinated prior point cloud data according to the rendering interaction progressive level data to generate rendering resource data; Step S53: performing point cloud data progressive rendering scheduling based on the rendering interaction progressive level data and the estimated rendering resource data to generate point cloud progressive rendering scheduling data; Step S54: performing lightweight real-time rendering processing on the point cloud progressive rendering scheduling data based on the lightweight point cloud rendering model to generate lightweight real-time rendering data; Step S55: Obtain user behavior data; Step S56: performing point cloud data rendering feedback analysis based on user behavior data to generate rendering feedback data; Step S57: Optimize the cloud-edge rendering coordination transmission granularity according to the rendering feedback data to generate cloud-edge rendering coordination transmission granularity data; Step S58: Based on the cloud-edge rendering coordinated transmission granularity data, the lightweight real-time rendering data is cached in the cloud-edge coordination and fed back to the terminal.

10. A cloud-edge coordinated point cloud model lightweight rendering processing system, characterized by: For executing the cloud-edge coordinated point cloud model lightweight rendering processing method according to claim 1, the cloud-edge coordinated point cloud model lightweight rendering processing system comprises: A point cloud data coordination preprocessing module is used to obtain the point cloud data of power facilities, and perform point cloud coordination prior processing based on the point cloud data of power facilities to generate coordinated prior point cloud data; A rendering model construction module is used to construct a point cloud rendering model based on the coordinated prior point cloud data; calculate the point cloud surface coefficient based on the coordinated prior point cloud data to generate the point cloud surface coefficient; perform surface reconstruction processing on the point cloud rendering model based on the point cloud surface coefficient to generate a point cloud surface reconstruction rendering model; The rendering model enhancement module is used to obtain the image perspective data corresponding to the power facilities, and perform rendering model pose alignment processing based on the image perspective data, the point cloud rendering model, and the point cloud surface reconstruction rendering model to generate a fused pose alignment model; and perform model texture enhancement mapping processing based on the fused pose alignment model to generate an enhanced texture rendering model; The rendering model deployment module performs rendering coordination and lightweight deployment of the enhanced texture rendering model based on the coordinated prior point cloud data to generate a lightweight point cloud rendering model; The lightweight rendering and transmission module performs lightweight real-time rendering processing on the coordinated prior point cloud data based on the lightweight point cloud rendering model to generate lightweight real-time rendering data; obtains user behavior data; performs point cloud data rendering feedback analysis based on the user behavior data to generate rendering feedback data; optimizes the cloud-edge rendering coordinated transmission granularity based on the rendering feedback data to generate cloud-edge rendering coordinated transmission granularity data; and performs cloud-edge coordinated caching of the lightweight real-time rendering data based on the cloud-edge rendering coordinated transmission granularity data and feeds it back to the terminal.

Images