Method and system for constructing three-dimensional scene of booster station

Abstract

The invention discloses a method and a system for constructing a three-dimensional scene of a booster station. The method comprises the following steps: acquiring multi-view image data and point cloud data of an internal environment of the booster station; identifying equipment components in the image data to obtain a semantic segmentation result; mapping a semantic segmentation result to a three-dimensional point cloud space to obtain a three-dimensional point cloud model, and meanwhile, aggregating point cloud data of the same equipment to obtain a point cloud cluster; performing triangularization processing on the discrete point cloud cluster to obtain a point cloud curved surface model; matching a three-dimensional equipment model, and performing rigid body transformation on the three-dimensional equipment model and the point cloud curved surface model to obtain an equipment three-dimensional model; and optimizing the spatial position and shape parameters of the three-dimensional model of the equipment to complete the construction of the three-dimensional scene of the booster station. According to the invention, intelligent three-dimensional reconstruction of complex electric power facilities is realized, and a reliable digital basis is provided for equipment management and state monitoring.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a method and system for constructing a three-dimensional scene of a booster station. Background Art

[0002] The core technical challenge of three-dimensional digital modeling of booster stations is how to achieve high-precision and high-realism three-dimensional reconstruction in complex environments. This involves the fusion and processing of multi-source heterogeneous data, the extraction and mapping of semantic information, the accurate reconstruction of geometric shapes, and realistic rendering. Specifically, how to accurately map the semantic information in the two-dimensional image to the three-dimensional point cloud space, and on this basis, to achieve accurate segmentation and identification of equipment and components; how to accurately reconstruct the geometric shape and spatial position relationship of the equipment in the presence of occlusion, missing, etc.; for slender components such as pipelines and cables, how to accurately extract their spatial direction and perform geometric fitting; in the process of model reconstruction, how to effectively use multi-view image information to geometrically constrain and optimize the three-dimensional model to improve the reconstruction accuracy; in addition, how to restore the material properties of the equipment while ensuring geometric accuracy to improve the realism of the model. The solution to these problems requires not only consideration of the accuracy and robustness of the algorithm, but also a balance between computational efficiency and model complexity.

[0003] In practical applications, how to select appropriate reconstruction strategies and parameters for different types of equipment and components to adapt to the diversity and complexity of the booster station environment is a key issue that needs to be solved urgently. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a method and system for constructing a three-dimensional scene of a booster station to solve the problems existing in the above-mentioned prior art.

[0005] To achieve the above object, the present invention provides a method for constructing a three-dimensional scene of a booster station, comprising the following steps:

[0006] Acquire multi-view image data and point cloud data of the internal environment of the booster station;

[0007] Identify device components in the image data based on a semantic segmentation algorithm to obtain a semantic segmentation result;

[0008] Mapping the semantic segmentation result to a three-dimensional point cloud space to obtain a three-dimensional point cloud model, and at the same time, aggregating point cloud data of the same device together to obtain a point cloud cluster;

[0009] The discrete point cloud clusters are triangulated based on the triangulated meshing algorithm to obtain the point cloud surface model;

[0010] Match the 3D device model, perform rigid body transformation on the 3D device model and the point cloud surface model to obtain the device 3D model, and fit the 3D model of the slender member;

[0011] Optimize the spatial positions and shape parameters of the device 3D model and the 3D model of the slender member to complete the construction of the 3D scene of the booster station.

[0012] Optionally, the process of mapping the semantic segmentation result to the 3D point cloud space to obtain the 3D point cloud model and aggregating the point cloud data of the same device together to obtain the point cloud cluster includes:

[0013] Based on the semantic label information of the points in the 3D point cloud model, use the clustering analysis algorithm to cluster the point cloud, aggregate the point cloud data belonging to the same device together to obtain the point cloud cluster, extract the geometric feature information of the point cloud cluster, judge the specific device or component category to which each point cloud cluster belongs, and obtain the 3D point cloud model with device component semantic information.

[0014] Optionally, after obtaining the point cloud surface model by triangulating the discrete point cloud cluster based on the triangulation algorithm, the following steps are also included:

[0015] Based on the point cloud surface model, use the surface reconstruction algorithm to identify the missing areas and occlusion areas on the surface; if missing areas or occlusion areas are identified, repair them by interpolation and extrapolation methods to obtain the repaired point cloud surface model; based on the repaired point cloud surface model, judge whether the integrity meets the preset threshold. If the integrity index does not reach the preset threshold, further repair the missing areas and occlusion areas until the integrity index meets the requirements; based on the repaired point cloud surface model, use the mesh simplification algorithm to reduce the number of triangular patches and obtain the simplified point cloud surface model and output it.

[0016] Optionally, the process of matching the 3D device model and performing rigid body transformation on the 3D device model and the point cloud surface model to obtain the device 3D model includes:

[0017] Use the 3D shape analysis algorithm to extract the key geometric attributes of the devices in the booster station and construct the feature description vector; in the pre-constructed 3D model library, extract the geometric features of each 3D model to construct the feature description vector; calculate the similarity between the device feature description vector and the feature description vectors of each model, and obtain the 3D model with the highest similarity; register the 3D model with the highest similarity with the point cloud surface model through rigid body transformation, and according to the registration result, transfer the geometric details of the 3D model with the highest similarity to the point cloud surface model to obtain the geometricized device 3D model.

[0018] Optionally, the process of fitting the 3D model of the slender member includes:

[0019] Extract the skeleton curve of the slender component based on the skeleton extraction algorithm, and fit the skeleton curve with geometric elements to obtain the three-dimensional model of the slender component after fitting.

[0020] Optionally, the process of optimizing the spatial position and shape parameters of the three-dimensional model of the equipment and the three-dimensional model of the slender component includes:

[0021] Obtain the material property information of the booster station equipment and apply it to the corresponding three-dimensional model;

[0022] Then, adopt the geometric constraint optimization technology to project the three-dimensional model of the equipment and the three-dimensional model of the slender component onto the original image respectively, and optimize the spatial position and shape parameters of each model by minimizing the projection error to complete the construction of the three-dimensional scene of the booster station.

[0023] Optionally, after optimizing the spatial position and shape parameters of the three-dimensional model of the equipment and the three-dimensional model of the slender component, it further includes:

[0024] Compare the constructed three-dimensional scene of the booster station with the actual environment, evaluate the reconstruction accuracy, and perform local optimization on the areas with large errors to obtain a realistic three-dimensional scene of the booster station.

[0025] The present invention also provides a construction system for the three-dimensional scene of the booster station, which is used to implement the construction method of the three-dimensional scene of the booster station, including:

[0026] A data acquisition module, which is used to acquire multi-view image data and point cloud data of the internal environment of the booster station;

[0027] A semantic segmentation module, which is used to identify the equipment components in the image data based on the semantic segmentation algorithm to obtain the semantic segmentation result;

[0028] A model construction module, which is used to map the semantic segmentation result to the three-dimensional point cloud space to obtain a three-dimensional point cloud model. At the same time, aggregate the point cloud data of the same equipment together to obtain a point cloud cluster; and is used to perform triangulation processing on the discrete point cloud clusters based on the triangulation algorithm to obtain a point cloud surface model;

[0029] A model matching module, which is used to match the three-dimensional equipment model, perform rigid body transformation on the three-dimensional equipment model and the point cloud surface model to obtain the three-dimensional model of the equipment, and fit the three-dimensional model of the slender component;

[0030] A model optimization module, which is used to optimize the spatial position and shape parameters of the three-dimensional model of the equipment and the three-dimensional model of the slender component to complete the construction of the three-dimensional scene of the booster station.

[0031] The present invention also provides an electronic device, including: a memory and a processor; the memory is used for storing a program; the processor is used for executing the program to implement each step of the method for constructing a three-dimensional scene of a booster station.

[0032] The present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each step of the method for constructing a three-dimensional scene of a booster station is implemented.

[0033] Compared with the prior art, the present invention has the following advantages and technical effects:

[0034] The method of the present invention obtains multi-view images and point cloud data, performs semantic segmentation using deep learning, maps the image information to the point cloud space, and forms a three-dimensional point cloud model with semantic labels. Through clustering analysis and triangular meshing, the point cloud is converted into a continuous surface model, and the refined device and component models are reconstructed by using model library matching and geometric feature extraction. The present invention also integrates material attribute information, adopts an image-based geometric constraint optimization technology to improve the reconstruction accuracy, and finally performs realistic rendering and error optimization to obtain a high-precision and high-realistic three-dimensional digital model of the booster station. This method realizes the intelligent three-dimensional reconstruction of complex power facilities and provides a reliable digital foundation for equipment management and condition monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0036] Figure 1 is a schematic flow chart of the method for constructing a three-dimensional scene of a booster station according to an embodiment of the present invention;

[0037] Figure 2 is a schematic flow chart of obtaining a simplified point cloud surface model according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the drawings and combine the embodiments to detail this application.

[0039] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0040] Embodiment 1

[0041] AsFigure 1 As shown in the figure, in this embodiment, a method for constructing a three-dimensional scene of a booster station is provided, which specifically may include:

[0042] S101. Obtain multi-view high-definition images and point cloud data of the internal environment of the booster station. For the image data, use a semantic segmentation algorithm based on deep learning to identify and classify the devices and components in the images; for the point cloud data, remove the noise points and extract geometric elements such as planes and cylinders.

[0043] Obtain multi-view image data and point cloud data of the inside of the booster station, where the image data and the point cloud data have a spatial position correspondence relationship; for the image data, use a pre-trained semantic segmentation model for processing to obtain the category and position information of the device components in the image; for the point cloud data, improve the point cloud quality through filtering processing and extract the geometric elements in the point cloud; perform spatial position matching on the device component information obtained by image semantic segmentation and the geometric elements extracted from the point cloud, judge the device component category to which the geometric elements belong, and obtain the device component recognition result in the three-dimensional space; according to the device component recognition result, obtain the geometric dimensions and spatial position relationships of each device component, and judge the standardization of the internal device layout of the booster station; for the identified important device components, extract their surface texture features, input the texture features into a pre-established device component defect feature library for matching, and obtain the defect detection result of the device components; generate an analysis report on the internal environment of the booster station according to the device component recognition, layout analysis, and defect detection results.

[0044] Exemplarily, multi-view high-definition images and high-precision point cloud data are collected inside the step-up substation. The image resolution reaches 4000×3000 pixels, and the point cloud density reaches 1000 points per square meter. The pre-trained PSPNet semantic segmentation model is used to process the images, and the pixel-level accuracy rate reaches over 95%, enabling the recognition of 20 main equipment components such as transformers, circuit breakers, disconnectors, etc. Statistical filtering is performed on the point cloud data to remove noise and outliers, and geometric elements such as planes, cylinders, and spheres are extracted, with a fitting accuracy better than 1 cm. The iterative closest point algorithm is used to register the image semantic segmentation results with the point cloud geometric elements, and the spatial position error is less than 5 cm, thereby obtaining the device component recognition results in the three-dimensional space. Based on the recognized spatial position relationships of the device components, analyze whether layout parameters such as their spacing and elevation difference meet the requirements of electrical codes, and the accuracy rate of layout compliance judgment reaches over 90%. For important components such as transformer tanks and bushings, extract their surface texture features and perform similarity matching with a pre-established defect feature library. A support vector machine classifier can identify defects such as rust and oil stains on the component surface, and the recognition accuracy rate reaches 90%. Finally, integrating the device component recognition, layout analysis, and defect detection results, automatically generate an internal environment analysis report of the step-up substation, and visually display information such as the device component types, spatial layout, and health status through three-dimensional visualization, providing a decision-making basis for the intelligent operation and maintenance of the step-up substation.

[0045] S102. According to the semantic segmentation results, map the image data to the three-dimensional point cloud space to obtain a three-dimensional point cloud model with semantic labels. At the same time, through the clustering analysis algorithm, aggregate the point cloud data belonging to the same device or component together to form independent point cloud clusters.

[0046] According to the acquired image data, use the semantic segmentation algorithm to segment the image to obtain the semantic label information of different objects in the image. Map the image data with semantic labels to the three-dimensional point cloud space to obtain a three-dimensional point cloud model with semantic information. According to the semantic label information of the points in the three-dimensional point cloud model, use the clustering analysis algorithm to perform clustering processing on the point cloud. Through clustering analysis, aggregate the point cloud data belonging to the same object together to form point cloud clusters representing independent objects. Analyze each independent point cloud cluster and extract its geometric feature information, such as size, position, shape, etc. According to the extracted geometric features, judge the specific device or component category to which each point cloud cluster belongs to obtain a point cloud model with device component semantic information. Visualize the point cloud model with device component semantic information to obtain the spatial distribution and relative position relationships of different device components in the three-dimensional scene, providing a data basis for subsequent application analysis.

[0047] Exemplarily, after obtaining multi-view high-definition images of the internal environment of the booster station, a pre-trained PSPNet semantic segmentation model is used to process the images. The model is trained on the Cityscapes dataset, which contains 19 categories such as buildings, roads, vehicles, etc. Through model inference, the probability distribution of each pixel belonging to different categories can be obtained, and the category with the highest probability is taken as the semantic label of the pixel. The segmentation result is mapped into the corresponding three-dimensional point cloud space, and each point cloud carries the category information of its corresponding pixel, forming a point cloud model with semantics. Then, the DBSCAN clustering algorithm is used to cluster the point cloud. The clustering radius is set to 5 meters and the minimum number of points is set to 100. Points in the space that are close in distance and have the same category can be clustered into one cluster, representing an independent object instance. For each point cloud cluster, the PCA method is used to extract its geometric features such as the main direction, length, width, and height. According to its semantic category and scale characteristics, it is matched with a predefined device component model library to identify the specific category to which each point cloud cluster belongs, such as transformers, busbars, insulators, etc. Finally, the recognition result is rendered onto the original point cloud, and different components are given different colors to generate a three-dimensional visualization scene, clearly showing the equipment layout and spatial relationship inside the booster station, providing data support for subsequent refined analysis.

[0048] S103. For each point cloud cluster, use the triangular meshing algorithm to convert the discrete point cloud into a continuous point cloud surface model. During the triangulation process, the surface reconstruction algorithm is used to repair missing and occluded areas to improve the integrity of the point cloud surface model.

[0049] As Figure 2 shown, obtain the point cloud cluster data. For each point cloud cluster, use the triangular meshing algorithm to convert the discrete point cloud into a continuous point cloud surface model; according to the point cloud surface model, use the surface reconstruction algorithm to identify the missing areas and occluded areas on the surface; if missing areas or occluded areas are identified, then for these areas, perform repairs through methods such as interpolation and extrapolation to obtain the repaired point cloud surface model; according to the repaired point cloud surface model, calculate its integrity index and determine whether the integrity meets the preset threshold; if the integrity index does not reach the preset threshold, then return to step 3 to further repair the missing and occluded areas until the integrity index meets the requirements; according to the repaired point cloud surface model, use the mesh simplification algorithm to reduce the number of triangular patches while ensuring accuracy to obtain the simplified point cloud surface model; output the simplified point cloud surface model as the final continuous surface model.

[0050] Exemplarily, for each point cloud cluster, the Ball Pivoting Algorithm triangulation algorithm is adopted, and the sphere radius is set to 5 times the average point cloud spacing to ensure that the generated point cloud surface model has good continuity and smoothness. The Poisson surface reconstruction algorithm is used to analyze the point cloud surface model. Through implicit surface fitting with an octree depth of 8, missing areas and occluded areas larger than 5 square centimeters on the surface are identified. For the identified missing areas and occluded areas, an interpolation method based on radial basis functions is used for surface repair. The radial basis function selects the Gaussian kernel function, and the shape parameter is set to 2 to smooth the repaired surface. By calculating the Hausdorff distance before and after repair, the integrity index of the surface is obtained. If the integrity index is greater than 95%, it is considered that the repaired surface model meets the integrity requirements. The mesh simplification algorithm with quadratic error metric is used to simplify the repaired point cloud surface model at a simplification rate of 30%. On the premise that the surface accuracy error is less than 1 mm, the number of triangular patches is reduced to about 70% of the original, and the simplified point cloud surface model is obtained as the final geometric model of the equipment component.

[0051] Repair is carried out by methods such as interpolation and extrapolation to obtain the repaired point cloud surface model:

[0052] Obtain the data of the triangular mesh surface model to be repaired. For the defective or incomplete areas in the model, judge the boundaries of the defective areas. According to the known points on the boundary of the defective area, the interpolation method is used to determine the new point coordinates inside the defective area. If the boundary point data of the defective area is insufficient, the coordinates of the missing points are obtained by extrapolation according to the known points near the defective area. According to the new point coordinates obtained by interpolation or extrapolation, triangular patches are regenerated in the defective area to realize the mesh surface reconstruction of the defective area. Smooth the repaired mesh surface of the defective area to make it smoothly transition with the surrounding area and eliminate the repair traces. Stitch the repaired mesh surface of the defective area with the complete area of the original model to obtain the repaired complete point cloud surface model. Perform topological optimization on the repaired complete point cloud surface model to eliminate possible long and narrow triangular patches and improve the model quality.

[0053] Exemplarily, in order to repair the defective area in the point cloud surface model, it is first necessary to obtain the data of the model to be repaired. By analyzing the topological structure and geometric features of the model, the boundary of the defective area can be determined. For example, using edge detection algorithms such as the Canny operator or the Sobel operator to extract the edges of the model, and determining the boundary of the defective area according to the connectivity and closure of the edges. After obtaining the boundary of the defective area, interpolation methods can be used to determine the new point coordinates inside the defective area. Commonly used interpolation methods include linear interpolation, spline interpolation, etc. Taking linear interpolation as an example, assuming there are two known points P1(x1, y1, z1) and P2(x2, y2, z2) on the boundary of the defective area, then the coordinates of any point P(x, y, z) inside the defective area can be calculated by the formulas x = (1 - t)x1 + tx2, y = (1 - t)y1 + ty2, z = (1 - t)z1 + tz2, where t is the interpolation coefficient and its value range is [0, 1]. If the data of the boundary points of the defective area is insufficient, the missing point coordinates can be obtained by extrapolation methods, such as using the least squares method to fit the known points near the defective area to obtain the fitting surface equation, and then extrapolating the missing point coordinates according to the equation. After obtaining the new point coordinates by interpolation or extrapolation, triangular patches can be regenerated in the defective area to achieve mesh surface reconstruction. For example, using the Delaunay triangulation algorithm to triangulate the new points and the boundary points of the defective area to generate new triangular patches. In order to eliminate the repair traces, it is necessary to smooth the mesh surface of the repaired defective area. Commonly used methods include Laplacian smoothing, Loop subdivision, etc. Finally, the mesh surface of the repaired defective area is spliced with the complete area of the original model to obtain the repaired point cloud surface model. In order to further improve the model quality, topological optimization can be performed on the repaired model, such as using the edge collapse algorithm to merge the narrow triangular patches with a length less than a threshold (such as 1 mm), so as to obtain a point cloud surface model with a more reasonable topological structure.

[0054] According to the repaired point cloud surface model, a mesh simplification algorithm is adopted to reduce the number of triangular patches on the premise of ensuring the accuracy, and a simplified point cloud surface model is obtained:

[0055] Obtain the data of the repaired point cloud surface model, set a preset simplification threshold, and process the model data using a mesh simplification algorithm; use the edge collapse algorithm to reduce the triangular patches. The algorithm iteratively selects an edge, collapses it into a vertex, and updates the adjacent triangular patches at the same time; perform a quality assessment on the simplified model. By calculating the geometric error and normal deviation index between the simplified model and the original model, it is judged whether the simplified model meets the preset accuracy requirements. If the quality assessment result does not meet the accuracy requirements, adjust the simplification threshold and perform mesh simplification again until the accuracy requirements are met.

[0056] Exemplarily, when performing mesh simplification on the repaired point cloud surface model, first set the simplification threshold to 0.05, that is, the geometric error between the simplified model and the original model does not exceed 5% of the diagonal length of the bounding box of the original model. Then, adopt the mesh simplification algorithm based on edge collapse. By iteratively selecting the edges to be collapsed, collapse them into a vertex and update the adjacent triangular meshes. When selecting the edges to be collapsed, preferentially select the edges with the distance between the two endpoints less than 0.1, the included angle less than 10 degrees, and the curvature less than 5. At the same time, according to the curvature information of the triangular meshes, perform more mesh merging on the regions with curvature less than 2, and retain more meshes in the regions with curvature greater than 8. To improve the simplification efficiency, adopt the multi-resolution mesh simplification strategy. Downsample the original model 3 times to generate simplified models with resolutions of 1 / 2, 1 / 4, and 1 / 8 of the original model, and establish corresponding relationships between different resolution models. Propagate the simplification results of the high-resolution model to the low-resolution model to guide the simplification process. After the simplification is completed, evaluate the simplification quality by calculating the geometric error and normal deviation between the models before and after simplification. If the geometric error exceeds 0.05 or the normal deviation exceeds 15 degrees, adjust the simplification threshold to 0.03 and re-perform the simplification until the accuracy requirements are met. To accelerate the simplification process, adopt the GPU parallel computing method based on CUDA. Implement key steps such as edge collapse and error calculation as GPU kernel functions, and perform parallel computing on the NVIDIA GeForce GTX 1080 graphics card, shortening the simplification time from 2 hours on the CPU to 10 minutes on the GPU. Finally, output the simplified model in OBJ format and compress it using the ZIP algorithm to generate a simplified model file that is convenient for storage and transmission.

[0057] S104. According to the geometric characteristics of the device, retrieve the 3D device model with the highest matching degree from the 3D model library, and register it with the point cloud surface model through rigid body transformation to obtain a refined 3D device model. For slender components such as pipelines and cables, extract their spatial orientations through the skeleton extraction algorithm and fit them with geometric elements such as cylinders.

[0058] According to the geometric characteristics of the device, a three-dimensional shape analysis algorithm is used to extract its key geometric attributes and construct a feature description vector. For each three-dimensional model in the pre-constructed three-dimensional model library, its geometric features are extracted to construct a feature description vector. The similarity between the device feature description vector and the feature description vectors of each model in the model library is calculated to obtain the three-dimensional model with the highest similarity. The matched three-dimensional model is registered with the device point cloud surface model through rigid body transformation, and the transformation matrix is optimized to maximize the coincidence degree between the two. According to the registration result, the fine geometric details of the three-dimensional model are transferred to the point cloud surface model to obtain a geometrically refined three-dimensional model of the device. For slender components such as pipelines and cables, the central curve is extracted through a topological skeleton extraction algorithm to obtain its three-dimensional spatial orientation. According to the cross-sectional characteristics of the slender component, geometric elements such as cylinders and elliptical cylinders are used to sweep the skeleton curve to obtain a fitted three-dimensional model of the slender component.

[0059] Exemplarily, when extracting the key geometric attributes of the device, a three-dimensional shape analysis algorithm such as geometric moments and Fourier descriptors can be used. For example, by calculating the invariant geometric moments of the device point cloud, features invariant to translation, rotation, and scaling are extracted to construct a 72-dimensional geometric moment feature vector. For each three-dimensional model in the model library, the same geometric moment features are also extracted. When calculating the similarity, the Euclidean distance metric can be used, and the model with the smallest distance is selected as the matching result. After matching, the ICP algorithm is used for rigid body registration. By minimizing the sum of the squared distances between the two point clouds, the rotation matrix and translation vector are optimized so that the root mean square error after registration is less than 1 mm. When transferring geometric details, the normal vector field of the model can be interpolated onto the device point cloud surface for geometric enhancement. For slender components such as pipelines, the central curve is extracted through a topological skeleton, and then geometric elements such as cylinders and elliptical cylinders are fitted by the least squares method to sweep the skeleton curve to generate a fine three-dimensional model of the pipeline and perform smoothing processing to improve the surface quality.

[0060] S105. Obtain the material property information of the booster station equipment, including parameters such as color, texture, and reflectivity, and apply it to the corresponding three-dimensional model to improve the realism of the model.

[0061] Obtain the material property information of the booster station equipment, including parameters such as color, texture, reflectivity, etc., and store it in a preset material property database. For each 3D model of the booster station equipment, obtain the corresponding material property information from the material property database. According to the obtained material property information, determine the visual parameters such as color, texture, and reflectivity on the surface of each 3D model. Adopt a physically based rendering algorithm to apply the determined visual parameters to the surface of the corresponding 3D model to improve the realism of the model. By calculating the lighting and shadows of the 3D model, further enhance the three-dimensional sense and spatial sense of the model. Perform post-processing on the rendered 3D model, adjust parameters such as color balance and contrast to optimize the final visual effect. Integrate the processed 3D model into the booster station visualization system to achieve high-fidelity 3D visualization of the booster station equipment.

[0062] Exemplarily, in order to obtain the material property information of the booster station equipment, a high-resolution camera can be used to photograph the surface of the equipment to obtain an image containing information such as color and texture. Through image processing algorithms such as color histogram analysis and texture feature extraction, extract the color distribution and texture features on the surface of the equipment. At the same time, use a photometer to measure the reflectivity of the equipment surface to obtain reflectivity data at different angles and wavelengths. Store the extracted color, texture, and reflectivity parameters in a preset material property database. The database uses a relational database such as MySQL and is associated with the 3D model through the equipment ID. During the rendering process, for each 3D model of the equipment, query the corresponding color, texture, and reflectivity parameters from the material property database through the ID and apply them to the surface of the 3D model. Adopt a physically based rendering algorithm such as the ray tracing algorithm to simulate the interaction between light and the model surface and calculate the lighting and shadow effects on the surface to improve the realism of the model. By adjusting parameters such as the position and intensity of the light source in the 3D scene, optimize the distribution of lighting and shadows to further enhance the three-dimensional sense of the model. The rendered 3D model is optimized for visual parameters such as color and contrast through post-processing algorithms such as the color balance algorithm and the contrast enhancement algorithm to obtain the best visual effect. Finally, integrate the processed high-fidelity 3D model into the booster station visualization system, and users can observe the detailed features of the equipment in all directions through interactive operations such as rotation and zooming, improving the intuitive understanding and analysis ability of the equipment.

[0063] S106. For the precision control in the 3D reconstruction process, adopt the geometric constraint optimization technology based on images. Project the 3D model onto the original image and optimize the spatial position and shape parameters of the model by minimizing the projection error to further improve the reconstruction accuracy.

[0064] Obtain the three-dimensional model to be reconstructed and the corresponding set of original images, and establish the projection relationship between the three-dimensional model and the images; according to the projection relationship, project the three-dimensional model onto each original image to obtain the two-dimensional projection of the model on the image; calculate the geometric error between the model projection and the image features on each image, and construct an error function; use an optimization algorithm such as the gradient descent method to optimize the spatial position and shape parameters of the model by minimizing the cumulative projection error function; determine whether the projection error converges below a preset threshold, and if not, return to step 4 to continue iterative optimization; if the projection error has converged, output the optimized three-dimensional model to obtain a reconstruction result with higher geometric accuracy; perform visual display and quantitative evaluation on the reconstruction result to determine whether the accuracy of the optimized model meets the application requirements.

[0065] Exemplarily, first obtain the three-dimensional model to be reconstructed and the corresponding set of original images, and establish the projection relationship between the three-dimensional model and the images. For example, the pinhole camera model can be used to project each vertex on the three-dimensional model onto the two-dimensional image plane through the internal and external camera parameter matrices. According to the projection relationship, use a graphics library such as OpenGL to render the three-dimensional model onto each original image to obtain the two-dimensional projection contour of the model on the image. Then, use an edge detection algorithm such as the Canny operator to extract the edge features in the image, calculate the geometric error between the model projection contour and the image edge features on each image, and construct an error function. Common error metrics include the Euclidean distance, Hausdorff distance, etc. For example, the sum of the distances from each pixel point on the model projection contour to the nearest image edge pixel can be calculated as the value of the error function. Based on the constructed error function, use an optimization algorithm such as the gradient descent method to optimize the model. Taking the model vertex coordinates and camera parameters as optimization variables, continuously update the spatial position and shape parameters of the model by minimizing the sum of the projection error functions on all images to make it more consistent with the original image features. After each iterative optimization, determine whether the projection error converges below a preset threshold. For example, the convergence condition can be set that the value of the error function is less than 01 or the change rate is less than 1%. If not, continue iterative optimization until the convergence condition is reached or the maximum number of iterations is reached. Finally, output the optimized three-dimensional model to obtain a reconstruction result with higher geometric accuracy. Perform visual display on the reconstruction result, such as rendering the model into a realistic image or comparing it with the original image, and use quantitative indicators such as reprojection error and surface distance to evaluate the reconstruction accuracy of the model to determine whether the optimized model meets the application requirements.

[0066] S107. Perform realistic rendering on the reconstructed three-dimensional scene of the booster station, compare the three-dimensional scene with the actual environment, evaluate the reconstruction accuracy, and perform local optimization on the areas with large errors, such as increasing the shooting density of local images, optimizing the point cloud registration parameters, etc., and finally obtain a high-precision and high-realistic three-dimensional digital model of the booster station.

[0067] Obtain the reconstructed 3D scene data of the booster station, perform realistic rendering on the 3D scene to obtain the rendered 3D model of the booster station; obtain the image data of the actual environment of the booster station, compare the rendered 3D model with the actual environment image, calculate the difference degree between the two to obtain the reconstruction accuracy evaluation result; according to the reconstruction accuracy evaluation result, determine the area with large errors in the 3D model, and perform local optimization processing on this area; obtain the local image data of the error area, and obtain more local detail information by increasing the image shooting density; obtain the point cloud data of the error area, and improve the registration accuracy of the point cloud data by optimizing the point cloud registration parameters; perform fusion processing on the obtained local image data and the optimized point cloud data, update the data of the error area in the 3D model to obtain the locally optimized 3D model of the booster station; re-perform realistic rendering on the locally optimized 3D model to obtain the final high-precision and high-realistic 3D digital model of the booster station.

[0068] Exemplarily, in order to obtain a high-precision and high-realistic 3D digital model of the booster station, first, it is necessary to perform realistic rendering on the reconstructed 3D scene data of the booster station. By using a physical engine such as Unreal Engine or Unity, combined with parameters such as the materials and lighting of the booster station scene, perform realistic rendering on the 3D model to make its visual effect close to the real environment. After rendering, compare the rendered image with the actual environment image of the booster station, and use an image similarity algorithm such as SSIM (structural similarity) or PSNR (peak signal-to-noise ratio) to calculate the difference degree between the two. If the SSIM value is lower than 8 or the PSNR value is lower than 30 dB, it indicates that the reconstruction accuracy needs to be improved. Determine the area with large errors according to the evaluation result, such as key equipment such as transformers and insulators, and perform targeted local optimization. By increasing the image shooting density in this area, such as increasing the number of images from 1 per square meter to 4, obtain more local detail features. At the same time, optimize the point cloud registration parameters, such as using the ICP (Iterative Closest Point) algorithm to reduce the registration error from 5 cm to within 2 cm. Finally, fuse the optimized local images and point cloud data, update the geometric and texture information of the corresponding area in the 3D model, and re-perform realistic rendering. After the above process, a high-precision and high-realistic 3D digital model of the booster station can be obtained, providing a reliable data basis for subsequent operation and maintenance management and simulation analysis.

[0069] The present invention also provides a construction system for the 3D scene of the booster station for implementing the construction method of the 3D scene of the booster station, and the two correspond to each other in terms of effects, including:

[0070] A data acquisition module for acquiring multi-view image data and point cloud data of the internal environment of the booster station;

[0071] A semantic segmentation module, configured to identify device components in the image data based on a semantic segmentation algorithm to obtain a semantic segmentation result;

[0072] A model construction module, configured to map the semantic segmentation result to a three-dimensional point cloud space to obtain a three-dimensional point cloud model. Meanwhile, aggregate the point cloud data of the same device together to obtain a point cloud cluster; and perform triangulation processing on the discrete point cloud clusters based on a triangulation algorithm to obtain a point cloud surface model;

[0073] A model matching module, configured to match a three-dimensional device model, perform a rigid body transformation on the three-dimensional device model and the point cloud surface model to obtain a three-dimensional device model, and fit a three-dimensional model of a slender component;

[0074] A model optimization module, configured to optimize the spatial positions and shape parameters of the three-dimensional device model and the three-dimensional model of the slender component to complete the construction of a three-dimensional scene of a booster station.

[0075] Embodiment 2

[0076] The present invention also provides an electronic device, including: a memory and a processor; the memory is configured to store a program; the processor is configured to execute the program to implement each step of the method for constructing a three-dimensional scene of a booster station.

[0077] Embodiment 3

[0078] The present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each step of the method for constructing a three-dimensional scene of a booster station is implemented.

[0079] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for constructing a three-dimensional scene of a booster station, characterized in that, Including the following steps: Obtain multi-view image data and point cloud data of the internal environment of the booster station; Based on the semantic segmentation algorithm, identify the equipment components in the image data to obtain the semantic segmentation result; Map the semantic segmentation result to the three-dimensional point cloud space to obtain a three-dimensional point cloud model. At the same time, aggregate the point cloud data of the same equipment together to obtain a point cloud cluster; Based on the triangular meshing algorithm, perform triangularization on the discrete point cloud clusters to obtain a point cloud surface model; Match the three-dimensional equipment model, perform rigid body transformation on the three-dimensional equipment model and the point cloud surface model to obtain the equipment three-dimensional model, and fit the three-dimensional model of the slender component; Optimize the spatial positions and shape parameters of the equipment three-dimensional model and the three-dimensional model of the slender component to complete the construction of the three-dimensional scene of the booster station.

2. The method for constructing a three-dimensional scene of a booster station according to claim 1, wherein: The process of mapping the semantic segmentation result to the three-dimensional point cloud space to obtain a three-dimensional point cloud model and, at the same time, aggregating the point cloud data of the same equipment together to obtain a point cloud cluster includes: Based on the semantic label information of the points in the three-dimensional point cloud model, use the clustering analysis algorithm to perform clustering on the point cloud, aggregate the point cloud data belonging to the same equipment together to obtain a point cloud cluster, extract the geometric feature information of the point cloud cluster, judge the specific equipment or component category to which each point cloud cluster belongs, and obtain a three-dimensional point cloud model with equipment component semantic information.

3. The method for constructing a three-dimensional scene of a booster station according to claim 1, wherein: After performing triangularization on the discrete point cloud clusters based on the triangular meshing algorithm to obtain a point cloud surface model, it further includes: Based on the point cloud surface model, use the surface reconstruction algorithm to identify the missing areas and occlusion areas on the surface; if missing areas or occlusion areas are identified, repair them by interpolation and extrapolation methods to obtain a repaired point cloud surface model; based on the repaired point cloud surface model, judge whether the integrity meets the preset threshold. If the integrity index does not reach the preset threshold, further repair the missing areas and occlusion areas until the integrity index meets the requirements; based on the repaired point cloud surface model, use the mesh simplification algorithm to reduce the number of triangular patches, obtain a simplified point cloud surface model and output it.

4. The method for constructing a three-dimensional scene of a booster station according to claim 3, wherein: The process of matching the three-dimensional equipment model and performing rigid body transformation on the three-dimensional equipment model and the point cloud surface model to obtain the equipment three-dimensional model includes: Adopt a three-dimensional shape analysis algorithm to extract the key geometric attributes of the equipment in the booster station and construct a feature description vector; in the pre-constructed three-dimensional model library, extract the geometric features of each three-dimensional model to construct a feature description vector; calculate the similarity between the equipment feature description vector and the feature description vectors of each model, and obtain the three-dimensional model with the highest similarity; perform registration on the three-dimensional model with the highest similarity and the point cloud surface model through rigid body transformation, and according to the registration result, transfer the geometric details of the three-dimensional model with the highest similarity to the point cloud surface model to obtain a geometricized equipment three-dimensional model.

5. The method for constructing a three-dimensional scene of a booster station according to claim 1, wherein The process of fitting the three-dimensional model of the slender member includes: Extracting the skeleton curve of the slender member based on the skeleton extraction algorithm, and fitting the skeleton curve with geometric elements to obtain the three-dimensional model of the slender member after fitting.

6. The method for constructing a three-dimensional scene of a booster station according to claim 1, wherein The process of optimizing the spatial positions and shape parameters of the three-dimensional device model and the three-dimensional model of the slender member includes: Obtaining the material attribute information of the booster station equipment and applying it to the corresponding three-dimensional model; Then, using the geometric constraint optimization technology, projecting the three-dimensional device model and the three-dimensional model of the slender member onto the original image respectively, and optimizing the spatial positions and shape parameters of each model by minimizing the projection error to complete the construction of the three-dimensional scene of the booster station.

7. The method for constructing a three-dimensional scene of a booster station according to claim 6, wherein After optimizing the spatial positions and shape parameters of the three-dimensional device model and the three-dimensional model of the slender member, it further includes: Comparing the constructed three-dimensional scene of the booster station with the actual environment, evaluating the reconstruction accuracy, and performing local optimization on the areas with large errors to obtain a realistic three-dimensional scene of the booster station.

8. A construction system for a three-dimensional scene of a booster station, characterized in that, For implementing the method for constructing a three-dimensional scene of a booster station according to any one of claims 1-7, it includes: A data acquisition module, configured to acquire multi-view image data and point cloud data of the internal environment of the booster station; A semantic segmentation module, configured to identify the equipment components in the image data based on the semantic segmentation algorithm to obtain a semantic segmentation result; A model construction module, configured to map the semantic segmentation result to the three-dimensional point cloud space to obtain a three-dimensional point cloud model. At the same time, aggregating the point cloud data of the same equipment together to obtain a point cloud cluster; and configured to perform triangulation processing on the discrete point cloud clusters based on the triangulation algorithm to obtain a point cloud surface model; A model matching module, configured to match the three-dimensional device model, perform rigid body transformation on the three-dimensional device model and the point cloud surface model to obtain a three-dimensional device model, and fit the three-dimensional model of the slender member; A model optimization module, configured to optimize the spatial positions and shape parameters of the three-dimensional device model and the three-dimensional model of the slender member to complete the construction of the three-dimensional scene of the booster station.

9. An electronic device, characterized in that, It includes: A memory and a processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the method for constructing a three-dimensional scene of a booster station according to any one of claims 1-7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the method for constructing a three-dimensional scene of a booster station according to any one of claims 1-7.

Images