A method and system for 3D modeling of chemical industrial parks

Abstract

The application discloses a kind of method and system for three-dimensional modeling of chemical industry park, method includes: using unmanned aerial vehicle carries multi-lens tilt camera and monitoring sensor, obtains the multi-view image data of chemical industry park, positioning and pose data and real-time data of chemical plant equipment operation;Multi-view image data is processed, and sparse point cloud is generated, and three-dimensional point cloud is generated by dense matching and depth estimation;Three-dimensional point cloud is handled in block, and global point cloud data is obtained by splicing using point cloud registration algorithm;Surface reconstruction is carried out to global point cloud data, and three-dimensional grid model is generated, and model texture mapping is completed by texture mapping algorithm;Three-dimensional visual management platform is constructed, integrates three-dimensional grid model and real-time monitoring data, realizes park global visualization, risk visualization and chemical plant equipment facility intelligent management.

Description

Technical Field

[0001] This invention relates to the field of safety management and 3D modeling technology for chemical industrial parks, specifically a method and system for 3D modeling of chemical industrial parks. Background Technology

[0002] With continuous economic development, industrial agglomeration has become a new trend in global economic development. Chemical industrial parks have gradually become important carriers for the clustering of the chemical industry, encompassing petrochemicals, coal chemicals, and fine chemicals. my country's chemical industrial parks are mainly divided into five types: petrochemical industrial parks, fine chemical industrial parks, urban relocation industrial parks, enterprise expansion industrial parks, and resource-based chemical industrial parks. According to statistics from the management department, as of the end of 2022, there were 943 chemical industrial parks of various types nationwide, with 53 companies having an output value exceeding 50 billion yuan. The revenue of enterprises in these parks accounted for more than 50% of the total revenue of the national petrochemical industry, effectively promoting local economic growth and industrial structure optimization. However, excessive clustering of chemical enterprises can easily lead to the rapid aggregation of high-risk materials, high-risk equipment, high-risk production, high-risk storage, and high-risk operations, drastically increasing production safety risks. Information on chemical equipment, indoor and outdoor scenes of chemical plant areas, external topography, and surrounding building environment is still limited to two-dimensional paper documents and traditional monitoring video footage. Data retrieval is difficult, video viewing is cumbersome, and there is a lack of intuitive integrated three-dimensional models of the plant's indoor and outdoor areas. This is specifically reflected in the following aspects:

[0003] The storage management of hazardous chemicals is chaotic. The failure to adhere to the three-tiered standards of isolated storage, separate storage, and compartmentalized storage of hazardous chemicals has led to frequent instances of incompatible materials such as oxidants and reducing agents, strong acids and strong alkalis being stored together. In the event of an accident, this would not only cause enormous casualties and property damage but also have a long-term negative impact on the ecological environment.

[0004] The inventory of equipment and facilities in chemical industrial parks is unclear. Currently, there are significant data gaps in the real-time management of equipment and facilities in chemical industrial parks, most notably in the lag in digital mapping and inaccurate full lifecycle records. In particular, older equipment and facilities, subjected to high-temperature and high-pressure processes, are highly susceptible to leaks of toxic and harmful gases, causing large-scale pollution of water sources, air, and soil, as well as poisoning and fatal accidents.

[0005] The risk and hazard records in the chemical industrial park are outdated. Currently, data collection still relies on manually entered unstructured Excel spreadsheets, failing to achieve data interconnection with core production management systems such as the safety production execution system and enterprise resource planning (ERP). This results in untimely updates to equipment status and hazard identification data. Furthermore, the data update mechanism is significantly lagging; traditional manual inspections are time-consuming, labor-intensive, and have limited coverage, creating an efficiency bottleneck.

[0006] With the deep integration of next-generation information technologies such as the Internet of Things, big data, cloud computing, artificial intelligence, and 5G with the safety risk management of chemical industrial parks, traditional safety management models are no longer sufficient for comprehensive and real-time monitoring and management, failing to meet the demands of modern safety risk management in chemical industrial parks. Digitalization of chemical industrial parks utilizes next-generation information technologies. On one hand, it achieves dynamic interaction between DCS data and 3D models through the OPCUA protocol, and relies on blockchain technology to solidify equipment change records. On the other hand, it uses laser point cloud scanning to establish an equipment spatial database, constructing a digital 3D reconstruction system for chemical industrial parks. By establishing 3D scenes of chemical industrial parks, it provides auxiliary support for promoting the informatization, digitalization, networking, and intelligentization of safety risk management systems in chemical industrial parks.

[0007] Research has found that existing 3D modeling technologies are mostly applied in fields such as medicine, architecture, and transportation. In the chemical industry, the focus is on monitoring key factors such as temperature, pressure, liquid level, and harmful gases from a local perspective, such as pipeline modeling and storage tank modeling. There is a lack of comprehensive consideration of the entire chemical industrial park, which fails to provide strong technical support for the safety management and emergency response of the park. For example, Gu Haifeng's research did not consider the special characteristics of the park and neglected key data collection; Gao Chenxu did not model the entire area of ​​the park; the data collection method in patent CN103791887A is limited and the modeling process is not described in detail; and patent CN116894316A only focuses on chemical pipeline modeling. Summary of the Invention

[0008] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method and system for 3D modeling of chemical industrial parks. Its purpose is to achieve 3D modeling of equipment in chemical industrial parks and integrated management encompassing basic management, production management, and safety management through an integrated and intelligent digital twin management system. This provides support for intelligent management of safe production and daily operation and maintenance in the park, and explores visualization of safety management in chemical enterprises, offering a more intuitive operating method. This solves the technical problems existing in current chemical industrial park management, such as high difficulty in safety management, high safety risks, data silos and low system integration, as well as the limitations of existing 3D modeling research in the chemical field.

[0009] To achieve the above objectives, according to one aspect of the present invention, a method for three-dimensional modeling of chemical industrial parks is provided, the method comprising:

[0010] Using drones equipped with multi-lens tilt cameras and monitoring sensors, multi-view image data, positioning and attitude data and real-time monitoring data of chemical industrial parks are obtained;

[0011] The multi-view image data is processed to generate a sparse point cloud, and a three-dimensional point cloud is generated through dense matching and depth estimation.

[0012] The three-dimensional point cloud is divided into blocks and stitched together using a point cloud registration algorithm to obtain global point cloud data.

[0013] The global point cloud data is used to perform surface reconstruction to generate a three-dimensional mesh model, and the model texture mapping is completed through a texture mapping algorithm.

[0014] A 3D visualization management platform was built, integrating a 3D mesh model with real-time monitoring data, to achieve full-area visualization of the park, risk visualization, and intelligent management of chemical plant equipment and facilities.

[0015] As a further improvement and supplement to the above solution, the present invention also includes the following additional technical features.

[0016] Preferably, surface reconstruction is performed on the global point cloud data to generate a three-dimensional mesh model, including the following steps:

[0017] A multi-level point cloud segmentation scale is predefined, and the global point cloud data is decomposed into a multi-scale scale based on the multi-level point cloud segmentation scale to obtain a multi-level local point cloud dataset; according to the facility attributes of the chemical industrial park, features are extracted from the multi-level local point cloud dataset to obtain a multi-level local point cloud feature set.

[0018] The multi-level local point cloud feature set is used to traverse and match the 3D model library, and the multi-level local 3D model set is output.

[0019] By spatial registration of point clouds, the multi-level local 3D model set is fused in model space to output a local mesh model;

[0020] The local mesh model is projected onto the global point cloud data, and matching defect point cloud data is obtained by filtering.

[0021] Surface reconstruction is performed on the matched defect point cloud data to obtain a compensation mesh model;

[0022] The local mesh model is spatially modeled and compensated using the compensation mesh model to obtain the three-dimensional mesh model.

[0023] Preferably, the multi-level local point cloud feature set is used to traverse and match the 3D model library to output a multi-level local 3D model set, including the following steps:

[0024] Based on the facility attributes of the chemical industrial park, the models are networked and integrated to obtain multiple sample grid models;

[0025] Feature extraction is performed on the multiple sample grid models based on a preset feature index set to obtain the features of the multiple sample models;

[0026] The multiple sample mesh models and features of the multiple sample models are associated and stored to complete the data filling of the 3D model library;

[0027] After combining the features of the first-level local point cloud through multiple sample models, a matching algorithm is used to filter the models and locate the first-level local 3D model.

[0028] Similarly, the multi-level local point cloud feature set is used to traverse and match the 3D model library, and the multi-level local 3D model set is output.

[0029] Preferably, after combining the features of the first-level local point cloud through multiple sample model features, feature similarity calculation is used to filter models and locate the first-level local 3D model, including the following steps:

[0030] Calculate the similarity between the first-level local point cloud features and the multiple structural features of the multiple sample model features;

[0031] If the similarity of P structural features meets the preset similarity threshold, then the proportional deviation between the P sample model features and the first-level local point cloud features is calculated, and the proportional deviation of P models is output.

[0032] Based on the P normalized results of the similarity of the P structural features and the P model ratio deviations, P sample mesh models are serialized to resolve model conflicts and locate the first-level local 3D model.

[0033] Preferably, the multi-lens tilt camera includes one vertically downward lens and four side lenses tilted at 45°. The monitoring sensor includes a thermal imaging sensor and a gas detector. The gas detector is used to collect the concentration, temperature, and pressure data of at least one gas selected from H2, Cl2, CO, SO2, NO2, O3, VOCs, and NH3.

[0034] Preferably, dense matching employs stereo vision, structured light, or time-of-flight technology to obtain pixel-level depth information through parallax calculation, light pattern deformation analysis, or light pulse time-of-flight measurement.

[0035] Preferably, the point cloud registration algorithm includes the following steps:

[0036] Calculate the center points and covariance matrix of the source point cloud and the point cloud to be measured;

[0037] The principal component orientation of the point cloud is determined by principal component analysis.

[0038] The rotation and translation matrices are solved using singular value decomposition, and the point cloud to be measured is aligned to the coordinate system of the source point cloud.

[0039] Preferably, the 3D visualization management platform includes the following functional modules:

[0040] The data visualization module supports 2D / 3D scene switching, map operations, and chemical plant equipment attribute queries;

[0041] The dynamic monitoring module maps temperature, pressure, or gas concentration data to a 3D model in real time.

[0042] The personnel positioning module uses Internet of Things (IoT) technology to achieve multi-dimensional spatial positioning and historical trajectory playback.

[0043] Preferably, the drone supports custom route planning, including rectangular routes, circular routes, straight routes, or hand-drawn routes, and can set parameters including at least flight altitude, speed, or overlap rate.

[0044] According to another aspect of the present invention, a system for three-dimensional modeling of chemical industrial parks is provided, the system comprising:

[0045] The data acquisition module includes a drone platform, a five-lens tilt camera, a thermal imaging sensor, a gas detector, a GPS positioning device, and a total station, used to acquire images, videos, pictures, and process parameter data.

[0046] The data processing module includes a spatial processing unit, a dense matching unit, a point cloud segmentation and stitching unit, a surface reconstruction unit, and a texture mapping unit, which are used to generate high-precision 3D models.

[0047] The intelligent management module includes a 3D visualization engine, dynamic data interface, personnel positioning system and equipment ledger database, supporting real-time monitoring, risk assessment and full life cycle management of equipment;

[0048] The visualization module includes two-dimensional / three-dimensional scenes of the park, real-time monitoring videos, personnel and vehicle location tracking, equipment operation attributes, safety risk level distribution maps, and a real-time data statistical analysis interface.

[0049] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0050] Efficient and comprehensive data acquisition: Integrating drone oblique photography with multiple sensors, it enables the simultaneous acquisition of multi-source data such as images, videos, pictures, and process parameters, covering the entire park and improving acquisition efficiency by more than 50%.

[0051] High-precision 3D modeling: Through dense matching and intelligent stitching algorithms, the point cloud positioning accuracy reaches the centimeter level, and the model texture fit is improved by 30%, meeting the detailed requirements of security management.

[0052] Intelligent analysis and visualization: Real-time integration of monitoring data and 3D models supports equipment anomaly alarms, personnel location and risk classification display, improving the enterprise's level of intelligent management in terms of rapid perception, real-time monitoring, early warning, dynamic optimization and intelligent decision-making.

[0053] System integration and scalability: It adopts distributed storage and parallel computing, supports massive data processing, is compatible with multi-source device access, and provides a unified platform for the digital transformation of chemical industrial parks. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0055] Figure 1 This is a flowchart of a method for 3D modeling of a chemical industrial park provided in Embodiment 1;

[0056] Figure 2 This is a schematic diagram of visibility analysis in this embodiment 1;

[0057] Figure 3 This is a schematic diagram of occlusion detection in Embodiment 1;

[0058] Figure 4 This embodiment two provides a system flowchart for 3D modeling of chemical industrial parks. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0060] Example 1:

[0061] This embodiment provides a method for 3D modeling of chemical industrial parks, such as... Figure 1 As shown, the method includes:

[0062] S101: Utilizes drones equipped with multi-lens tilt cameras and monitoring sensors to acquire multi-view image data, positioning and attitude data, and real-time operating data of chemical equipment in chemical industrial parks.

[0063] In this first embodiment, a technical approach combining oblique photogrammetry modeling and manual modeling is primarily employed. This is achieved through four main methods: oblique photogrammetry data acquisition, video data acquisition, indoor and outdoor field data acquisition, and monitoring data acquisition.

[0064] Oblique photogrammetry data acquisition utilizes drones equipped with multi-lens high-resolution cameras to plan flight areas, automatically generate flight paths based on the overlap rate of flight directions, and execute flight missions to collect orthophoto and oblique image data. It overturns the limitations of traditional orthophotos, which can only be captured from a vertical angle. By mounting multiple sensors on the same flight platform, it simultaneously acquires images from five different angles: vertical, front, rear, left, and right, bringing users into a realistic and intuitive world that conforms to human vision. The images contain rich and accurate information, allowing for data mining. Furthermore, low-altitude ground photogrammetry can obtain near-ground high-resolution aerial survey images. With the assistance of a high-precision positioning and attitude determination system, each point on the image has three-dimensional coordinates, enabling measurement accuracy from centimeters to decimeters. Compared to orthophotos, it can also obtain more precise elevation accuracy, allowing direct measurement of the height of buildings and other ground features.

[0065] Video data acquisition can be achieved by utilizing real-time image transmission from the drone's image sensors during flight to mark specific points, lines, and areas on the map. Simultaneously, drone video footage can be streamed live using streaming media technology. Viewers can use rectangles to mark targets and track their movement, or mark specific points, lines, and areas. All marked points, lines, and areas can then be used as references for other personnel.

[0066] Indoor and outdoor data collection can be carried out using drones during flight. When the objects to be collected are elements such as building structure and pipeline routing, the main collection items include element texture, element geographical location information, element actual size, and other ancillary information.

[0067] Texture acquisition: High-resolution digital cameras are used to photograph and sample the texture of the outer contour of the elements. During the sampling process, data acquisition is ensured to be comprehensive and without blind spots.

[0068] Geographic location: Use GPS and other positioning devices to collect the coordinate information of the elements, and use the inflection points of the outer contour lines of the elements to establish the coordinate point information of the outer contour.

[0069] Actual dimensions: Using surveying tools such as a total station, the outer contour of the feature is measured, a two-dimensional CAD map is drawn, and the elevation of the current feature, the floor height of the building, and the feature attributes are measured.

[0070] Data collection can be achieved by using drones equipped with thermal imaging sensors to monitor the temperature, humidity, and operating conditions of key equipment in real time, day and night. Any anomalies can be promptly displayed and alarms triggered. Drones equipped with gas detectors can also be used to detect gases emitted from chemical plants, collecting data on the concentration, temperature, and pressure of at least one of the following gases: H2, Cl2, CO, SO2, NO2, O3, VOCs, and NH3. This allows for comprehensive monitoring of potential safety hazards in chemical enterprises, achieving effective supervision and prevention.

[0071] S102: Perform data processing on the multi-view image data to generate a sparse point cloud, and generate a three-dimensional point cloud through dense matching and depth estimation.

[0072] A sparse 3D point cloud is generated by feature point matching and bundle adjustment in space. These point clouds represent the 3D positions of feature points in the image. However, these points cannot form the data required for the triangular mesh of the 3D model reconstruction. Given the image camera matrix, corresponding points in two images are found, and the 3D coordinates of the corresponding ground features can be obtained using bundle adjustment. To obtain enough pairs of corresponding points, dense matching is performed.

[0073] Depth estimation is typically performed after feature extraction, sparse point cloud generation, and dense matching. Dense matching provides detailed image pairs or multi-view data for depth estimation. The goal of depth estimation is to obtain depth information for each pixel in order to generate a high-precision 3D point cloud, supporting subsequent operations such as point cloud stitching and surface reconstruction.

[0074] S103: The three-dimensional point cloud is divided into blocks and stitched together using a point cloud registration algorithm to obtain global point cloud data.

[0075] In this first embodiment, block processing is equivalent to block modeling. Block modeling is a technique for optimizing the 3D modeling process, particularly suitable for handling large-scale or complex scenes. The core idea is to divide the entire 3D model or point cloud into multiple smaller blocks for block-by-block processing and optimization. Specifically, by applying a 3D point cloud stitching algorithm, the gaps in data collection by the UAV can be effectively utilized to independently model each region, while reducing the computational load required for each modeling step. This method allows for the complete modeling of the entire region during UAV descent, improving efficiency. In subsequent steps, these independent block data are automatically stitched together into a single 3D model, forming a complete model of the flight target area.

[0076] Block-based modeling provides a structured data foundation for subsequent surface reconstruction and texture mapping, allowing these steps to be performed independently on each block before merging, thus improving the overall model quality and consistency. Furthermore, block-based modeling reduces computational complexity and memory requirements, making the processing smoother and more stable.

[0077] Block-based modeling also significantly improves modeling efficiency, transforming the exponentially increasing time requirements of traditional methods into linear growth. It allows for real-time modeling during UAV flight and generates high-precision 3D models through efficient block-level processing and stitching. Overall, block-based modeling not only improves modeling speed and accuracy but also optimizes the use of computing resources, providing an effective solution for 3D modeling of large-scale and complex scenes.

[0078] S104: Perform surface reconstruction on the global point cloud data to generate a three-dimensional mesh model, and complete the model texture mapping through a texture mapping algorithm.

[0079] Texture mapping is a parametric problem of object surfaces. In oblique photogrammetry, texture mapping, also known as texture mapping, refers to applying textures to a pre-built 3D model mesh to make the model more realistic. The oblique model is generated from 2D images, and texture mapping naturally uses the images used to generate the model. Model mapping is essentially a mapping process from 2D space to 3D space. Because the 3D model is obtained by matching, spatially refining, and generating point clouds from images, the 2D-to-3D mapping relationship already exists. The camera matrix for each image is obtained during the generation of the dense point cloud. However, a problem to consider is that points on the model may have multiple texture sources, corresponding to multiple photos. Generally, images are selected as texture mapping source images by judging occlusion and visibility to achieve the best texture mapping effect.

[0080] S105: Construct a 3D visualization management platform that integrates 3D mesh models with real-time monitoring data to achieve full-area visualization of the park, risk visualization, and intelligent management of chemical plant equipment and facilities.

[0081] In conjunction with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, the multi-lens tilt camera includes one vertically downward lens and four side lenses tilted at 45°. The monitoring sensor includes a thermal imaging sensor and a gas detector. The gas detector is used to collect the concentration, temperature and pressure data of at least one gas selected from H2, Cl2, CO, SO2, NO2, O3, VOCs and NH3.

[0082] In this first embodiment, the image data acquired during aerial photography is post-processed using the lens's built-in software. Because the tilting camera is equipped with five cameras, each tilted at a 45° angle in different directions, inconsistencies in intensity and light contrast can occur at the moment of exposure. This can lead to different brightness levels and colors of the same feature in images taken from or near each camera, affecting subsequent processing and modeling. Oblique image data acquisition involves using multiple sensors mounted on the same flight platform to simultaneously acquire vertical and oblique images and location information of features and terrain from various angles. Images taken perpendicular to the ground are called orthographic films, while images taken with the lens at a certain angle to the ground are called oblique films. Oblique photography cameras typically use a combination of five cameras (five lenses), with one camera pointing vertically downwards and the other four tilted at a certain angle.

[0083] In conjunction with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, dense matching employs stereo vision, structured light, or time-of-flight technology, and obtains pixel-level depth information through parallax calculation, light pattern deformation analysis, or light pulse time-of-flight measurement.

[0084] Stereo vision is a technique that uses two or more cameras to capture the same scene from different perspectives, mimicking the principles of the human visual system to reconstruct 3D scenes and measure depth. The core of stereo vision lies in disparity estimation and triangulation.

[0085] First, two images of the same scene are captured using a binocular camera system, with the two cameras positioned at a certain baseline spacing. Next, feature points are located and matched in the left and right images to calculate a disparity map. The disparity map represents the disparity value of each pixel in the image, i.e., the horizontal distance between corresponding points in the left and right images. Using the disparity value and the camera baseline length, the depth information of the object can be calculated according to a formula. Finally, the depth information is converted into a 3D point cloud to generate a 3D model for further analysis and processing.

[0086] Key steps in stereo vision include: camera calibration, estimating the camera's intrinsic parameters (such as focal length, principal point position, and distortion coefficients) and extrinsic parameters (i.e., the relative position and pose between the two cameras); image correction, transforming the left and right images to parallel viewpoints, eliminating geometric distortions of the cameras, and ensuring that points on the same horizontal line in the left and right images have the same ordinate; disparity map generation, calculating the matching cost between the left and right images, using a cost function (such as absolute difference or squared difference) to generate a disparity map, and reducing mismatches and noise through optimization techniques; and depth map generation, generating a depth map based on the disparity map and camera parameters, representing the depth information of each pixel. These steps ensure that the stereo vision system can accurately reconstruct 3D scenes.

[0087] Structured light is a technique that obtains the three-dimensional shape of an object by projecting a known light pattern (such as stripes or grids) onto the object's surface and capturing the deformed light pattern using an image sensor. The difference between structured light and stereo vision is that structured light requires a projector to project grating stripes. The structured light workflow includes light source projection, image acquisition, feature matching, and 3D reconstruction.

[0088] In the light projection stage, a structured light projector projects light of a known pattern onto the object's surface. Next, a camera captures deformed images of the light pattern on the object's surface, and these images are matched against the original pattern. Finally, the object's 3D shape is calculated based on the deformation of the light pattern. Structured light technology offers advantages in high-precision measurement and real-time data acquisition, making it suitable for applications such as 3D scanning, industrial inspection, and robot vision. However, it is sensitive to ambient lighting and surface characteristics, and is generally suitable for measuring small to medium-sized objects.

[0089] Time-of-flight (TOF) technology is a ranging technique based on the time-of-flight of light pulses. It calculates the distance to an object by emitting a light pulse and measuring the time it takes for it to reflect back, thus providing real-time depth information. Its operation includes light pulse emission, reflection, time measurement, and depth calculation. In the light pulse emission phase, the transmitter emits a short pulse of light, which is reflected back to the receiver after encountering an object. By measuring the time it takes for the light pulse to travel from emission to reception, the distance to the object can be calculated, and a depth map or 3D point cloud can be generated.

[0090] The 3D measurement technology of time-of-flight technology is similar to the ultrasonic system of bats, timing the round trip of light spots and bouncing them back to the sensor from the surface of the target person.

[0091] In conjunction with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, the point cloud registration algorithm includes the following steps:

[0092] S201: Calculate the center point and covariance matrix of the source point cloud and the point cloud to be measured.

[0093] In this first embodiment, principal component analysis (PCA) is used to register point cloud data from multiple point cloud images. Let the set of n-dimensional point cloud data in a certain point cloud image be q = {q1, q2, ... q...}. n}, construct the source point cloud model matrix Q based on the x, y, and z axes positions of the point cloud data within q in the original coordinate space:

[0094]

[0095] Using the source point cloud model matrix Q as the reference point cloud, establish a reference point cloud set r = {r1, r2, ... r}. m} and the m-dimensional point cloud matrix R to be measured:

[0096]

[0097] Calculate the point cloud centers O of the source point cloud model matrix Q and the point cloud matrix R to be measured. Q O R :

[0098] At this point, the covariance matrix cov of the point cloud within the R and Q matrices Q , cov R The calculation process is as follows:

[0099]

[0100] Where T is the matrix transpose symbol.

[0101] S202: Determine the principal component orientation of the point cloud through principal component analysis.

[0102] Let μ = (μ x ,μ y ,μ z Let ) be the eigenvectors of the matrix. According to the principal component analysis point cloud configuration algorithm, cov·Q=μQ, from which the eigenvectors of the two matrices Q and R can be derived.

[0103] The eigenvector μ of the source point cloud model matrix Q x ,μ y ,μ z Establish a three-dimensional spatial coordinate system as the direction of the three-dimensional spatial coordinates. This is the final registration space for the point cloud image.

[0104] S203: Solve the rotation and translation matrices using singular value decomposition to align the point cloud to be measured to the source point cloud coordinate system.

[0105] The singular value decomposition algorithm is used to obtain the translation matrix U and rotation matrix S between the source point cloud and the point cloud to be tested. Based on the inverse operation of the matrix, point cloud transformation is performed on all points in the point cloud data to be tested.

[0106] After obtaining the final registration spatial coordinate system for the point cloud image, the transformation matrix of each point cloud image is calculated by combining the point cloud center calculation method and the singular value decomposition algorithm. The data points in the point cloud images taken from multiple angles are then aligned to the final registration spatial coordinate system, so that the originally scattered and fragmented point cloud data has spatial consistency and avoids the occurrence of misalignment problems in the subsequent graphic stitching process.

[0107] In this first embodiment, the point cloud registration algorithm includes the following steps:

[0108] Extract local geometric features (such as corners and edges) and global structural features (such as the cylindrical outline of a storage tank) from the point cloud, generate descriptor-coded neighborhood information such as FPFH and SHOT, and use KD-Tree or RANSAC to establish the correspondence between feature points and eliminate mismatches.

[0109] Preliminary alignment of point clouds based on 4PCS or semantic matching reduces pose differences, solves the problem of initial position deviation in large-scale scenes, and provides optimized initial values ​​of transformation matrices for fine registration.

[0110] The ICP algorithm is used for iterative optimization. The optimal rigid body transformation (rotation + translation) is calculated through nearest point search and SVD decomposition. The robust loss function or color information is combined to improve the noise resistance until the point cloud alignment error converges to the sub-millimeter level.

[0111] By fusing multi-viewpoint clouds and detecting closed loops to correct accumulated errors, global consistency is optimized using pose graphs, and accuracy is quantified through RMSE and overlap rate.

[0112] In this first embodiment, texture mapping includes visibility determination, occlusion detection verification, and texture fusion, and includes the following steps:

[0113] The visibility analysis calculates the angle between the normal of the triangular facet and the line connecting the camera center, filters visible images within the range of 0° to 90°, and performs a visibility determination on each triangular facet in the 3D mesh model.

[0114] a) Calculate the angle θ between the normal vector n of the triangular facet and the line-of-sight vector v from the center of the camera to the centroid of the facet, and establish the visibility discrimination condition: cosθ≥∈, where ∈ is set as an adjustable threshold of 0≤ε≤1, preferably ε=0 corresponding to θ=90°;

[0115] b) Construct a candidate image set, where N is the total number of photo stations;

[0116] c) Calculate the weights based on the cosine of the included angle (cosθ) and the image resolution factor (ρk), and select the top M images with the largest weights as the effective visible image set.

[0117] The occlusion detection is based on an elevation ray tracing algorithm to determine whether the line connecting a ground point and the camera center is occluded. Based on elevation spatial analysis using a digital surface model, pixel-by-pixel occlusion verification is performed on the candidate image set V.

[0118] a) Establish the line-of-sight equation from ground point P(x,y,z) to the photography center Ok: t∈[0,1];

[0119] b) Perform 3D ray tracing along the line of sight L within the digital surface model data space, using an adaptive step size strategy.

[0120] For the effective image set V' that passes visibility analysis and occlusion detection, texture weight allocation based on projected area is applied:

[0121] a) Calculate the area Ak of ​​the projected region of each triangular facet in each valid image;

[0122] b) Establish a hybrid weighting function;

[0123] c) Perform multi-resolution texture blending, band-weighted synthesis in Laplacian pyramid space, eliminate seams and preserve high-frequency details.

[0124] In conjunction with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, the global point cloud data is reconstructed to generate a three-dimensional mesh model. Step S104 further includes the following steps:

[0125] S1041: Predefine a multi-level point cloud segmentation scale, and perform multi-scale decomposition on the global point cloud data based on the multi-level point cloud segmentation scale to obtain a multi-level local point cloud dataset.

[0126] S1042: Based on the facility attributes of the chemical industrial park, perform feature extraction on the multi-level local point cloud dataset to obtain a multi-level local point cloud feature set;

[0127] S1043: Use the multi-level local point cloud feature set to traverse and match the 3D model library, and output the multi-level local 3D model set;

[0128] S1044: By spatial registration of point clouds, the multi-level local 3D model set is fused in model space to output a local mesh model;

[0129] S1045: Project the local mesh model onto the global point cloud data, and filter to obtain matching defect point cloud data;

[0130] S1046: Perform surface reconstruction on the matching defect point cloud data to obtain a compensation mesh model;

[0131] S1047: The local mesh model is spatially modeled and compensated using the compensation mesh model to obtain the three-dimensional mesh model.

[0132] In conjunction with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, the multi-level local point cloud feature set is used to traverse and match the 3D model library to output a multi-level local 3D model set. Step S1043 further includes the following steps:

[0133] S1043a: Based on the facility attributes of the chemical industrial park, the model is networked and integrated to obtain multiple sample grid models;

[0134] S1043b: Perform feature extraction on the multiple sample grid models based on a preset feature index set to obtain multiple sample model features;

[0135] S1043c: Associate and store the multiple sample mesh models and multiple sample model features to complete the data filling of the three-dimensional model library;

[0136] S1043d: After combining the features of the first-level local point cloud through multiple sample model features, a matching algorithm is used to filter the model and locate the first-level local 3D model.

[0137] S1043e: Similarly, the multi-level local point cloud feature set is used to traverse and match the 3D model library, and the multi-level local 3D model set is output.

[0138] In conjunction with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, after traversing and combining the features of the first-level local point cloud, the features of the multiple sample models are combined, and feature similarity calculation is used to filter the models and locate the first-level local 3D model. Step S1043d further includes the following steps:

[0139] S1043d-1: Calculate the similarity between the first-level local point cloud features and the multiple structural features of the multiple sample model features;

[0140] S1043d-2: If the similarity of P structural features meets the preset similarity threshold, then the proportional deviation between the P sample model features and the first-level first local point cloud features is calculated, and the proportional deviation of the P models is output.

[0141] S1043d-3: Based on the P normalized results of the similarity of the P structural features and the P model ratio deviations, serialize the P sample mesh models to resolve model conflicts and locate the first-level first local three-dimensional model.

[0142] It should be understood that directly reconstructing the surface of the global point cloud data to generate a three-dimensional mesh model has the drawbacks of excessive consumption of computing resources of the modeling software and long modeling time. Based on this, this embodiment combines the publicly available models of existing chemical industrial parks to optimize the modeling computing resources and time consumption.

[0143] Specifically, a multi-level point cloud segmentation scale is predefined based on the scale and complexity of the chemical industrial park. The global point cloud data is then decomposed into multi-level local point cloud datasets at different levels based on the multi-level point cloud segmentation scale. This multi-scale decomposition facilitates subsequent targeted processing and lays the foundation for subsequent feature extraction and model matching.

[0144] Based on the facility attributes of the chemical industrial park, such as buildings, pipelines, and equipment, feature extraction is performed on the decomposed multi-level local point cloud dataset. The extracted features may include geometry, size, and texture, forming a multi-level local point cloud feature set, which provides crucial information for subsequent model matching.

[0145] By utilizing a multi-level local point cloud feature set, a traversal matching process is performed in a 3D model library. By comparing the features of the feature set with those of each model in the model library, the 3D model that best matches the local point cloud is found, and a multi-level local 3D model set is output. This enables rapid modeling of local point clouds and reduces the workload of modeling from scratch.

[0146] The specific technical implementation process for determining multi-level local 3D model sets through feature matching is as follows:

[0147] Based on the facility attributes of the chemical industrial park, multiple sample grid models are integrated through network access. These sample grid models can be derived from existing publicly available models of chemical industrial parks or other reliable model sources, providing a reference for subsequent model matching.

[0148] For multiple integrated sample mesh models, features are extracted based on a pre-defined feature index set to obtain the feature set for each sample mesh model. The pre-defined feature index set may include geometric features, texture features, size features, etc., which are used for subsequent matching with local point cloud features.

[0149] Multiple sample mesh models are associated and stored with their corresponding sample model features (sets) to form a complete 3D model library. This step ensures that each model in the library comes with a corresponding feature description, facilitating subsequent feature matching.

[0150] The first-level local point cloud features are traversed and combined with features from multiple sample models, and a matching algorithm is used for filtering. By comparing indicators such as the similarity between features, the sample model that best matches the first-level local point cloud features is found, thereby locating the first-level local 3D model.

[0151] The process of finding the sample model that best matches the first-level local point cloud features involves a matching conflict problem between multiple sample models. This conflict problem is resolved through the following process.

[0152] Calculate the structural feature similarity between the first-level local point cloud features and the features of multiple sample models. The calculation of structural feature similarity is a conventional technique, which can be based on features such as geometric shape, size, and texture. Structural feature similarity is used to measure the degree of similarity between the local point cloud and the sample model.

[0153] If P sample model features and local point cloud features have structural feature similarity that meets a preset similarity threshold, then the proportional deviation between these P sample model features and local point cloud features is further calculated. The proportional deviation is used to measure the difference between the sample model and the local point cloud in terms of global structural size, and P model proportional deviations are output.

[0154] Based on the normalized results of P structural feature similarities and P model scale deviations, P sample mesh models are serialized. The normalization results are used to comprehensively evaluate the similarity and size matching of the models. By resolving model conflicts through serialization, the most suitable first-level local 3D model is finally located.

[0155] Similarly, the multi-level local point cloud feature set is used to traverse and match the 3D model library, and the multi-level local 3D model set is output.

[0156] Point cloud spatial registration is performed on the multi-level local 3D model set obtained by matching, aligning different local models to the same spatial coordinate system, and then spatial fusion is performed to generate a spatially connected local mesh model, so that each local model is seamlessly connected in space to form a coherent overall model.

[0157] The fused local mesh model is projected onto the original global point cloud data. By comparison, point cloud data that is not covered by the model or is not accurately matched is identified, i.e., defective point cloud data, which provides a basis for subsequent supplementary modeling.

[0158] For the selected matching defect point cloud data, surface reconstruction is performed to generate a compensation mesh model for compensating for defects in the local mesh model coverage. This step aims to repair defects in the model or supplement uncovered parts, improving the model's integrity and accuracy.

[0159] The compensated mesh model is spatially fused with the previous local mesh model, and the local mesh model is compensated to obtain a complete three-dimensional mesh model.

[0160] Compared to directly reconstructing the surface from the global point cloud data to generate a 3D mesh model, this embodiment significantly improves modeling efficiency, reduces the computational resource requirements of the modeling process, and ensures the modeling accuracy of the 3D mesh model. In conjunction with this embodiment, there is also a preferred implementation scheme, specifically, texture mapping includes visibility analysis and occlusion detection:

[0161] The visibility analysis filters visible images within the range of 0° to 90° by calculating the angle between the normal of the triangular facet and the line connecting the center of the image.

[0162] On a 3D mesh model, the normal to a polygon forms an angle with the line connecting the center of the camera to the center of the image. This angle is used to determine whether the image is visible on that polygon. For example... Figure 2 As shown, angle θ is the angle between the normal ON of the triangular facet and the line connecting the center of the camera to the center of the image OS. Within the range of 0° to 90°, the triangular facet is visible on the image; outside this range, it is invisible.

[0163] Occlusion detection is based on an elevation ray tracing algorithm to determine whether the line connecting a ground point and the center of the camera is occluded.

[0164] Occlusion detection, such as Figure 3 As shown, points A and B are projected onto the image plane at point a. From the camera center, point A is visible (point V in the visible point range), while point B is obscured by A and invisible (point O in the visible point range). Assuming a camera ray corresponds to multiple points (i.e., multiple 3D points projected into the image correspond to one image point), the point closest to the camera center is the visible point, and the rest are invisible.

[0165] In conjunction with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, the three-dimensional visualization management platform includes the following functional modules:

[0166] The data visualization module supports switching between 2D / 3D scenes, map operations, and querying the attributes of chemical plant equipment.

[0167] Basic map operations: Load the 3D model onto the 3D map according to the actual location information, and use the mouse to view the surrounding environment inside and outside the factory area, as well as zoom in, zoom out, rotate, and pan the scene.

[0168] Navigation and Positioning: The management object information is presented in a tree-like directory according to the chemical enterprise classification standards, making it easy for users to understand the management hierarchy and quickly access the list of management objects of interest. The positioning function allows the view to be directly switched to the user's target.

[0169] It can switch between two-dimensional and three-dimensional display modes. After entering the three-dimensional mode, you can enter the interior of the model building and manage objects down to each floor, room, facility, equipment and even sensor.

[0170] Layer management: Allows for hierarchical management of spatial data, objects, and models.

[0171] It provides an intuitive, realistic, and accurate display of the chemical plant's external topography, surrounding buildings, and the distribution of various facilities and equipment within the plant, as well as the production organization. Users can browse the entire chemical plant on computers and mobile devices.

[0172] The dynamic monitoring module maps data such as temperature, pressure, and gas concentration to a 3D model in real time.

[0173] Dynamic Data Visualization Module: Provides dynamic visualization of various real-time data within the chemical industrial park. It integrates data streams from various sensors and monitoring equipment through data interfaces, mapping information such as temperature, pressure, and gas concentration onto a 3D model in real time. This module supports custom data layers and multi-dimensional data display, allowing users to select different data views according to their needs, enabling real-time monitoring and analysis of the park's operational status.

[0174] The personnel positioning module uses Internet of Things (IoT) technology to achieve multi-dimensional spatial positioning and historical trajectory playback.

[0175] By utilizing IoT technology, facial recognition, AI video surveillance, and other intelligent equipment, the system monitors, identifies, tracks, and analyzes people entering and exiting the park, obtaining real-time data on population movement within the area. Based on the positioning needs of different areas within the park, it achieves multi-dimensional spatial positioning, enabling personnel supervision and data statistical analysis. Furthermore, based on a 3D visualization scene, it dynamically plays historical personnel trajectories. Monitoring personnel can quickly locate an individual using image search and can select any staff member to view their real-time location and work status, thus tracking the staff's behavioral paths over any given time period.

[0176] In this first embodiment, the UAV supports custom route planning, including rectangular routes, circular routes, straight routes, or hand-drawn routes, and can set parameters including at least flight altitude, speed, or overlap rate.

[0177] Example 2:

[0178] This second embodiment provides a system for 3D modeling of chemical industrial parks, such as... Figure 4 As shown, the system includes:

[0179] The data acquisition module includes a drone platform, a five-lens tilt camera, a thermal imaging sensor, a gas detector, a GPS positioning device, and a total station, used to acquire images, videos, pictures, and process parameter data.

[0180] The data processing module includes a spatial processing unit, a dense matching unit, a point cloud segmentation and stitching unit, a surface reconstruction unit, and a texture mapping unit, which are used to generate high-precision 3D models.

[0181] The intelligent management module includes a 3D visualization engine, dynamic data interface, personnel positioning system and equipment ledger database, supporting real-time monitoring, risk assessment and full life cycle management of equipment;

[0182] The visualization module includes two-dimensional / three-dimensional scenes of the park, real-time monitoring videos, personnel and vehicle location tracking, equipment operation attributes, safety risk level distribution maps, and a real-time data statistical analysis interface.

[0183] The intelligent management module also includes a distributed storage system and a parallel computing framework for processing massive point cloud data and image data, and supports an immersive interactive interface for multiple terminals.

[0184] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as Embodiment 1 of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0185] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0186] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for 3D modeling of a chemical park, characterized in that the method The method comprises the steps of: acquiring multi-view image data, positioning and orientation data and real-time monitoring data of a chemical industrial park by using a multi-lens tilt camera and monitoring sensors mounted on a UAV; performing data processing on the multi-view image data to generate sparse point clouds, and generating three-dimensional point clouds through dense matching and depth estimation; performing block processing on the three-dimensional point clouds, performing splicing through a point cloud registration algorithm, and obtaining global point cloud data; performing surface reconstruction on the global point cloud data to generate a three-dimensional mesh model, and completing texture mapping of the model through a texture mapping algorithm; constructing a three-dimensional visualization management platform, integrating the three-dimensional mesh model and real-time monitoring data, and realizing global visualization of the park, risk visualization and intelligent management of chemical device equipment and facilities; wherein the surface reconstruction on the global point cloud data to generate a three-dimensional mesh model comprises the following steps: predefining multi-level point cloud segmentation scales, performing multi-scale decomposition on the global point cloud data based on the multi-level point cloud segmentation scales, obtaining a multi-level local point cloud data set, and performing feature extraction on the multi-level local point cloud data set according to the facility attributes of the chemical industrial park to obtain a multi-level local point cloud feature set; traversing and matching the multi-level local point cloud feature set with a three-dimensional model library to output a multi-level local three-dimensional model set; performing model space fusion on the multi-level local three-dimensional model set through point cloud space registration to output a local mesh model; projecting the local mesh model to the global point cloud data to screen and obtain matching defect point cloud data; performing surface reconstruction on the matching defect point cloud data to obtain a compensation mesh model; performing spatial modeling compensation on the local mesh model by using the compensation mesh model to obtain the three-dimensional mesh model; wherein the traversing and matching of the multi-level local point cloud feature set with the three-dimensional model library to output the multi-level local three-dimensional model set comprises the following steps: performing model networking call integration according to the facility attributes of the chemical industrial park to obtain a plurality of sample mesh models; performing feature extraction on the plurality of sample mesh models based on a preset feature index set to obtain a plurality of sample model features; associatively storing the plurality of sample mesh models and the plurality of sample model features to complete data filling of the three-dimensional model library; after traversing and combining the plurality of sample model features with a first local point cloud feature, performing model screening by using a matching algorithm to locate a first local three-dimensional model; by analogy, the multi-level local point cloud feature set is traversed and matched with the three-dimensional model library to output the multi-level local three-dimensional model set.

2. The method for 3D modeling of a chemical park according to claim 1, characterized in that, after traversing and combining the plurality of sample model features with a first local point cloud feature, performing model screening by using feature similarity calculation to locate a first local three-dimensional model, comprising the following steps: calculating a plurality of structure feature similarities between the first local point cloud feature and the plurality of sample model features; if P structure feature similarities satisfy a preset similarity threshold, performing scale deviation calculation on P sample model features and the first local point cloud feature to output P model scale deviations; According to the P normalized results of the P structural feature similarities and the P model proportion deviations, a P sample grid model is serialized to eliminate model conflicts and locate the first-level first local three-dimensional model.

3. The method for 3D modeling of a chemical park as claimed in claim 1 wherein, The multi-lens tilt camera comprises one vertical downward lens and four lateral lenses tilted by 45 degrees, and the monitoring sensor comprises a thermal imaging sensor and a gas detector, wherein the gas detector is used to collect concentration, temperature and pressure data of at least one of H2, CI2, CO, SO2, NO2, O3, VOCs and NH3.

4. The method for 3D modeling of a chemical park as claimed in claim 1 wherein, The dense matching adopts stereo vision, structured light or time-of-flight technology to obtain pixel-level depth information through parallax calculation, light pattern deformation analysis or light pulse time-of-flight measurement.

5. The method for 3D modeling of a chemical park as claimed in claim 1 wherein, The point cloud registration algorithm comprises the following steps: calculating the center point and the covariance matrix of the source point cloud and the to-be-measured point cloud; determining the principal component direction of the point cloud through principal component analysis; solving the rotation matrix and the translation matrix through singular value decomposition to align the to-be-measured point cloud to the source point cloud coordinate system.

6. The method for 3D modeling of a chemical park as claimed in claim 1 wherein, The three-dimensional visualization management platform comprises the following functional modules: a data visualization module supporting two-dimensional / three-dimensional scene switching, map operation and chemical plant equipment attribute query; a dynamic monitoring module for real-time mapping of temperature, pressure or gas concentration data to a three-dimensional model; a personnel positioning module for multi-dimensional spatial positioning and historical trajectory playback through Internet of Things technology.

7. The method for 3D modeling of a chemical park as claimed in claim 1 wherein, The unmanned aerial vehicle supports custom flight path planning, including rectangular flight path, circular flight path, straight flight path or hand-drawn flight path, and at least flight height, speed or overlap rate parameters can be set.

8. A system for the method of three-dimensional modeling of a chemical park according to any one of claims 1-7, characterized by, The system comprises: a data acquisition module comprising an unmanned aerial vehicle platform, a five-lens tilt camera, a thermal imaging sensor, a gas detector, a GPS positioning device and a total station, for acquiring image, video, picture and process parameter data; a data processing module comprising a spatial processing unit, a dense matching unit, a point cloud block splicing unit, a surface reconstruction unit and a texture mapping unit, for generating a high-precision three-dimensional model; an intelligent management module comprising a three-dimensional visualization engine, a dynamic data interface, a personnel positioning system and a device account database, supporting real-time monitoring, risk assessment and device full life cycle management; a visualization module comprising a park two-dimensional / three-dimensional scene, real-time monitoring video, personnel and vehicle positioning tracking, device running attribute, safety risk level distribution map and real-time data statistical analysis interface.

Images