Method and system for three-dimensional modeling of chemical industry park

Abstract

The invention discloses a three-dimensional modeling method and system for a chemical industrial park, and the method comprises the steps: carrying a multi-lens inclined camera and a monitoring sensor through employing an unmanned plane, and obtaining the multi-view image data, positioning and attitude determination data of the chemical industrial park, and the real-time operation data of chemical equipment; performing data processing on the multi-view image data to generate sparse point clouds, and generating three-dimensional point clouds through dense matching and depth estimation; performing block processing on the three-dimensional point cloud, and performing splicing by adopting a point cloud registration algorithm to obtain global point cloud data; performing surface reconstruction on the global point cloud data to generate a three-dimensional grid model, and completing model texture mapping through a texture mapping algorithm; a three-dimensional visual management platform is constructed, a three-dimensional grid model and real-time monitoring data are integrated, and park global visualization, risk visualization and intelligent management of chemical engineering devices, equipment and facilities are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of chemical park safety management and three-dimensional modeling, and in particular to a method and system for three-dimensional modeling of a chemical park. Background Art

[0002] With the continuous development of the economy, the concentration of industrial production has become a new trend in the current global economic development. Chemical parks have gradually become a key carrier of chemical industry clusters, covering industries such as petrochemicals, coal chemicals, and fine chemicals. my country's chemical parks are mainly divided into five types: petrochemical chemical parks, fine chemical chemical parks, urban relocation chemical parks, enterprise expansion chemical parks, and resource-based chemical parks. According to statistics from the Ministry of Management, by the end of 2022, there were 943 chemical parks of various types nationwide, 53 of which had an output value exceeding 50 billion yuan. The revenue of park enterprises accounted for over 50% of the total revenue of the national petrochemical industry, effectively driving local economic growth and optimizing industrial structure. However, the excessive concentration of chemical enterprises can easily lead to the rapid accumulation of high-risk materials, high-risk equipment, high-risk production, high-risk storage, and high-risk operations, sharply increasing production safety risks. Information on chemical equipment, indoor and outdoor scenes of chemical plants, external topography of chemical plants, and surrounding buildings is still limited to two-dimensional paper materials and traditional surveillance video footage. This makes data retrieval difficult and video review cumbersome. In addition, there is a lack of intuitive, integrated three-dimensional models of plant interiors and exteriors to display data. This is reflected in the following aspects:

[0003] The storage and management of hazardous chemicals is chaotic. Hazardous chemicals are not stored in accordance with the three-level standards for isolated, compartmentalized, and separated storage. This has led to the frequent mixing of prohibited materials such as oxidizers and reducers, and strong acids and bases. If an accident occurs, it will not only cause huge casualties and property losses, but will also have long-term negative impacts on the ecological environment.

[0004] Chemical park equipment and facilities are under-utilized. Current real-time management of chemical park equipment and facilities suffers from significant data gaps, particularly in the form of lagging digital mapping and inaccurate lifecycle records. In particular, older equipment and facilities, subject to high temperatures and high pressures, are highly susceptible to toxic and hazardous gas leaks, causing widespread water, air, and soil pollution and resulting in casualties.

[0005] Chemical park risk and hazard records are lagging behind. Currently, data collection relies on manually entered, unstructured Excel spreadsheets, which fail to connect with core production management systems like safety production execution systems and enterprise resource planning. This results in delayed updates of real-time equipment status and hazard identification data. Furthermore, data update mechanisms are significantly lagging, and traditional manual investigations are time-consuming and labor-intensive, with limited coverage and efficiency bottlenecks.

[0006] With the deep integration of next-generation information technologies such as the Internet of Things, big data, cloud computing, artificial intelligence, and 5G into chemical park safety risk management, traditional chemical park safety management models are no longer able to achieve comprehensive, real-time monitoring and management, and are unable to meet the needs of modern chemical park safety risk management. Chemical park digitization has become a new generation of information technology. On the one hand, it uses the OPCUA protocol to dynamically interact with DCS data and 3D models, relying on blockchain technology to solidify equipment change records. On the other hand, it uses laser point cloud scanning to establish an equipment spatial database and build a digital 3D reconstruction system for chemical parks. By establishing a 3D chemical park scene, it provides auxiliary support for promoting the informatization, digitization, networking, and intelligentization of chemical park safety risk management systems.

[0007] Research has found that existing 3D modeling technology is primarily used in fields such as medicine, construction, and transportation. In the chemical industry, it focuses on localized monitoring of key factors such as temperature, pressure, liquid level, and hazardous gases through pipeline and tank modeling. This lacks a comprehensive consideration of the chemical park's overall landscape, and therefore cannot provide strong technical support for its safety management and emergency response. For example, Gu Haifeng's research failed to consider the specificities of the park and neglected key data collection; Gao Chenxu failed to model all areas of the park; the data collection method in patent CN103791887A is limited, and the modeling process is not detailed; and patent CN116894316A focuses solely on chemical pipeline modeling. Summary of the Invention

[0008] In response to the above-mentioned deficiencies or improvement needs of the existing technology, the present invention provides a method and system for three-dimensional modeling of chemical parks. Its purpose is to achieve three-dimensional modeling of chemical park equipment and integrated management covering basic management, production management, safety management and other aspects through an integrated and intelligent digital twin management system. It provides support for intelligent management of park production safety and daily operation and maintenance, explores the visualization of safety management of chemical enterprises, and provides a more intuitive operation method. This solves the technical problems existing in the management of existing chemical parks, such as the difficulty of safety management, high safety risks, data silos and low system integration, as well as the limitations of existing three-dimensional modeling research in the chemical industry.

[0009] To achieve the above object, according to one aspect of the present invention, a method for three-dimensional modeling of a chemical park is provided, the method comprising:

[0010] Using drones equipped with multi-lens tilt cameras and monitoring sensors to obtain multi-view imaging data, positioning data, and real-time monitoring data for chemical parks;

[0011] Processing the multi-view image data to generate a sparse point cloud, and generating a three-dimensional point cloud through dense matching and depth estimation;

[0012] The three-dimensional point cloud is divided into blocks and spliced ​​using a point cloud registration algorithm to obtain global point cloud data;

[0013] Performing surface reconstruction on the global point cloud data to generate a three-dimensional mesh model, and completing model texture mapping through a texture mapping algorithm;

[0014] Build a three-dimensional visualization management platform, integrate three-dimensional grid models and real-time monitoring data, and realize park-wide visualization, risk visualization and intelligent management of chemical equipment and facilities.

[0015] As a further improvement and supplement to the above solution, the present invention also includes the following additional technical features.

[0016] Preferably, performing surface reconstruction on the global point cloud data to generate a three-dimensional mesh model comprises the following steps:

[0017] Predefining a multi-level point cloud segmentation scale, and performing multi-scale decomposition on the global point cloud data based on the multi-level point cloud segmentation scale to obtain a multi-level local point cloud dataset; performing feature extraction on the multi-level local point cloud dataset based on the facility attributes of the chemical park to obtain a multi-level local point cloud feature set;

[0018] Using the multi-level local point cloud feature set to traverse and match a three-dimensional model library, and output a multi-level local three-dimensional model set;

[0019] Performing model space fusion on the multi-level local three-dimensional model set through point cloud spatial registration to output a local grid model;

[0020] Projecting the local grid model onto the global point cloud data, and screening to obtain matching defect point cloud data;

[0021] Performing surface reconstruction on the matching defect point cloud data to obtain a compensation grid model;

[0022] The compensation grid model is used to perform spatial modeling compensation on the local grid model to obtain the three-dimensional grid model.

[0023] Preferably, using the multi-level local point cloud feature set to traverse and match a three-dimensional model library, and matching and outputting a multi-level local three-dimensional model set, comprises the following steps:

[0024] Performing model networking and integration according to the facility attributes of the chemical park to obtain multiple sample grid models;

[0025] Performing feature extraction on the plurality of sample grid models based on a preset feature index set to obtain a plurality of sample model features;

[0026] Associatively storing the plurality of sample grid models and the plurality of sample model features, and completing data filling of the three-dimensional model library;

[0027] After traversing and combining the first-level first local point cloud features with the plurality of sample model features, a matching algorithm is used to perform model screening and locate the first-level first local three-dimensional model;

[0028] By analogy, the multi-level local point cloud feature set is used to traverse the matching three-dimensional model library, and the multi-level local three-dimensional model set is matched and output.

[0029] Preferably, after traversing and combining the first-level first local point cloud features with the plurality of sample model features, model screening is performed using feature similarity calculation to locate the first-level first local three-dimensional model, including the following steps:

[0030] Calculating the similarity of the first-level local point cloud feature with the multiple structural features of the multiple sample model features;

[0031] If the similarity of the P structural features meets the preset similarity threshold, a proportional deviation calculation is performed on the P sample model features and the first-level local point cloud features, and P model proportional deviations are output;

[0032] According to the P normalized results of the P structural feature similarities and the P model proportion deviations, the P sample grid models are serialized to resolve model conflicts and locate the first-level local three-dimensional model.

[0033] Preferably, the multi-lens tilt camera includes one vertical downward lens and four side lenses tilted at 45°, and the monitoring sensor includes a thermal imaging sensor and a gas detector, which is used to collect concentration, temperature and pressure data of at least one gas among H2, CI2, CO, SO2, NO2, O3, VOCs and NH3.

[0034] Preferably, the dense matching adopts stereo vision, structured light or time-of-flight technology to obtain pixel-level depth information through parallax calculation, light pattern deformation analysis or light pulse flight time measurement.

[0035] Preferably, the point cloud registration algorithm comprises the following steps:

[0036] Calculate the center point and covariance matrix of the source point cloud and the point cloud to be measured;

[0037] Determine the principal component direction of the point cloud through principal component analysis;

[0038] The rotation matrix and translation matrix are solved using singular value decomposition to align the point cloud to be measured to the source point cloud coordinate system.

[0039] Preferably, the 3D visualization management platform includes the following functional modules:

[0040] Data visualization module, supporting 2D / 3D scene switching, map operation, and chemical plant equipment attribute query;

[0041] Dynamic monitoring module, real-time mapping of temperature, pressure or gas concentration data to 3D models;

[0042] The personnel positioning module uses IoT technology to achieve multi-dimensional spatial positioning and historical trajectory playback.

[0043] Preferably, the drone supports custom route planning, including rectangular routes, circular routes, straight routes or hand-drawn routes, and can set at least flight altitude, speed or overlap rate parameters.

[0044] According to another aspect of the present invention, a system for three-dimensional modeling of a chemical park is provided, the system comprising:

[0045] Data acquisition module, including a drone platform, a five-lens tilt camera, a thermal imaging sensor, a gas detector, a GPS positioning device, and a total station, used to obtain images, videos, pictures, and process parameter data;

[0046] Data processing module, including spatial processing unit, dense matching unit, point cloud block splicing unit, surface reconstruction unit and texture mapping unit, used to generate high-precision 3D models;

[0047] Intelligent management module, including a 3D visualization engine, dynamic data interface, personnel positioning system, and equipment ledger database, supports real-time monitoring, risk assessment, and full life cycle management of equipment;

[0048] The visualization module includes two-dimensional / three-dimensional park scenes, real-time monitoring videos, personnel and vehicle positioning and tracking, device and equipment operation properties, safety risk level distribution maps, and real-time data statistical analysis interface.

[0049] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0050] Efficient and comprehensive data collection: Integrating drone oblique photography with multiple sensors enables simultaneous acquisition of multi-source data such as images, videos, pictures, and process parameters, covering the entire park scene and improving collection efficiency by more than 50%.

[0051] High-precision 3D modeling: Through dense matching and intelligent stitching algorithms, point cloud positioning accuracy reaches the centimeter level, and model texture fit is improved by 30%, meeting the detailed requirements of security management.

[0052] Intelligent analysis and visualization: Real-time integration of monitoring data and 3D models, supporting equipment abnormality alarms, personnel positioning and risk classification display, improving the company's rapid perception, real-time monitoring, advanced warning, dynamic optimization, intelligent decision-making and intelligent management capabilities.

[0053] System integration and scalability: It adopts distributed storage and parallel computing, supports massive data processing, is compatible with multi-source equipment access, and provides a unified platform for the digital transformation of chemical parks. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0055] Figure 1 This embodiment provides a flow chart of a method for three-dimensional modeling of a chemical park;

[0056] Figure 2 is a schematic diagram of visibility analysis in the first embodiment;

[0057] Figure 3 is a schematic diagram of occlusion detection in the first embodiment;

[0058] Figure 4 This embodiment 2 provides a system flow chart for three-dimensional modeling of a chemical park. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0060] Example 1:

[0061] This embodiment provides a method for three-dimensional modeling of a chemical park, such as Figure 1 As shown, the method includes:

[0062] S101: Use drones equipped with multi-lens tilt cameras and monitoring sensors to obtain multi-view imaging data, positioning and attitude data, and real-time operation data of chemical plant equipment in chemical parks.

[0063] In the first embodiment, the main method is to combine the oblique photography modeling technology with the manual modeling technology, mainly through the use of four methods: oblique photography data collection, video data collection, indoor and outdoor data collection, and monitoring data collection.

[0064] Oblique photogrammetry data acquisition utilizes drones equipped with multiple, high-resolution cameras to plan flight areas, automatically generate flight routes based on the overlap of headings, execute flight missions, and collect both orthophoto and oblique imagery data. This technology overcomes the previous limitation of orthophotos, which were limited to vertical capture. By equipping a single flight platform with multiple sensors, it simultaneously captures images from five different angles: vertical, forward, rear, left, and right. This introduces users to a realistic and intuitive world that reflects human vision. The images contain real-world information and are rich in information, enabling data mining. Furthermore, photogrammetry from low altitudes can produce high-resolution aerial imagery close to the ground. Assisted by a high-precision positioning and attitude system, each point in the image has three-dimensional coordinates, enabling measurement accuracy ranging from centimeters to decimeters. Compared to orthophotos, it also achieves higher elevation accuracy, enabling direct measurement of the heights of features such as buildings.

[0065] Video data collection can be performed during a drone flight, using the real-time image transmission from the drone's imaging sensor to mark specific points, lines, and surfaces on a map. Drone video footage can also be streamed live via streaming technology. Video viewers can use rectangular boxes to mark targets and track their movement, or mark specific points, lines, and surfaces. All marked points, lines, and surfaces can be used as a reference for other personnel.

[0066] Indoor and outdoor data collection can be used during the flight of drones. When the collection objects are elements such as building structures and pipeline directions, the main collection items include element texture, element geographic location information, actual size of elements and other ancillary information.

[0067] Texture collection: Use a high-resolution digital camera to take photos and samples of the outer contour texture of the elements. During the sampling process, ensure that the data collection coverage is in place and there are no blind spots in all directions.

[0068] Geographic location: Use GPS and other positioning equipment to collect the coordinate information of the elements, and use the inflection points of the element's outer contour line to establish the outer contour coordinate point information.

[0069] Actual size: Use surveying tools such as total stations to measure the outer contours of features, draw two-dimensional CAD maps, and measure the elevation of the current feature, the floor height of the building, feature attributes, and other information.

[0070] Monitoring data collection can utilize drones equipped with thermal imaging sensors to monitor the temperature, humidity, and operating conditions of key equipment day and night in real time. Any anomalies can be displayed and alarmed promptly. Using drones equipped with gas detectors to detect gases emitted from chemical areas, data on the concentration, temperature, and pressure of at least one of the following gases: H2, CI2, CO, SO2, NO2, O3, VOCs, and NH3 can be collected. This allows for comprehensive monitoring of safety hazards in chemical companies, achieving effective supervision and prevention.

[0071] S102: Process the multi-view image data to generate a sparse point cloud, and generate a three-dimensional point cloud through dense matching and depth estimation.

[0072] By matching feature points in space and performing bundle adjustment, a sparse 3D point cloud is generated. This point cloud represents the 3D positions of the feature points in the image. However, these points cannot form the data required for triangulation to reconstruct the 3D model. Given a known image camera matrix, the 3D coordinates of the corresponding ground feature points can be obtained by finding the points with the same name in the two photos and using bundle adjustment. To obtain sufficient pairs of points with the same name, dense matching is performed.

[0073] Depth estimation is typically performed after feature extraction, sparse point cloud generation, and dense matching. Dense matching provides detailed image pairs or multi-view data for depth estimation. The goal of depth estimation is to obtain depth information for each pixel in order to generate a high-precision 3D point cloud, supporting subsequent operations such as point cloud stitching and surface reconstruction.

[0074] S103: Divide the three-dimensional point cloud into blocks and stitch them together using a point cloud registration algorithm to obtain global point cloud data.

[0075] In the first embodiment of the present invention, block processing is block modeling, which is a technology for optimizing the three-dimensional modeling process and is particularly suitable for processing large-scale or complex scenes. The core idea is to divide the entire three-dimensional model or point cloud into multiple smaller blocks so that they can be processed and optimized block by block. Specifically, by applying the three-dimensional point cloud stitching algorithm, the gaps when the drone collects data can be effectively utilized to independently model each area, while reducing the amount of calculation required for each modeling. In this way, the modeling of the entire area can be completed during the landing of the drone, thereby improving efficiency. In subsequent steps, these independent block data will be automatically spliced ​​into an overall three-dimensional model, thereby forming a complete model of the flight target area.

[0076] Block modeling provides a structured data foundation for subsequent surface reconstruction and texture mapping, allowing these steps to be performed independently on each block and then merged, improving the quality and consistency of the overall model. Furthermore, block modeling reduces computational complexity and memory requirements, making the process smoother and more stable.

[0077] Block modeling also significantly improves modeling efficiency, transforming the exponentially increasing time required by traditional methods into a linear increase. It allows for real-time modeling while the drone is in flight, and through efficient block-level processing and splicing, produces highly accurate 3D models. Overall, block modeling not only improves modeling speed and accuracy but also optimizes the use of computing resources, providing an effective solution for 3D modeling of large-scale and complex scenes.

[0078] S104: performing surface reconstruction on the global point cloud data to generate a three-dimensional mesh model, and completing model texture mapping through a texture mapping algorithm.

[0079] Texture mapping is a parameterization problem for object surfaces. Texture mapping in oblique photogrammetry, also known as texture mapping, involves applying texture to a pre-constructed 3D model mesh to enhance its realism. Oblique models are generated from 2D images, so texture mapping naturally uses the generated model image. Model mapping is essentially a mapping process from 2D to 3D space. Because 3D models are obtained through image matching, spatial encryption, and point cloud generation, the 2D-to-3D mapping relationship already exists. The camera matrix for each image is already obtained during the dense point cloud generation process. However, a consideration is that points on the model may have multiple texture sources, corresponding to multiple photos. Images are typically selected as texture mapping sources based on metrics such as occlusion and visibility to achieve optimal texture mapping results.

[0080] S105: Build a three-dimensional visualization management platform, integrate three-dimensional grid models with real-time monitoring data, and realize park-wide visualization, risk visualization, and intelligent management of chemical equipment and facilities.

[0081] In combination with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, the multi-lens tilt camera includes 1 vertical downward lens and 4 side lenses tilted at 45°. The monitoring sensor includes a thermal imaging sensor and a gas detector. The gas detector is used to collect the concentration, temperature and pressure data of at least one gas among H2, CI2, CO, SO2, NO2, O3, VOCs and NH3.

[0082] In the first embodiment of the present invention, the image data obtained by the aerial photography flight is post-processed using the lens's built-in software. Because the oblique camera is equipped with 5 cameras, the angles of the cameras tilted in different directions are all 45°. At the moment of shooting exposure, due to the inconsistency of the camera shooting angles, there will be inconsistencies in intensity, light contrast, etc.; this may cause different brightness levels, colors, etc. of the same ground feature image on or near the bridge under each camera, affecting subsequent processing and modeling. Oblique image data acquisition is to carry multiple sensors on the same flight platform, and simultaneously obtain vertical and oblique images and position information of the ground features and landforms from various angles. The image shot vertically to the ground is called a positive film, and the image shot with the lens facing a certain angle to the ground is called an oblique film. Oblique photography cameras generally use a combination of 5 cameras (5 lenses), 1 camera is vertically downward, and the other 4 cameras are tilted at a certain angle.

[0083] In combination with the embodiments of the present invention, there is also a preferred implementation scheme. Specifically, dense matching uses stereo vision, structured light or time-of-flight technology to obtain pixel-level depth information through parallax calculation, light pattern deformation analysis or light pulse flight time measurement.

[0084] Stereo vision is a technology that uses two or more cameras to capture the same scene from different perspectives, simulating the principles of the human visual system to reconstruct three-dimensional scenes and measure depth. The core of stereo vision lies in disparity estimation and triangulation.

[0085] First, a binocular camera system captures two images of the same scene, with the two cameras positioned with a fixed baseline spacing. Next, feature points are found and matched in the left and right images to calculate a disparity map. This map represents the disparity value for each pixel in the image—that is, the horizontal distance between corresponding points in the left and right images. Using this disparity value and the camera baseline length, the depth of the object can be calculated using a formula. Finally, this depth information is converted into a 3D point cloud, generating a 3D model for further analysis and processing.

[0086] The key steps of stereo vision include: camera calibration, estimating the camera's internal parameters (such as focal length, principal point position, and distortion coefficient) and external parameters (i.e., the relative position and posture between the two cameras); image correction, transforming the left and right images into parallel perspectives, eliminating the camera's geometric distortion, and making the points on the same horizontal line in the left and right images have the same vertical coordinate; disparity map generation, by calculating the matching cost of the left and right images, using a cost function (such as absolute difference, squared difference) to generate a disparity map, and using optimization techniques to reduce mismatches and noise; depth map generation, generating a depth map based on the disparity map and camera parameters to represent the depth information of each pixel. These steps ensure that the stereo vision system can accurately reconstruct the three-dimensional scene.

[0087] Structured light technology projects a known light pattern (such as stripes or a grid) onto an object's surface and uses an image sensor to capture the deformed light pattern, thereby capturing the object's three-dimensional shape. The difference between structured light and stereo vision is that structured light requires a projector to project raster stripes. The structured light workflow includes light source projection, image acquisition, feature matching, and 3D reconstruction.

[0088] During the light projection phase, a structured light projector is used to project a known pattern of light onto the object's surface. A camera then captures images of the light pattern deformed on the object's surface and matches these images with the original pattern. Finally, the object's 3D shape is calculated based on the deformation of the light pattern. Structured light technology offers the advantages of high-precision measurement and real-time data acquisition, making it suitable for applications such as 3D scanning, industrial inspection, and robotic vision. However, it is sensitive to ambient lighting and surface characteristics and is generally suitable for measuring small and medium-sized objects.

[0089] Time-of-flight (TOF) technology is a distance measurement technique based on the time-of-flight of light pulses. It calculates the distance to an object by emitting a light pulse and measuring the time it takes for it to reflect back, providing real-time depth information. Its operating process involves light pulse emission, reflection, time measurement, and depth calculation. During the light pulse emission phase, the transmitter sends a short pulse of light, which hits an object and is reflected back to the receiver. By measuring the time from emission to reception, the distance to the object can be calculated and a depth map or 3D point cloud can be generated.

[0090] Time-of-flight 3D measurement technology is similar to a bat's ultrasonic system, timing the round trip of a point of light that bounces off the surface of the target and back to the sensor.

[0091] In conjunction with the embodiment of the present invention, there is also a preferred implementation scheme. Specifically, the point cloud registration algorithm includes the following steps:

[0092] S201: Calculate the center point and covariance matrix of the source point cloud and the point cloud to be measured.

[0093] In the first embodiment, the principal component analysis algorithm is used to register the point cloud data in multiple point cloud images. Assume that the set of n-dimensional point cloud data in a certain point cloud image is q = {q1, q2, ...q n}, construct the source point cloud model matrix Q according to the x, y, and z axis positions of the point cloud data in q in the original coordinate space:

[0094]

[0095] The source point cloud model matrix Q is used as the reference point cloud, and a reference point cloud set r = {r1, r2, ...r m} and the m-dimensional point cloud matrix R to be tested:

[0096]

[0097] Calculate the point cloud center O of the source point cloud model matrix Q and the point cloud matrix R to be measured Q , O R :

[0098] At this time, the covariance matrix cov of the point cloud in the R and Q matrices Q 、cov R The calculation process is as follows:

[0099]

[0100] Where T is the matrix transpose symbol.

[0101] S202: Determine the principal component direction of the point cloud through principal component analysis.

[0102] Let μ = (μ x ,μ y ,μ z ) is the matrix eigenvector. According to the principal component analysis point cloud configuration algorithm, cov·Q=μQ, from which the eigenvectors of the Q and R matrices can be derived.

[0103] Take the eigenvector μ of the source point cloud model matrix Q x ,μ y ,μ z As the three-dimensional space coordinate direction, establish a three-dimensional space coordinate system is the final registration space of the point cloud image.

[0104] S203: Using singular value decomposition to solve the rotation matrix and the translation matrix, aligning the point cloud to be measured to the source point cloud coordinate system.

[0105] The singular value decomposition algorithm is used to obtain the translation matrix U and rotation matrix S between the source point cloud and the point cloud to be measured, and the point cloud transformation is performed on all point clouds in the point cloud data to be measured based on the inverse operation of the matrix.

[0106] After obtaining the final registration space coordinate system for the point cloud image, the point cloud center calculation method and the singular value decomposition algorithm are combined to calculate the transformation matrix of each point cloud image. Then, the data points in the point cloud images taken from multiple angles are aligned to the final registration space coordinate system, so that the originally scattered and fragmented point cloud data have spatial consistency, avoiding the occurrence of misalignment problems in the subsequent graphics splicing process.

[0107] In this embodiment 1, the point cloud registration algorithm includes the following steps:

[0108] Extract local geometric features (such as corners and edges) and global structural features (such as the cylindrical outline of the tank) of the point cloud, generate descriptors such as FPFH and SHOT to encode neighborhood information, and use KD-Tree or RANSAC to establish feature point correspondences and eliminate false matches.

[0109] Based on 4PCS or semantic matching, the point cloud is preliminarily aligned to reduce the pose difference, solve the initial position deviation problem in a large range of scenes, and provide an optimized initial value of the transformation matrix for precise registration.

[0110] The ICP algorithm is used for iterative optimization, and the optimal rigid body transformation (rotation + translation) is calculated through nearest point search and SVD decomposition. The robust loss function or color information is combined to improve the noise resistance until the point cloud alignment error converges to the sub-millimeter level.

[0111] It fuses multi-view point clouds and detects closed loops to correct accumulated errors, optimizes global consistency using pose graphs, and quantifies accuracy using RMSE and overlap rate.

[0112] In the first embodiment, texture mapping includes visibility determination, occlusion detection and verification, and texture fusion, including the following steps:

[0113] The visibility analysis calculates the angle between the triangle normal and the line connecting the photographic center, filters the visible images within the range of 0° to 90°, and performs visibility determination on each triangle in the 3D mesh model:

[0114] a) Calculate the angle θ between the triangle normal vector n and the line of sight vector v from the camera center to the patch centroid, and establish the visibility judgment condition: cosθ ≥ ∈, where ∈ is set as an adjustable threshold of 0 ≤ ε ≤ 1, preferably ε = 0 corresponds to θ = 90°;

[0115] b) Construct a candidate image set, where N is the total number of photography stations;

[0116] c) Calculate the weight based on the angle cosine value cosθ and the image resolution factor ρk, and select the first M images with the largest weight as the valid visible image set

[0117] The occlusion detection is based on the elevation ray tracing algorithm to determine whether the line connecting the ground point and the photography center is blocked. Based on the elevation space analysis of the digital surface model, pixel-by-pixel occlusion verification is performed on the candidate image set V:

[0118] a) Establish the line of sight equation from the ground point P(x, y, z) to the photography center Ok: , t∈[0,1];

[0119] b) Perform 3D ray tracing along the line of sight L in the digital surface model data space, using an adaptive step size strategy.

[0120] For the valid image set V' that passes visibility analysis and occlusion detection, texture weight allocation based on projection area is adopted:

[0121] a) Calculate the projection area Ak of ​​each triangle in each valid image;

[0122] b) establishing a hybrid weight function;

[0123] c) Perform multi-resolution texture fusion and perform band-weighted synthesis in the Laplacian pyramid space to eliminate seams and preserve high-frequency details.

[0124] In conjunction with the embodiment of the present invention, there is also a preferred implementation scheme. Specifically, the surface of the global point cloud data is reconstructed to generate a three-dimensional mesh model. The step S104 further includes the following steps:

[0125] S1041: pre-defining a multi-level point cloud segmentation scale, and performing multi-scale decomposition on the global point cloud data based on the multi-level point cloud segmentation scale to obtain a multi-level local point cloud dataset;

[0126] S1042: performing feature extraction on the multi-level local point cloud dataset according to the facility attributes of the chemical park to obtain a multi-level local point cloud feature set;

[0127] S1043: using the multi-level local point cloud feature set to traverse and match a three-dimensional model library, and outputting a multi-level local three-dimensional model set;

[0128] S1044: performing model space fusion on the multi-level local 3D model set through point cloud spatial registration, and outputting a local grid model;

[0129] S1045: Projecting the local grid model onto the global point cloud data, and screening to obtain matching defect point cloud data;

[0130] S1046: Performing surface reconstruction on the matching defect point cloud data to obtain a compensation mesh model;

[0131] S1047: Using the compensation grid model to perform spatial modeling compensation on the local grid model to obtain the three-dimensional grid model.

[0132] In conjunction with the embodiment of the present invention, there is also a preferred implementation scheme. Specifically, the multi-level local point cloud feature set is used to traverse the matching 3D model library to match and output the multi-level local 3D model set. The step S1043 further includes the following steps:

[0133] S1043a: Performing model networking call integration based on the facility attributes of the chemical park to obtain multiple sample grid models;

[0134] S1043b: Perform feature extraction on the multiple sample grid models based on a preset feature index set to obtain multiple sample model features;

[0135] S1043c: Associatively storing the multiple sample grid models and the multiple sample model features, completing data filling of the three-dimensional model library;

[0136] S1043d: After traversing and combining the first-level local point cloud features with the plurality of sample model features, a matching algorithm is used to perform model screening and locate the first-level local three-dimensional model;

[0137] S1043e: Similarly, the multi-level local point cloud feature set is used to traverse the matching three-dimensional model library, and the matching output is the multi-level local three-dimensional model set.

[0138] In conjunction with the embodiment of the present invention, there is also a preferred implementation scheme. Specifically, after traversing and combining the first-level local point cloud features with the multiple sample model features, feature similarity calculation is used to perform model screening and locate the first-level local three-dimensional model. The step S1043d further includes the following steps:

[0139] S1043d-1: Calculating the similarity between the first-level local point cloud feature and the multiple structural features of the multiple sample model features;

[0140] S1043d-2: If the similarity of the P structural features meets the preset similarity threshold, then calculate the proportional deviation of the P sample model features and the first-level first local point cloud features, and output P model proportional deviations;

[0141] S1043d-3: Serialize the P sample grid models based on the P normalized results of the P structural feature similarities and the P model proportion deviations to resolve model conflicts and locate the first-level local three-dimensional model.

[0142] It should be understood that directly performing surface reconstruction on the global point cloud data to generate a three-dimensional mesh model has the disadvantages of excessive consumption of computing resources of the modeling software and a long modeling time. Based on this, this embodiment combines the existing public model of the chemical park to optimize the modeling computing resources and time consumption.

[0143] Specifically, a multi-level point cloud segmentation scale is predefined according to the scale and complexity of the chemical park, and the global point cloud data is decomposed into multi-level local point cloud datasets at different levels based on the multi-level point cloud segmentation scale. This multi-scale decomposition facilitates subsequent targeted processing and lays the foundation for subsequent feature extraction and model matching.

[0144] Based on the attributes of chemical park facilities, such as buildings, pipelines, and equipment, feature extraction is performed on the decomposed multi-level local point cloud dataset. The extracted features may include geometric shape, size, texture, etc., forming a multi-level local point cloud feature set, which provides key information for subsequent model matching.

[0145] By using the multi-level local point cloud feature set, traversal matching is performed in the 3D model library. By comparing the features of the feature set with the features of each model in the model library, the 3D model that best matches the local point cloud is found, and the multi-level local 3D model set is output to achieve rapid modeling of the local point cloud and reduce the workload of modeling from scratch.

[0146] The specific technical implementation process of feature matching to determine the multi-level local 3D model set is as follows:

[0147] According to the facility attributes of the chemical park, multiple sample grid models are integrated through network calls. These sample grid models can be derived from existing chemical park public models or other reliable model sources to provide a reference for subsequent model matching.

[0148] For the integrated multiple sample mesh models, feature extraction is performed based on a preset feature index set to obtain a feature set for each sample mesh model. The preset feature index set may include geometric features, texture features, size features, etc., which are then used for matching with local point cloud features.

[0149] Multiple sample mesh models are stored in association with their corresponding sample model features (sets) to form a complete 3D model library. This step ensures that each model in the model library is accompanied by a corresponding feature description, which facilitates subsequent feature matching.

[0150] The first-level local point cloud features are traversed and combined with the features of multiple sample models, and then screened using a matching algorithm. By comparing indicators such as similarity between features, the sample model that best matches the first-level local point cloud features is found, thereby locating the first-level local 3D model.

[0151] Here, in the process of finding the sample model that best matches the first-level local point cloud feature, there is a matching conflict problem in the adaptation of multiple sample models. This conflict problem is resolved through the following process.

[0152] Calculate multiple structural feature similarities between the first-level local point cloud features and the features of multiple sample models. Structural feature similarity is calculated using conventional techniques and can be based on features such as geometry, size, and texture. Structural feature similarity measures the degree of similarity between the local point cloud and the sample models.

[0153] If the structural similarity between P sample model features and the local point cloud features meets the preset similarity threshold, the proportional deviation between these P sample model features and the local point cloud features is further calculated. The proportional deviation is used to measure the difference in global structural size between the sample model and the local point cloud, and P model proportional deviations are output.

[0154] Based on the normalized results of P structural feature similarities and P model scale deviations, P sample mesh models are serialized. The normalized results are used to comprehensively evaluate the model similarity and size matching. Serialization resolves model conflicts and ultimately locates the most appropriate first-level local 3D model.

[0155] By analogy, the multi-level local point cloud feature set is used to traverse the matching three-dimensional model library, and the multi-level local three-dimensional model set is matched and output.

[0156] The matched multi-level local 3D model set is subjected to point cloud spatial registration, and different local models are aligned to the same spatial coordinate system. Then, spatial fusion is performed to generate a local grid model with complete spatial connection, so that each local model is seamlessly connected in space to form a coherent overall model.

[0157] The fused local grid model is projected onto the original global point cloud data, and the point cloud data that is not covered by the model or is inaccurately matched is found through comparison, that is, the matching defective point cloud data, providing a basis for subsequent supplementary modeling.

[0158] Surface reconstruction is performed on the selected matching defect point cloud data to generate a compensation mesh model for compensating for coverage defects in the local mesh model. This step aims to repair defects in the model or supplement uncovered areas, thereby improving the integrity and accuracy of the model.

[0159] The compensated grid model is spatially fused with the previous local grid model to compensate the local grid model, and finally a complete three-dimensional grid model is obtained.

[0160] Compared to directly reconstructing the surface of the global point cloud data to generate a three-dimensional mesh model, this embodiment achieves the technical effect of significantly improving modeling efficiency, reducing the demand for computing resources in the modeling process, and ensuring the modeling accuracy of the three-dimensional mesh model. In conjunction with the embodiment of the present invention, there is also a preferred implementation scheme. Specifically, texture mapping includes visibility analysis and occlusion detection:

[0161] The visibility analysis screens visible images within a range of 0° to 90° by calculating the angle between the normal line of the triangle and the line connecting the photographic center.

[0162] The normal of the polygon on the 3D mesh model forms an angle with the line connecting the photographic center to the center of the film, and this angle is used to determine whether the image is visible on the face. Figure 2 As shown in Figure 1, the angle θ is the angle between the facet normal ON and the line connecting the photographic center to the image center OS. If the angle is within the range of 0° to 90°, the triangle is visible on the image; if it is outside the range, it is invisible.

[0163] Occlusion detection is based on the elevation ray tracing algorithm to determine whether the line connecting the ground point and the photography center is blocked.

[0164] Occlusion detection Figure 3 As shown, both A and B are projected onto point a on the image plane. When viewed from the camera center, point A is visible (point V in the visible points), and point B is occluded by A and invisible (point O in the visible points). Assuming that a camera ray corresponds to multiple points (i.e., multiple 3D points projected into the image correspond to one image point), the point closest to the camera center is the visible point, and the rest are invisible points.

[0165] In conjunction with the embodiment of the present invention, there is also a preferred implementation scheme. Specifically, the three-dimensional visualization management platform includes the following functional modules:

[0166] The data visualization module supports 2D / 3D scene switching, map operation, and chemical plant equipment property query.

[0167] Basic map operations: Load the 3D model onto the 3D map according to the actual location information, and use the mouse to view the surrounding environment inside and outside the factory, as well as zoom in, zoom out, rotate, and translate the scene.

[0168] Navigation and Positioning: The management object information is described in a tree-like directory format according to the chemical enterprise classification standards, making it easier for users to understand the management hierarchy and quickly access the list of management object information they are interested in. Use the positioning function to switch the window directly to the user's target of interest.

[0169] It can switch between two-dimensional and three-dimensional display modes. After entering the three-dimensional mode, you can enter the interior of the model building and manage objects accurately to each floor, room, facility, equipment and even sensor.

[0170] Layer management: spatial data, objects and models can be managed in layers.

[0171] It intuitively, realistically, and accurately displays the external topography of the chemical plant, surrounding building information, as well as the distribution of various facilities and equipment within the plant, as well as the production organization relationship. Users can browse the entire chemical plant on their computers or mobile phones.

[0172] Dynamic monitoring module maps temperature, pressure, gas concentration and other data to the 3D model in real time.

[0173] Dynamic Data Visualization Module: This module provides dynamic visualization of various real-time data within the chemical park. It integrates data streams from various sensors and monitoring devices through data interfaces, mapping information such as temperature, pressure, and gas concentration to the 3D model in real time. This module supports customizable data layers and multi-dimensional data presentation, allowing users to select different data views based on their needs, enabling real-time monitoring and analysis of the park's operating status.

[0174] The personnel positioning module uses IoT technology to achieve multi-dimensional spatial positioning and historical trajectory playback.

[0175] Through intelligent devices such as IoT technology, facial recognition, and AI video surveillance, people entering and leaving the park are monitored, identified, tracked, and analyzed, providing real-time data on the flow of people within the area. Based on the positioning requirements of each area of ​​the park, multi-dimensional spatial positioning is achieved, enabling personnel supervision and data statistical analysis. Based on a three-dimensional visualization scene, personnel history is dynamically displayed. Monitoring personnel can quickly locate a specific person through the image search function, and can select any staff member to view their real-time location and work status, thus understanding the staff member's behavior path over any time period.

[0176] In the first embodiment, the drone supports custom route planning, including rectangular routes, circular routes, straight routes or hand-drawn routes, and can set at least flight altitude, speed or overlap rate parameters.

[0177] Example 2:

[0178] In the second embodiment, a system for three-dimensional modeling of a chemical park is provided. Figure 4 As shown, the system includes:

[0179] Data acquisition module, including a drone platform, a five-lens tilt camera, a thermal imaging sensor, a gas detector, a GPS positioning device, and a total station, used to obtain images, videos, pictures, and process parameter data;

[0180] Data processing module, including spatial processing unit, dense matching unit, point cloud block splicing unit, surface reconstruction unit and texture mapping unit, used to generate high-precision 3D models;

[0181] Intelligent management module, including a 3D visualization engine, dynamic data interface, personnel positioning system, and equipment ledger database, supports real-time monitoring, risk assessment, and full life cycle management of equipment;

[0182] The visualization module includes two-dimensional / three-dimensional park scenes, real-time monitoring videos, personnel and vehicle positioning and tracking, device and equipment operation properties, safety risk level distribution maps, and real-time data statistical analysis interface.

[0183] The intelligent management module also includes a distributed storage system and a parallel computing framework for processing massive point cloud data and image data, and supports immersive interactive interfaces for multiple terminals.

[0184] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the embodiment 1 of the present invention. The specific content can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0185] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk or an optical disk, etc.

[0186] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for three-dimensional modeling of a chemical park, characterized in that the method include: Using drones equipped with multi-lens tilt cameras and monitoring sensors to obtain multi-view imaging data, positioning data, and real-time monitoring data for chemical parks; Processing the multi-view image data to generate a sparse point cloud, and generating a three-dimensional point cloud through dense matching and depth estimation; The three-dimensional point cloud is divided into blocks and spliced ​​using a point cloud registration algorithm to obtain global point cloud data; Performing surface reconstruction on the global point cloud data to generate a three-dimensional mesh model, and completing model texture mapping through a texture mapping algorithm; Build a three-dimensional visualization management platform, integrate three-dimensional grid models and real-time monitoring data, and realize park-wide visualization, risk visualization and intelligent management of chemical equipment and facilities.

2. The method for three-dimensional modeling of a chemical park according to claim 1, wherein: Performing surface reconstruction on the global point cloud data to generate a three-dimensional mesh model includes the following steps: Predefining a multi-level point cloud segmentation scale, and performing multi-scale decomposition on the global point cloud data based on the multi-level point cloud segmentation scale to obtain a multi-level local point cloud dataset; performing feature extraction on the multi-level local point cloud dataset based on the facility attributes of the chemical park to obtain a multi-level local point cloud feature set; Using the multi-level local point cloud feature set to traverse and match a three-dimensional model library, and output a multi-level local three-dimensional model set; Performing model space fusion on the multi-level local three-dimensional model set through point cloud spatial registration to output a local grid model; Projecting the local grid model onto the global point cloud data, and screening to obtain matching defect point cloud data; Performing surface reconstruction on the matching defect point cloud data to obtain a compensation grid model; The compensation grid model is used to perform spatial modeling compensation on the local grid model to obtain the three-dimensional grid model.

3. The method for three-dimensional modeling of a chemical park according to claim 2, wherein: The multi-level local point cloud feature set is used to traverse and match a three-dimensional model library, and a multi-level local three-dimensional model set is matched and output, comprising the following steps: Performing model networking and integration according to the facility attributes of the chemical park to obtain multiple sample grid models; Performing feature extraction on the plurality of sample grid models based on a preset feature index set to obtain a plurality of sample model features; Associatively storing the plurality of sample grid models and the plurality of sample model features, and completing data filling of the three-dimensional model library; After traversing and combining the first-level first local point cloud features with the plurality of sample model features, a matching algorithm is used to perform model screening and locate the first-level first local three-dimensional model; By analogy, the multi-level local point cloud feature set is used to traverse the matching three-dimensional model library, and the multi-level local three-dimensional model set is matched and output.

4. The method for three-dimensional modeling of a chemical park according to claim 3, wherein: After traversing and combining the first-level first local point cloud features with the plurality of sample model features, model screening is performed using feature similarity calculation to locate the first-level first local three-dimensional model, including the following steps: Calculating the similarity of the first-level local point cloud feature with the multiple structural features of the multiple sample model features; If the similarity of the P structural features meets the preset similarity threshold, a proportional deviation calculation is performed on the P sample model features and the first-level local point cloud features, and P model proportional deviations are output; According to the P normalized results of the P structural feature similarities and the P model proportion deviations, the P sample grid models are serialized to resolve model conflicts and locate the first-level local three-dimensional model.

5. The method for three-dimensional modeling of a chemical park according to claim 1, wherein: The multi-lens tilt camera includes one vertical downward lens and four side lenses tilted at 45°. The monitoring sensor includes a thermal imaging sensor and a gas detector. The gas detector is used to collect concentration, temperature and pressure data of at least one gas among H2, CI2, CO, SO2, NO2, O3, VOCs and NH3.

6. The method for three-dimensional modeling of a chemical park according to claim 1, wherein: The dense matching adopts stereo vision, structured light or time-of-flight technology to obtain pixel-level depth information through parallax calculation, light pattern deformation analysis or light pulse flight time measurement.

7. The method for three-dimensional modeling of a chemical park according to claim 1, wherein: The point cloud registration algorithm includes the following steps: Calculate the center point and covariance matrix of the source point cloud and the point cloud to be measured; Determine the principal component direction of the point cloud through principal component analysis; The rotation matrix and translation matrix are solved using singular value decomposition to align the point cloud to be measured to the source point cloud coordinate system.

8. The method for three-dimensional modeling of a chemical park according to claim 1, wherein: The three-dimensional visualization management platform includes the following functional modules: Data visualization module, supporting 2D / 3D scene switching, map operation, and chemical plant equipment attribute query; Dynamic monitoring module, real-time mapping of temperature, pressure or gas concentration data to 3D models; The personnel positioning module uses IoT technology to achieve multi-dimensional spatial positioning and historical trajectory playback.

9. The method for three-dimensional modeling of a chemical park according to claim 1, wherein: The drone supports custom route planning, including rectangular routes, circular routes, straight routes or hand-drawn routes, and can set at least flight altitude, speed or overlap rate parameters.

10. A system for three-dimensional modeling of a chemical park, characterized in that: The system includes: Data acquisition module, including a drone platform, a five-lens tilt camera, a thermal imaging sensor, a gas detector, a GPS positioning device, and a total station, used to obtain images, videos, pictures, and process parameter data; Data processing module, including spatial processing unit, dense matching unit, point cloud block splicing unit, surface reconstruction unit and texture mapping unit, used to generate high-precision 3D models; Intelligent management module, including a 3D visualization engine, dynamic data interface, personnel positioning system, and equipment ledger database, supports real-time monitoring, risk assessment, and full life cycle management of equipment; The visualization module includes two-dimensional / three-dimensional park scenes, real-time monitoring videos, personnel and vehicle positioning and tracking, device and equipment operation properties, safety risk level distribution maps, and real-time data statistical analysis interface.

Images