Project quality management method and system based on AI large model

Abstract

The invention discloses a project quality management method and system based on an AI large model, and relates to the field of project quality management.The project quality management method comprises the steps that panoramic equipment is installed on a tower crane on a project site, and a panoramic image is collected through the panoramic equipment; performing semantic segmentation on the panoramic image by using a pre-constructed risk control model; carrying out hidden danger investigation on each grid region, and collecting a hidden danger target image of a hidden danger target through panoramic equipment; inputting the hidden danger target images into a pre-trained hidden danger classification model, and classifying the hidden danger target images; according to the hidden danger target type label and the position coordinate, collecting a local image of the hidden danger target in a preset range; fusing the hidden danger target image with the corresponding local image; and mapping the fused image into a BIM model through three-dimensional reconstruction to obtain a digital twinborn model of an engineering site, and performing engineering quality management by using the digital twinborn model. Aiming at the problem that hidden danger investigation in the engineering field is not timely in the prior art, the hidden danger investigation efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of engineering quality management, and particularly to an engineering quality management method and system based on an AI large model. Background Art

[0002] With the rapid development and continuous expansion of the construction engineering industry, the construction environment at the engineering site has become increasingly complex, and various potential safety hazards and quality problems occur frequently, posing severe challenges to engineering management. Traditional engineering quality management mainly relies on manual inspections and sampling inspections to conduct potential hazard investigations and quality monitoring, which have problems such as low efficiency, poor accuracy, and insufficient real-time performance, and are difficult to meet the needs of modern engineering management.

[0003] In recent years, digital technologies have been widely applied in the engineering field, providing new ideas for solving engineering quality management problems. Among them, BIM (Building Information Modeling) technology provides a data foundation and a collaborative platform for the whole life cycle management of engineering by digitally expressing and integrally managing the three-dimensional geometric information, physical information, functional characteristics, etc. of construction projects. However, due to the lack of effective on-site data collection and fusion means, the BIM model is difficult to reflect the dynamic changes at the engineering site in real time and cannot support the rapid discovery and handling of potential hazards and quality problems.

[0004] At the same time, the rapid development of artificial intelligence technology provides a new tool for solving the perception and decision-making problems in complex engineering scenarios. AI technologies such as computer vision and deep learning have made remarkable progress in object detection, semantic segmentation, 3D reconstruction, etc., making it possible to intelligently perceive and extract information from complex environments. Combining artificial intelligence with BIM technology to build a digital twin of the engineering site, realizing real-time perception of the on-site state, automatic discovery and identification of potential hazards, and intelligent analysis and early warning of quality problems, has become an inevitable way to improve the level of engineering quality management.

[0005] However, existing research is mainly limited to the application of local problems and single data, and there is still a lack of overall top-level design and systematic technical solutions. Engineering quality management involves multiple links such as panoramic perception, multi-source data fusion, potential hazard identification, positioning guidance, and information integration. How to build an overall solution for the entire engineering scenario, process, and element, break through the data barrier, achieve seamless connection of each link, and form a replicable and popularizable intelligent quality management model is a key problem that urgently needs to be solved. Summary of the Invention

[0006] In view of the problem of untimely hidden danger investigation at the engineering site in the existing technology, the present application provides an engineering quality management method and system based on an AI large model. The risk control model is used to perform semantic segmentation on the panoramic image to form grid regions, and hidden danger target detection and positioning are carried out in each grid region to obtain the hidden danger target image and its position coordinates. The hidden danger type is identified through the hidden danger classification model, and the image information of the local environment around the hidden danger target is collected. The hidden danger target image is fused with the local environment image, and finally, through the three-dimensional reconstruction technology, the fused image information is mapped into the BIM model to construct a digital twin model of the engineering site, etc., improving the efficiency of hidden danger investigation.

[0007] The object of the present application is achieved through the following technical solutions.

[0008] One aspect of the present application provides an engineering quality management method based on an AI large model, including: S1, installing panoramic equipment on the tower crane at the engineering site, and collecting panoramic images through the panoramic equipment; S2, using the pre-constructed risk control model to perform semantic segmentation on the panoramic image to obtain panoramic images of multiple grid regions; S3, conducting hidden danger investigation on each grid region to obtain the position coordinates of the hidden danger target; and according to the position coordinates of the hidden danger target, collecting the hidden danger target image of the hidden danger target through the panoramic equipment; S4, inputting the hidden danger target image into the pre-trained hidden danger classification model to classify the hidden danger target image, obtaining the hidden danger target type label and position coordinates; S5, collecting local images within the preset range of the hidden danger target according to the hidden danger target type label and position coordinates; S6, fusing the hidden danger target image with the corresponding local image to obtain a fused image containing semantic description and environmental information; S7, mapping the fused image into the BIM model through three-dimensional reconstruction to obtain a digital twin model of the engineering site, and using the digital twin model for engineering quality management.

[0009] Further, the types of the hidden danger investigation include open fire monitoring, smoke monitoring, safety helmet monitoring, reflective vest monitoring, and bare soil coverage monitoring.

[0010] Further, in S2, use the pre-built risk control model to perform semantic segmentation on the panoramic image to obtain a panoramic image of multiple grid regions, including: S21, construct a risk control model using PanoNet; S22, use the risk control model to perform feature extraction and pixel-level classification on the panoramic image to obtain the semantic segmentation result of the panoramic image; when performing feature extraction, unfold the panoramic image into a planar image according to the rectangular projection method, and use the SphericalConv convolution kernel to perform feature extraction on the planar image; according to the pixel-level classification result, divide the panoramic image into a foreground region and a background region; S23, according to the semantic segmentation result, divide the divided foreground region into NxN grids; S24, output the divided grid regions to obtain a panoramic image containing grid images and corresponding position information; the grid image is a local image within each grid region, and the position information includes the coordinate position and size of each grid in the panoramic image.

[0011] Among them, SphericalConv is a spherical convolution kernel specifically used for processing panoramic images. Different from traditional two-dimensional convolution kernels, SphericalConv performs convolution operations on images in the spherical coordinate system, which can effectively capture the spatial features in panoramic images. It projects and transforms the spherical image to unfold it into a planar image, and then applies convolution operations in the planar coordinate system. This method can well adapt to the geometric characteristics of panoramic images and extract local and global feature representations of panoramic images.

[0012] Further, in S3, according to the position coordinates of the potential hazard target, collect the potential hazard target image of the potential hazard target through the panoramic device, including: S31, for the panoramic image of each grid region, use the improved object detection algorithm PanoLoc to detect and locate the potential hazard target to obtain the bounding box coordinates of the potential hazard target; S32, according to the bounding box coordinates of the potential hazard target, use spherical central projection to calculate the central point coordinates of the potential hazard target in the panoramic image of the grid region as the relative position coordinates of the potential hazard target in the corresponding grid; S33, map the relative position coordinates of the potential hazard target to the coordinate system of the panoramic image of the grid region to obtain the absolute position coordinates of the potential hazard target in the panoramic image; S34, according to the absolute position coordinates of the potential hazard target, collect the local image centered on the potential hazard target through the panoramic device as the potential hazard target image; S35, output the absolute position coordinates of the potential hazard target and the potential hazard target image as the input data for S4 hazard classification.

[0013] Among them, PanoLoc is an object detection algorithm specifically for panoramic images. Different from common object detection algorithms such as Faster R-CNN and YOLO, PanoLoc fully considers the spherical geometric characteristics of panoramic images. It first uses SphericalConv to extract multi-scale features of panoramic images, and then generates potential target regions through a Region Proposal Network. After that, these regions are classified and position regression is performed to obtain the final target bounding box. While detecting, PanoLoc can accurately predict the position of the target in the spherical coordinate system.

[0014] Spherical central projection is a method of projecting points in the spherical coordinate system onto the planar coordinate system. In panoramic image processing, it is usually necessary to convert between spherical and planar coordinate systems. Spherical central projection can map points on the sphere (such as longitude and latitude) to pixel coordinates on a two-dimensional plane. Specifically, it determines the corresponding position of the spherical point on the plane by connecting the center of the sphere with the point on the sphere and calculating the intersection of this line with the projection plane. This projection method can unfold the panoramic image into a rectangular planar image for subsequent processing and analysis.

[0015] Furthermore, in S31, an improved object detection algorithm PanoLoc is used to detect and locate potential hazard targets, obtaining the bounding box coordinates of potential hazard targets, including: inputting the panoramic image of the grid area into the feature extraction network of the object detection algorithm PanoLoc, extracting multi-scale features through a convolutional neural network to obtain the feature map of the grid image; inputting the feature map of the grid image into the Region Proposal Network (RPN) of the object detection algorithm PanoLoc to generate candidate regions, and calculating the confidence score and preliminary bounding box coordinates corresponding to the candidate regions; inputting the candidate regions and the feature map into the detection and location branch of the object detection algorithm PanoLoc, classifying and regressing the candidate regions through Region of Interest (ROI) pooling and fully connected layers to obtain the class probability of potential hazard targets and the corrected bounding box coordinates; associating the corrected bounding box coordinates with the position information of the grid image in S24 to obtain the bounding box coordinates of potential hazard targets in the panoramic image of the grid area.

[0016] Further, S4, classifies the hidden danger target images to obtain hidden danger target type labels and position coordinates, including: S41, collects panoramic images of the construction site, and annotates the collected panoramic images, locates and classifies the hidden danger targets in each image, annotates open flames, smoke, not wearing a safety helmet, not wearing reflective clothing and bare soil cover, and generates corresponding hidden danger type labels to obtain a training set; S42, according to the training set, constructs a hidden danger classification model based on the DenseNet network, and sets the last layer of the hidden danger classification model to a fully connected layer, and the number of neurons in the fully connected layer is 5; S43, inputs the hidden danger target image output by S35 and the corresponding absolute position coordinates into the hidden danger classification model, performs feature extraction and classification prediction, and obtains the hidden danger target type label representing the hidden danger type, and the position coordinates representing the hidden danger location.

[0017] Further, S5, according to the hidden danger target type label and position coordinates, collect a local image within a preset range of the hidden danger target, including: S51, according to the hidden danger target type label and position coordinates, determine the center position (x c ,y c ); S52, according to the hidden danger target type label, determine the standard width w and height h of the corresponding hidden danger target; calculate the center position (x c ,y c ) is the width of the rectangular area centered on b and height h b ; with (x c ,y c ) is the center, w b is the major axis, h b As the short axis, a circumscribed ellipse is generated on the panoramic image, and the inner area of ​​the circumscribed ellipse is used as the preset range of the hidden danger target; S53, according to the internal parameter matrix K and the external parameter matrix T of the panoramic device, a projection relationship between the panoramic image and the imaging plane of the panoramic device is established; each pixel point (x, y) in the circumscribed ellipse is converted into a corresponding point (u, v) on the imaging plane of the panoramic device through the projection relationship; according to the corresponding point (u, v) on the imaging plane, the minimum circumscribed rectangular area is determined as the local imaging area; S54, the image of the local imaging area is collected by the panoramic device to obtain a local image within the preset range of the hidden danger target.

[0018] S6. Integrate the potential hazard target image with the corresponding local image to obtain a fused image containing semantic descriptions and environmental information, including: preprocessing the potential hazard target image and the local image, including size normalization, brightness adjustment, contrast enhancement, etc., to make the data distributions of the two images more consistent; input the preprocessed potential hazard target image and local image into a pre-trained image fusion model, extract multi-scale features of the two images through a feature extraction network, and obtain the feature map F_t of the potential hazard target image and the feature map F c .

[0019] Generate an attention matrix A with the same size as F t and F c . Each element a ij in the matrix represents the correlation between the i-th position in F t and the j-th position in F c . Calculate the weighted sum of F t and F c according to the attention matrix A to obtain the fused feature map F f , that is Input the fused feature map F f into a decoding network, and generate a fused image with the same size as the input image through multi-layer transposed convolution and upsampling operations; the fused image contains both high-resolution detailed features of the potential hazard target and the surrounding environmental information where the hazard occurs.

[0020] Furthermore, S7. Map the fused image to the BIM model through 3D reconstruction to obtain a digital twin model of the engineering site, including: S71. Register the fused image obtained in S6 with the panoramic image to obtain the mapping position and orientation of the fused image in the panoramic image; S72. According to the position and attitude parameters of the panoramic device, map the panoramic image to 3D space to obtain 3D point cloud data; map the fused image to the corresponding position in the 3D point cloud data through spatial transformation to obtain a colored point cloud with potential hazard target information; filter and denoise the colored point cloud and downsample it to obtain a sparse point cloud; S73. According to the sparse point cloud, use the incremental reconstruction algorithm PMVS to generate a dense point cloud to obtain a 3D real-scene model of the engineering site; S74. According to the BIM model of the engineering site, register the 3D real-scene model, and perform point cloud registration with the BIM model through the ICP algorithm to obtain the mapping relationship between the 3D real-scene model and the BIM model; S75. According to the mapping relationship between the 3D real-scene model and the BIM model, add the potential hazard target information to the corresponding components of the BIM model to obtain a BIM model integrated with hazard information as the digital twin model of the engineering site; S75. According to the digital twin model, obtain the position of the potential hazard target in the BIM component and the type of the potential hazard target, and generate a hazard rectification task.

[0021] Further, in S72, the panoramic image is mapped into a three-dimensional space to obtain three-dimensional point cloud data, including: obtaining the position coordinates (x, y, z) and attitude angles (roll, pitch, yaw) of the panoramic device, and constructing the external parameter matrix T of the panoramic device according to the position coordinates (x, y, z) and attitude angles (roll, pitch, yaw); establishing the mapping relationship between the panoramic image pixel coordinate system and the panoramic device camera coordinate system according to the internal parameter matrix K of the panoramic device; according to the external parameter matrix T and the internal parameter matrix K, through matrix multiplication [X, Y, Z, 1] = K × T × [u, v, 1], each pixel point (u, v) on the panoramic image is projected into the three-dimensional space to obtain the corresponding three-dimensional point coordinates (X, Y, Z), forming three-dimensional point cloud data; where (u, v) are the pixel coordinates of the panoramic image, and (X, Y, Z) are the three-dimensional space coordinates; in the three-dimensional point cloud data, the sub-point cloud corresponding to the fused image region is extracted, and the coordinate system of the sub-point cloud is aligned with the coordinate system of the panoramic image through rigid body transformation to realize the spatial registration of the fused image and the three-dimensional point cloud data; according to the registered sub-point cloud, the pixel points in the fused image are mapped into the three-dimensional space to obtain the color point cloud data corresponding to the fused image, and each point contains RGB color information and three-dimensional coordinate information; the mapped color point cloud data is merged with the original three-dimensional point cloud data to update the color information of the corresponding points in the three-dimensional point cloud data, and the fused color point cloud data with hidden danger target information is obtained.

[0022] Another aspect of the present application also provides an engineering quality management system based on an AI large model for executing an engineering quality management method based on an AI large model of the present application.

[0023] Compared with the prior art, the advantages of the present application are as follows:

[0024] By deploying panoramic image acquisition devices at the construction site, the image data of the engineering site can be comprehensively collected. The risk control model is used to perform semantic segmentation on the panoramic image to form a grid area, dividing the complex panoramic scene into multiple local areas, which simplifies the difficulty of subsequent hidden danger target detection. Among them, the risk control model adopts the PanoNet network structure, introduces the SphericalConv convolution kernel to process the special data format of the panoramic image, extracts the multi-scale features of the panoramic image, and can accurately and efficiently realize the semantic segmentation of the panoramic image.

[0025] Use the improved object detection algorithm PanoLoc to detect and locate potential hazard targets in the grid area. By extracting multi-scale features and generating candidate regions of potential hazard targets through the Region Proposal Network, and further classifying and regressing the candidate regions by combining Region of Interest Pooling, the position and bounding box information of potential hazard targets can be accurately detected in the local images of the grid area. At the same time, introduce the spherical central projection algorithm to map the detected position of potential hazard targets into the panoramic image coordinate system.

[0026] Construct a dedicated potential hazard classification model to classify and identify the images of detected potential hazard targets and determine the specific types of potential hazards. This classification model adopts the DenseNet network structure, fully excavates and fuses the multi-scale features of potential hazard targets through densely connected convolutional layers, and combines the fully connected layer to classify and judge the types of potential hazards. At the same time, this method collects a large amount of potential hazard data on the construction site and annotates it, constructs a dedicated potential hazard classification dataset, and through training, obtains a potential hazard classification model with strong robustness and good generalization ability, which can accurately identify different types of potential hazards.

[0027] For the local area around the potential hazard target, use panoramic equipment to supplement and collect environmental information. By fusing the detected image of the potential hazard target with the local environmental image, a fused image containing richer semantic information and scene information can be obtained, which can more comprehensively represent the potential hazard situation. Among them, this method adaptively determines the spatial range of the local area according to the type of potential hazard, maps the panoramic image coordinates to the panoramic camera imaging plane coordinates using the projection relationship, and then obtains an accurate local imaging area, improving the accuracy and controllability of local image acquisition. Image fusion makes full use of the environmental information of panoramic data and provides a more comprehensive data perspective for potential hazard analysis.

[0028] Map the fused image into the BIM model through 3D reconstruction technology to generate a digital twin model of the engineering site, realizing the accurate association and visual expression of potential hazard information and BIM components. This method maps the potential hazard information in the image space into the 3D space of the BIM model through a series of processes such as registration of panoramic images and point cloud data, filtering and reconstruction of point cloud data, and registration of point cloud and BIM model, so that the potential hazard targets form accurate spatial positioning and semantic associations in the BIM components. Integrating the potential hazard information into the BIM model forms a digital twin model with rich information and intuitive visualization. Brief Description of the Drawings

[0029] This application will be further described in the form of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0030] Figure 1It is a schematic diagram of an exemplary application scenario of an engineering quality management method based on an AI large model shown in some embodiments of the present application;

[0031] Figure 2 It is an exemplary flowchart of an engineering quality management method based on an AI large model shown in some embodiments of the present application;

[0032] Figure 3 It is an exemplary flowchart of generating a hidden danger target image shown in some embodiments of the present application;

[0033] Figure 4 It is an exemplary flowchart of generating a digital twin model of an engineering site shown in some embodiments of the present application. Detailed implementation manners

[0034] The methods and systems provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0035] As Figure 1 and Figure 2 shown, on the tower crane at the engineering site, a panoramic device is installed, and panoramic images are collected through the panoramic device; the pre-built risk control model is used to perform semantic segmentation on the panoramic images to obtain panoramic images of multiple grid regions; hidden danger investigation is carried out on each grid region to obtain the position coordinates of the hidden danger target; and according to the position coordinates of the hidden danger target, the hidden danger target image of the hidden danger target is collected through the panoramic device; the hidden danger target image is input into the pre-trained hidden danger classification model to classify the hidden danger target image to obtain the hidden danger target type label and the position coordinates; according to the hidden danger target type label and the position coordinates, local images within the preset range of the hidden danger target are collected; the hidden danger target image is fused with the corresponding local images to obtain a fused image containing semantic descriptions and environmental information; the fused image is mapped into the pre-established BIM model through 3D reconstruction to obtain a digital twin model of the engineering site, and the digital twin model is used for engineering quality management.

[0036] Specifically, in S1, considering that the tower crane can provide a high perspective and a wide coverage range, the panoramic device is selected to be installed on the tower crane, and 360-degree panoramic images of the engineering site are obtained through rotational shooting. The panoramic device adopts imaging methods such as multi-lens stitching or fisheye lenses, and generates complete panoramic images by stitching and synthesizing images from different perspectives. In order to obtain high-quality panoramic images, the panoramic device needs to perform precise internal and external parameter calibration to ensure the stitching accuracy and color consistency between the lenses. At the same time, the shooting parameters of the panoramic device such as exposure time, white balance, etc. need to be dynamically adjusted according to the on-site environment to adapt to different lighting conditions.

[0037] In order to achieve large - scale and multi - angle data collection, multiple panoramic devices can be deployed at different heights and positions. The spatial coordinates of each device are determined based on the positional relationship between the devices, forming a panoramic image acquisition array. Each device is connected to the central control unit through wired or wireless means to achieve synchronous data collection and transmission.

[0038] The collected panoramic images need to be pre - processed, including image distortion correction, noise filtering, image enhancement, etc., to improve the image quality and provide more reliable input data for subsequent semantic segmentation and hazard detection. At the same time, in order to reduce the transmission and storage overhead, the original images can be compressed and encoded to reduce the data volume while ensuring the quality.

[0039] S2. Use the pre - constructed risk control model to perform semantic segmentation on the panoramic images, dividing the complex panoramic scene into multiple grid - shaped regions, which simplifies the difficulty of subsequent hazard target detection. Specifically, when implementing, the panoramic image is unfolded into a planar image according to the rectangular projection method, and the PanoNet network structure and SphericalConv convolution kernel are used for feature extraction and pixel - level classification to obtain the semantic segmentation result of the panoramic image. Then, according to the segmentation result, the panoramic image is divided into a foreground region and a background region, and NxN grids are divided on the foreground region, and finally, structured data including grid images and corresponding position information is output.

[0040] S21. Use PanoNet as the basic network architecture of the risk control model. PanoNet is a convolutional neural network specifically designed for panoramic images. By performing convolution operations in the spherical coordinate system, it can effectively process the special data format of 360 - degree panoramic images. The PanoNet network includes multiple convolutional layers and pooling layers, and continuously extracts and abstracts image features to generate high - level semantic representations.

[0041] S22. Use the constructed PanoNet risk control model to perform end-to-end semantic segmentation on the panoramic image. First, unfold the panoramic image into a planar image according to the rectangular projection method, mapping the 360-degree spherical information into a 2D planar coordinate system for subsequent convolution operations. During the feature extraction process, use the SphericalConv convolution kernel to replace the traditional rectangular convolution kernel to extract features from the planar image. SphericalConv can better adapt to the data characteristics of panoramic images by defining convolution operations in the spherical coordinate system and extract the unique semantic information of panoramas. Through layer-by-layer convolution and downsampling, generate multi-scale feature maps of the image. Finally, through upsampling and concatenation operations, fuse the feature maps of different scales to generate a pixel-level segmentation result. The segmentation result divides each pixel of the image into different semantic categories, such as ground, buildings, mechanical equipment, construction materials, etc. According to the segmentation result, divide the image into a foreground area (the construction scene of interest) and a background area (non-key areas such as the sky and distant views).

[0042] S23. Further divide the foreground area generated by semantic segmentation into NxN grids. The choice of grid size needs to balance the computational complexity and regional granularity, and is generally set according to the resolution of the panoramic image and the requirements of object detection. The grid processing divides the large-scale panoramic image into multiple small-scale local areas, and each grid corresponds to a specific area of the construction site. Through gridification, the global detection task is transformed into multiple local detection tasks, reducing the complexity of the problem.

[0043] S24. Output the divided grid areas to generate structured data including grid images and corresponding position information. Among them, the grid image is a local image within each grid area cropped from the original panoramic image, representing a local view of the construction site. The position information records the spatial position and size of each grid in the panoramic image for subsequent spatial mapping and 3D reconstruction. Through grid output, the continuous panoramic image is discretized, facilitating parallel processing and data organization management.

[0044] This application transforms the complex panoramic image into a structured local representation through semantic segmentation and grid processing, laying a foundation for subsequent hidden danger detection. Among them, the use of PanoNet and SphericalConv solves the problem of processing the special format of panoramic images. Semantic segmentation realizes the automatic understanding and regional division of image content, and gridification simplifies the scale of the detection problem.

[0045] Such as Figure 3As shown in the figure, in S3, based on the grid regions generated in S2, the detection and positioning of potential hazard targets are performed for each grid region, the position coordinates of the potential hazard targets are obtained, and local images of the potential hazard targets are collected by panoramic devices according to the position coordinates, providing input data for subsequent potential hazard classification.

[0046] S31, The improved object detection algorithm PanoLoc is used to detect and locate potential hazard targets in the panoramic images of each grid region. Since panoramic images have a large field of view and resolution, the sizes and proportions of potential hazard targets in the images vary greatly. To adapt to this scale change, PanoLoc adopts a multi-scale feature extraction strategy. Specifically, the feature extraction network adopts a structure similar to FPN (Feature Pyramid Network), generating feature maps of multiple scales at different levels of the backbone network. Taking ResNet-50 as an example, PanoLoc selects the outputs of four stages, namely Conv2_x, Conv3_x, Conv4_x, and Conv5_x, as multi-scale features. The size of the feature map corresponding to Conv2_x is 1 / 4 of the input image, Conv3_x is 1 / 8, Conv4_x is 1 / 16, and Conv5_x is 1 / 32. Through multi-scale feature extraction, PanoLoc can capture the visual features of potential hazard targets at different receptive fields and abstraction levels, improving the robustness of detection.

[0047] To efficiently screen out candidate regions that may contain potential hazard targets from the images, PanoLoc adopts the Region Proposal Network (RPN). RPN is a two-stage object detection method based on sliding windows. By densely sampling anchor boxes at a fixed stride on the feature map and performing binary classification (whether it contains a target) and bounding box regression for each anchor box, candidate region proposals are generated. In PanoLoc, considering the perspective distortion characteristics of panoramic images, the design of the anchor boxes is optimized, and SphericalAnchor defined in the spherical coordinate system is adopted. SphericalAnchor is generated by uniformly sampling on the sphere and can better adapt to the spatial distribution characteristics of panoramic data. At the same time, the sliding window stride and Anchor size of RPN are also adaptively adjusted according to the resolution of the panoramic image and the target size to balance detection accuracy and efficiency.

[0048] For the candidate regions generated by RPN, since their positions and sizes are not fixed, they cannot be directly fed into the subsequent classification and regression branches. Therefore, PanoLoc introduces a Region of Interest (ROI) pooling layer to map candidate regions of different sizes into feature representations of a fixed size. Different from traditional ROIPooling, PanoLoc adopts the ROIAlign operation to achieve more refined feature alignment through bilinear interpolation, reducing feature distortion and information loss. In the specific implementation, PanoLoc samples 7x7 grid points on the feature map according to the bounding box coordinates of the candidate regions, and calculates the feature values of each grid point through bilinear interpolation to obtain a fixed-size 7x7x256 ROI feature map as the input for classification and regression.

[0049] After obtaining the ROI feature map of the candidate regions, PanoLoc classifies and regresses the candidate regions through two parallel fully-connected layer branches. The classification branch predicts the probabilities of the candidate regions belonging to various hidden danger targets through the Softmax layer, and the regression branch corrects the positions and sizes of the candidate regions through a bounding box regressor to more accurately frame the hidden danger targets. During the training process, PanoLoc uses a weighted combination of classification loss (such as cross-entropy loss) and regression loss (such as SmoothL1 loss) as the optimization objective, and performs end-to-end learning on the network parameters through backpropagation and gradient descent algorithms. In the inference stage, the final hidden danger target detection results are obtained through post-processing operations such as setting a probability threshold and non-maximum suppression (NMS).

[0050] The bounding box coordinates of the hidden danger targets detected by PanoLoc are associated with the grid image position information of S24, and through coordinate transformation and mapping, the precise coordinates of the hidden danger targets in the panoramic image of the grid area are obtained. This process realizes the spatial information association from local to global.

[0051] S32. To determine the relative position of the hidden danger targets in the grid, PanoLoc adopts the method of spherical central projection to convert the detected bounding boxes of the hidden danger targets from the planar coordinate system to the spherical coordinate system, and calculates the spherical center point coordinates as the relative position coordinates. Specifically, assume that the four vertex coordinates of the bounding box of the hidden danger target in the planar rectangular coordinate system (x, y) are (x1, y1), (x2, y2), (x3, y3), (x4, y4), where x represents the width direction of the image and y represents the height direction. PanoLoc first converts the vertex coordinates in the planar coordinate system to the planar coordinate system (x', y') with the center of the panoramic image as the origin according to the aspect ratio of the panoramic image (such as 2:1) and the viewing angle (such as 360°x180°), and then according to the spherical coordinate system Calculate the coordinates of the four vertices on the spherical surface based on the correspondence with the planar coordinate system (x', y'). Where θ represents the azimuth angle and φ represents the zenith angle, and the conversion formulas are as follows: Where r represents the radius of the spherical surface, usually set to 1. Based on the spherical coordinates of the four vertices, PanoLoc calculates their arithmetic mean As the spherical center point coordinates of the potential hazard target, that is: Through the above calculations, PanoLoc obtains the relative position coordinates of the potential hazard target in the spherical coordinate system Indicates the azimuth angle and zenith angle offset of the potential hazard target relative to the center of the grid area where it is located. The relative position coordinates establish a local spatial association with the grid area, facilitating subsequent spatial information fusion.

[0052] S33, PanoLoc further maps the relative position coordinates of the potential hazard target to the global coordinate system of the panoramic image of the grid area to obtain the absolute position coordinates of the potential hazard target. Specifically, assume that the spherical coordinate range of the grid area in the panoramic image is By combining the relative position coordinates with the boundary coordinates of the grid area, the absolute spherical coordinates of the potential hazard target are obtained

[0053] Where 0 < θ ≤ 2π and Represent the value ranges of the azimuth angle and zenith angle respectively. Through the above coordinate mapping, PanoLoc extends the position information of the potential hazard target from the grid local space to the global space of the panoramic image, establishing a spatial association between the potential hazard target and the entire image. The absolute position coordinates not only represent the precise position of the potential hazard target in the panoramic image but also provide a spatial reference for subsequent 3D reconstruction and visualization management.

[0054] S34, PanoLoc obtains a local image centered on the potential hazard target through the directional acquisition of the panoramic device as the potential hazard target image. Specifically, PanoLoc first converts the absolute spherical coordinates of the potential hazard target to the three-dimensional direction vector (x, y, z) in the camera coordinate system, and then controls the pan angle (p a , t a ) and zoom factor (z) of the panoramic camera to align the camera's field of view with the potential hazard target, where: p a = θ abs , Among them, f represents the camera focal length, and fov represents the camera field of view angle. By setting appropriate shooting parameters, such as the field of view angle fov = 60°, and the focal length f = 800 pixels, PanoLoc controls the panoramic camera to collect local images centered on the potential hazard target, obtaining a high-resolution image focused on the potential hazard target as the potential hazard target image. Compared with the panoramic image of the grid area, the potential hazard target image contains richer and more detailed potential hazard feature information, such as the shape, texture, color, etc. of the potential hazard target, providing a targeted discrimination basis for subsequent potential hazard classification.

[0055] Finally, in S35, the absolute position coordinates of the potential hazard target and the collected potential hazard target image are packaged and output as the input data for the S4 potential hazard classification step. The position coordinates are used to identify the spatial position information of the potential hazard target, and the potential hazard target image provides the visual feature information of the potential hazard target. The combination of the two provides the necessary prior knowledge and discrimination basis for potential hazard classification.

[0056] S4, input the potential hazard target image into the pre-trained potential hazard classification model to classify the potential hazard target image, obtaining the potential hazard target type label and position coordinates, including: S41 To construct the potential hazard classification model, first, a certain amount of construction site panoramic images need to be collected as training data. In the data annotation stage, through manual inspection, the potential hazard targets appearing in each panoramic image are located and classified, and common potential hazard types such as open fire, smoke, not wearing a safety helmet, not wearing a reflective vest, and uncovered bare soil are marked, and the corresponding potential hazard type labels are generated. Taking 100 panoramic images of a construction site as an example, through annotation, the position coordinates and potential hazard type labels of the potential hazard targets are obtained, forming a training set containing 2000 potential hazard target samples. Among them, there are 500 cases of open fire hazards, 200 cases of smoke hazards, 800 cases of not wearing a safety helmet hazards, 300 cases of not wearing a reflective vest hazards, and 200 cases of uncovered bare soil hazards. The training set obtained through manual annotation provides high-quality supervision information for subsequent model training.

[0057] S42. Based on the collected training set data, a hidden danger classification model based on the DenseNet network was constructed. DenseNet is a densely connected convolutional neural network structure. By introducing dense connections between different layers of the network, the reuse and fusion of features are achieved, improving the expression ability and generalization performance of the model. In the hidden danger classification task, the last layer of DenseNet is set as a fully connected layer containing 5 neurons, and each neuron corresponds to a type of hidden danger. The output of the neuron is converted into a probability distribution of the hidden danger type through the Softmax function. The input of the model is a 256x256 hidden danger target image, and the output is a 5-dimensional probability vector of the hidden danger type. During the training process, the cross-entropy loss function and the Adam optimizer are used to learn the model parameters through iterative optimization, enabling it to accurately classify the hidden danger target image into the corresponding hidden danger type.

[0058] S43. The hidden danger target image output by S35 and the corresponding absolute position coordinates are input into the pre-trained hidden danger classification model, and image features are extracted and classified for prediction through forward propagation. Specifically, the hidden danger classification model first extracts features from the input 256x256 hidden danger target image. Through multiple convolutional and pooling operations of the DenseNet network, a high-dimensional feature representation vector is obtained. Then, the feature representation vector is input into the fully connected layer. After linear transformation and Softmax normalization, a 5-dimensional probability distribution vector of the hidden danger type is obtained, and each element represents the confidence that the hidden danger target image belongs to a certain hidden danger type. By setting a confidence threshold (such as 0.5), the hidden danger type with the highest confidence and exceeding the threshold is used as the final classification result, and the hidden danger target type label representing the hidden danger type is output. At the same time, the hidden danger classification model also directly outputs the absolute position coordinates obtained in S35, indicating the spatial position of the hidden danger in the panoramic image.

[0059] S5. According to the type label and position coordinates of the hidden danger target, adaptively determine the preset range of the hidden danger target in the panoramic image, and collect the local image within this range to obtain targeted hidden danger target area data, including: In S51, using the hidden danger target type label and position coordinates output by S4, determine the central position (x c , y c ) of the hidden danger target in the panoramic image. Specifically, the position coordinates are usually represented as the upper left and lower right coordinates (x min , y min , x max , y max ) of the bounding box of the hidden danger target. By calculating the center point coordinates of the bounding box, the central position of the hidden danger target in the panoramic image is obtained:

[0060] In S52, according to the type label of the potential hazard target, the standard width w and height h of the corresponding potential hazard target are determined. Different types of potential hazard targets exhibit different sizes and appearance characteristics in the image. By statistically analyzing the geometric sizes of a large number of potential hazard samples, the standard aspect ratios of each potential hazard type are obtained. For example, the standard aspect ratio of not wearing a safety helmet is 1:1.2, and the standard aspect ratio of an open fire is 1.5:1. Then, with the center position (x c , y c ) of the potential hazard target as the center, the width w b and height h b of the circumscribed rectangular area are calculated according to the standard aspect ratio: w b = k × w, h b = k × h, where k is a scale factor used to control the magnification degree of the circumscribed rectangular area relative to the potential hazard target, and usually takes a value of 1.2 to 1.5. By reasonably setting the scale factor, while covering the potential hazard target, the range of the region of interest can be appropriately expanded.

[0061] To better adapt to the irregular shape of the potential hazard target, in S52, further with (x c , y c ) as the center, w b as the major axis, and h b as the minor axis, an circumscribed ellipse is generated on the panoramic image, and the area inside the ellipse is used as the preset range of the potential hazard target. Compared with the rectangular area, the elliptical area can cover the potential hazard target more compactly and reduce the interference of irrelevant background. The ellipse equation is:

[0062] By judging whether each pixel point (x, y) on the image satisfies the ellipse equation, a set of pixel points belonging to the preset range of the potential hazard target is obtained.

[0063] In S53, in order to map the pixel points within the preset range of the potential hazard target to the imaging plane of the panoramic device, it is necessary to establish the projection relationship between the panoramic image and the imaging plane. This process involves the internal parameter matrix K and the external parameter matrix T of the panoramic device. The internal parameter matrix K describes the internal imaging geometric characteristics of the panoramic device, including parameters such as focal length and optical center; the external parameter matrix T describes the pose transformation relationship of the panoramic device relative to the global coordinate system, including the rotation matrix and the translation vector. Through the combined action of the internal and external parameters, the conversion between the two-dimensional image coordinate system and the three-dimensional imaging plane coordinate system is achieved.

[0064] For each pixel point (x, y) in the interior region of the ellipse, it is transformed into the corresponding point (u, v) on the imaging plane through the internal parameter matrix K and the external parameter matrix T: [u, v, 1]' = K × T × [x, y, 1]', where [x, y, 1]' represents the homogeneous coordinates of the pixel point, and [u, v, 1]' represents the homogeneous coordinates of the corresponding point on the imaging plane. f x , f y represents the focal length, c x , c y represents the optical center coordinates, R represents the rotation matrix, and t represents the translation vector; by performing coordinate transformation on all pixel points inside the ellipse, the preset range of the hidden danger target on the imaging plane is obtained. To facilitate the acquisition of local images, the minimum bounding rectangle of this preset range is further calculated as the local imaging area on the imaging plane.

[0065] In S54, the panoramic device is used to collect the image of the local imaging area to obtain a high-resolution local image within the preset range of the hidden danger target. Specifically, by controlling the pan-tilt rotation and zoom parameters of the panoramic camera, the camera's field of view is aligned with the local imaging area, and the focal length and resolution of the camera are dynamically adjusted according to the size of this area to obtain a clear and complete close-up image of the hidden danger target. The resolution of the local image is usually higher than that of the panoramic image, which can provide more detailed and comprehensive feature information such as the texture and shape of the hidden danger target, providing a richer basis for hidden danger analysis.

[0066] S6, the hidden danger target image is fused with the corresponding local image to obtain a fused image containing semantic descriptions and environmental information, including: in the data preprocessing stage, operations such as size normalization, brightness adjustment, and contrast enhancement are performed on the hidden danger target image and the local image to make the data distributions of the two images more consistent and reduce the differences between the images. Specifically, first, the two images are scaled to a unified size (such as 256x256), and then the brightness distribution of the images is adjusted through the histogram equalization algorithm to make it more uniform at the gray level. At the same time, by stretching the pixel value range of the images, the contrast of the images is enhanced to highlight the texture details of the hidden danger target and the scene. The preprocessed images are more conducive to feature extraction and fusion operations.

[0067] In the feature extraction stage, the preprocessed hidden danger target image and local image are input into a pre-trained image fusion model, and multi-scale features of the two images are extracted through a feature extraction network. The image fusion model adopts an encoder-decoder architecture. The encoder part consists of multiple convolutional layers and pooling layers, which are responsible for extracting hierarchical features of the image. By setting different receptive fields and downsampling rates, the encoder generates a multi-scale feature map containing global information and local details of the image. Taking ResNet-50 as an example, feature maps of different sizes such as 64x64, 32x32, 16x16, and 8x8 are extracted, representing semantic information of the image at different abstraction levels. Denote the highest-level feature maps of the hidden danger target image and local image as Ft and Fc, which are used as inputs for subsequent fusion operations.

[0068] In the feature fusion stage, an attention mechanism is introduced. By calculating the correlation between F t and F c , the weights of the two feature maps are adaptively adjusted to achieve semantic feature alignment and fusion. First, an attention matrix A of the same size as F t and F c is generated. Each element a ij of the matrix represents the correlation between the i-th position in F t and the j-th position in F c . The calculation of the correlation is based on the similarity measure of the feature map. Commonly used measure functions include dot product, cosine similarity, etc. Intuitively, each row of the attention matrix A represents the correlation between a position in F t and all positions in F c , and each column represents the correlation between a position in F c and all positions in F t . The rows and columns are normalized through the Softmax function to obtain the final value of the attention matrix A.

[0069] According to the attention matrix A, calculate the weighted sum of F t and F c to obtain the fused feature map F f :

[0070] Among them, means to perform a weighted sum of each position of F t with F c , and the weights are determined by the attention matrix A; means to perform a weighted sum of each position of F c with F t , and the weight is 1 - A. Through the control of the attention matrix, the fused feature map F f semantically realizes F t and Fc Adaptive fusion, that is, when F t and F c have a high correlation at a certain position, the fusion result is more biased towards the eigenvalue at that position; when the correlation is low, the fusion result more evenly considers the contributions of the two feature maps.

[0071] In the image reconstruction stage, the fused feature map F_f is input into the decoding network. Through multi-layer transposed convolution and upsampling operations, the spatial resolution of the image is gradually restored, and finally a fused image with the same size as the input image is generated. The structure of the decoding network is the opposite of that of the encoder. By using skip connections to splice the multi-scale feature maps of the encoder with the corresponding layers of the decoder, cross-layer feature transfer and refinement are achieved. During the generation process, the fused image not only inherits the high-resolution detail features of the hidden danger target image but also incorporates the surrounding environment information provided by the local image, forming a comprehensive image representation that includes the hidden danger target and the scene context.

[0072] This application introduces an attention mechanism to achieve adaptive fusion of hidden danger target features and scene information. Traditional image fusion methods, such as weighted average and pyramid fusion, usually adopt fixed fusion rules and are difficult to consider the semantic correlation of image content. The S6 scheme dynamically adjusts the fusion weights through an attention matrix, enabling the fusion result to adaptively highlight the features of relevant regions according to the image content, suppress the interference of irrelevant regions, and generate a more semantic and information-rich fused image.

[0073] As Figure 4 shown, in S7, the fused image is mapped into the BIM model through 3D reconstruction to obtain a digital twin model of the engineering site. The digital twin model is used for engineering quality management, including: S71, registering the fused image generated in S6 with the original panoramic image to determine the mapping position and orientation of the fused image in the panoramic image. Specifically, a set of corresponding feature point pairs are extracted from the two images through a feature point matching algorithm (such as SIFT, SURF, etc.), and then the homography transformation matrix H between the two images is estimated using the RANSAC algorithm. The homography transformation matrix H describes the projection relationship from the fused image to the panoramic image, and the pixel coordinates in the fused image can be mapped to the corresponding positions in the panoramic image through matrix multiplication. At the same time, according to the decomposition of the homography transformation matrix H, the rotation angle and scale factor of the fused image relative to the panoramic image can be obtained to determine the direction and size of the fused image in the panoramic image. The result of image registration provides a coordinate reference for subsequent 3D mapping.

[0074] In S72, according to the position and attitude parameters of the panoramic device, the panoramic image is mapped into the three-dimensional space by using the multi-view geometry principle to obtain the three-dimensional point cloud data. First, the position coordinates (x, y, z) and attitude angles (roll, pitch, yaw) of the panoramic device are obtained, and the external parameter matrix T of the panoramic device is constructed according to these parameters. The external parameter matrix T describes the rigid body transformation relationship between the camera coordinate system of the panoramic device and the world coordinate system, including the rotation matrix R and the translation vector t. Then, according to the internal parameter matrix K of the panoramic device, the mapping relationship between the panoramic image pixel coordinate system and the panoramic device camera coordinate system is established. The internal parameter matrix K contains parameters such as the focal length, optical center, and distortion coefficient of the camera, and describes the projection process of the three-dimensional space points on the camera imaging plane.

[0075] Through the combined action of the internal and external parameters, each pixel point (u, v) on the panoramic image is projected into the three-dimensional space to obtain the corresponding three-dimensional point coordinates (X, Y, Z). The projection process is realized by matrix multiplication: [X, Y, Z, 1] = K × T × [u, v, 1], where [u, v, 1] represents the homogeneous coordinates of the pixel point, and [X, Y, Z, 1] represents the homogeneous coordinates of the three-dimensional space point. By projecting all the pixel points of the panoramic image, a dense three-dimensional point cloud data is obtained, and each point contains three-dimensional coordinates and color information.

[0076] In order to integrate the hidden danger target information in the fusion image into the three-dimensional point cloud data, spatial transformation and mapping are required. First, the sub-point cloud corresponding to the fusion image area is extracted from the three-dimensional point cloud data, and the coordinate system of the sub-point cloud is aligned with the coordinate system of the fusion image through rigid body transformation (translation + rotation) to achieve the spatial registration of the two. The registration process can be completed by manual calibration or automatic optimization. Then, according to the registered sub-point cloud, each pixel point in the fusion image is mapped into the three-dimensional space to obtain the colored point cloud data corresponding to the fusion image. The mapping process is similar to the projection process of the panoramic image, and the pixel coordinates are converted into three-dimensional coordinates by matrix multiplication, and the pixel color is assigned to the corresponding three-dimensional point. Finally, the mapped colored point cloud data is merged with the original three-dimensional point cloud data to update the color information of the corresponding points in the three-dimensional point cloud data, and the fused colored point cloud data with hidden danger target information is obtained. In order to improve the efficiency and robustness of subsequent processing, the colored point cloud data is filtered and denoised and downsampled. Statistical filtering methods (such as voxel filtering, mean filtering, etc.) are used to remove the outlier points and noise in the point cloud, and the data volume of the point cloud is reduced by voxel downsampling or uniform downsampling to obtain a sparse point cloud.

[0077] In S73, based on the sparse point cloud, a dense point cloud is generated using the incremental reconstruction algorithm PMVS (Patch-based Multi-View Stereo) to obtain a three-dimensional real-scene model of the engineering site. PMVS extracts features, matches feature points, and performs triangulation in images from multiple viewpoints to incrementally recover the three-dimensional structure of the scene. The algorithm first extracts a set of initial seed points from the sparse point cloud, and then, centered on the seed points, performs local matching within their neighborhoods through photometric consistency constraints and geometric constraints to obtain a set of locally reconstructed three-dimensional patches. By iteratively expanding and optimizing the three-dimensional patches, the density and coverage of the point cloud are continuously increased, and finally a dense three-dimensional point cloud model is obtained. The dense point cloud generated by PMVS not only contains the geometric structure of the scene but also the texture details of the object surface, and it is a high-precision and high-fidelity three-dimensional reconstruction method.

[0078] In S74, in order to associate the three-dimensional real-scene model with the BIM model, spatial registration of the two is required. First, based on the BIM model of the engineering site, the three-dimensional geometric information of the building components is extracted to obtain the point cloud representation of the BIM model. Then, the ICP (Iterative Closest Point) algorithm is used for point cloud registration between the three-dimensional real-scene model and the BIM model. The ICP algorithm finds the optimal rigid transformation (rotation + translation) between the two sets of point clouds through iterative optimization, minimizing the distance error between the registered point clouds. The registration process takes into account the geometric structure and semantic attributes of the point cloud, such as matching by extracting features such as building edges and corners, or using the type information of components for semantic-guided registration. Through ICP registration, the coordinate transformation relationship between the three-dimensional real-scene model and the BIM model is obtained, achieving geometric alignment of the two models.

[0079] In S75, according to the mapping relationship between the three-dimensional real-scene model and the BIM model, the hidden danger target information is added to the corresponding components of the BIM model to obtain a BIM model integrated with hidden danger information, which serves as the digital twin model of the engineering site. Specifically, by performing a spatial intersection operation between the hidden danger target point cloud in the three-dimensional real-scene model and the BIM model, the BIM components where the hidden danger targets are located are determined, and the attribute information such as the type, location, and size of the hidden danger targets is added to the parameter table of the components. At the same time, a three-dimensional model of the hidden danger target is generated at the corresponding position in the BIM model, and the fused image is used as the texture map of the model to achieve visual display of the hidden danger targets. In this way, the image space, point cloud space, and BIM semantic space are associated and fused to form a multi-dimensional and multi-scale digital twin model of the engineering site. The digital twin model not only reflects the actual progress and quality status of the project but also integrates multi-source heterogeneous data such as images, point clouds, and BIM, providing a globalized and semantic digital means for project management.

[0080] In S76, based on the digital twin model, further obtain the spatial location of the potential hazard target in the BIM component and the type attributes of the potential hazard target, generate potential hazard rectification tasks, and assist in engineering quality management. Specifically, by locating and querying the potential hazard target in the digital twin model, automatically extract the location information such as the name of the component where the potential hazard occurs, the floor, and the spatial area, as well as the attribute information such as the type, level, and description of the potential hazard, to form a structured list of potential hazard problems. Then, according to the pre-set potential hazard handling rules and processes, automatically generate a potential hazard rectification task sheet, including content such as potential hazard description, responsible person, rectification measures, and deadline requirements, and send it to the relevant responsible person through a mobile application or a workflow system. During the rectification process, track and record the potential hazard rectification status through the digital twin model to form a closed-loop management of potential hazards. At the same time, through the visualization interface of the digital twin model, intuitively display the distribution of potential hazards in the three-dimensional space and the rectification progress, providing decision-making support for project managers.

[0081] This application deeply integrates image semantic information and three-dimensional geometric information to construct an engineering site digital twin model integrating multi-source data. Traditional engineering quality management often relies on manual inspections and paper records, suffering from problems such as information silos and inefficiency. The S7 solution realizes the mapping and transformation of potential hazard targets from the image space to the three-dimensional space and then to the semantic space through the association and integration of images, point clouds, and BIM models, corresponding the detected potential hazard problems to the actual engineering scenarios and digital models, making potential hazard management more intuitive, accurate, and comprehensive. The management processes such as potential hazard location, traceability, and rectification based on the digital twin model improve the efficiency of potential hazard discovery and handling.

Claims

1. A project quality management method based on AI big model, characterized in that: include: S1, install panoramic equipment on the tower crane at the construction site and collect panoramic images through the panoramic equipment; S2, using a pre-built wind control model to perform semantic segmentation on the panoramic image to obtain panoramic images of multiple gridded areas; S3, conduct hidden danger inspection on each grid area and obtain the location coordinates of the hidden danger target; And according to the position coordinates of the hidden danger target, the hidden danger target image of the hidden danger target is collected through the panoramic equipment; S4, inputting the hidden danger target image into the pre-trained hidden danger classification model, classifying the hidden danger target image, and obtaining the hidden danger target type label and location coordinates; S5, collecting a local image within a preset range of the hidden danger target according to the hidden danger target type label and position coordinates; S6, fusing the hidden danger target image with the corresponding local image to obtain a fused image containing semantic description and environmental information; S7, maps the fused image to the pre-established BIM model through 3D reconstruction to obtain the digital twin model of the construction site, and uses the digital twin model to manage the construction quality.

2. The engineering quality management method based on AI big model according to claim 1 is characterized by: The types of hidden danger inspections include open flame monitoring, smoke monitoring, safety helmet monitoring, reflective clothing monitoring and bare soil cover monitoring.

3. The engineering quality management method based on AI big model according to claim 2 is characterized by: S2, uses the pre-built wind control model to perform semantic segmentation on the panoramic image to obtain panoramic images of multiple gridded areas, including: S21, uses PanoNet to build a risk control model; S22, using the wind control model to perform feature extraction and pixel-level classification on the panoramic image to obtain a semantic segmentation result of the panoramic image; During feature extraction, the panoramic image is expanded into a plane image using a rectangular projection method, and the SphericalConv convolution kernel is used to extract features from the plane image to obtain pixel-level classification results. According to the pixel-level classification results, the panoramic image is divided into foreground area and background area to obtain the semantic segmentation result; S23, dividing the segmented foreground area into NxN grids according to the semantic segmentation result; S24, output the divided gridded areas to obtain a panoramic image of the gridded areas including grid images and corresponding position information; the panoramic image of the gridded areas is a local image within each grid area, and the position information includes the coordinate position and size of each grid in the panoramic image.

4. The engineering quality management method based on AI big model according to claim 3 is characterized by: S3, according to the position coordinates of the hidden danger target, collecting the hidden danger target image of the hidden danger target through the panoramic device, including: S31, for each gridded area panoramic image, using the improved target detection algorithm PanoLoc to detect and locate the hidden danger target, and obtain the boundary box coordinates of the hidden danger target; S32, according to the bounding box coordinates of the hidden danger target, using spherical center projection to calculate the center point coordinates of the hidden danger target in the panoramic image of the gridded area as the relative position coordinates of the hidden danger target in the corresponding grid; S33, mapping the relative position coordinates of the hidden danger target to the coordinate system of the panoramic image of the gridded area to obtain the absolute position coordinates of the hidden danger target in the panoramic image; S34, according to the absolute position coordinates of the hidden danger target, a local image centered on the hidden danger target is collected by a panoramic device as a hidden danger target image; S35, outputting the absolute position coordinates of the hidden danger target and the hidden danger target image as input data for hidden danger classification in S4.

5. The engineering quality management method based on AI big model according to claim 4 is characterized by: S31, using the improved target detection algorithm PanoLoc to detect and locate the hidden danger target, and obtain the bounding box coordinates of the hidden danger target, including: The panoramic image of the gridded area is input into the feature extraction network of the target detection algorithm PanoLoc, and multi-scale features are extracted through the convolutional neural network to obtain the feature map of the grid image. The feature map of the grid image is input into the region proposal network RPN of the target detection algorithm PanoLoc to generate candidate regions and calculate the confidence scores and preliminary bounding box coordinates corresponding to the candidate regions; The candidate regions and feature maps are input into the detection and positioning branches of the target detection algorithm PanoLoc. The candidate regions are classified and regressed through the region of interest ROI pooling and the fully connected layer to obtain the category probability and corrected bounding box coordinates of the hidden danger target. The corrected bounding box coordinates are associated with the position information of the grid image in S24 to obtain the bounding box coordinates of the hidden danger target in the panoramic image of the gridded area.

6. The engineering quality management method based on AI big model according to claim 1 is characterized by: S4, classify the hidden danger target image to obtain the hidden danger target type label and location coordinates, including: S41, collecting panoramic images of the construction site, annotating the collected panoramic images, locating and classifying the hidden danger targets in each image, annotating open flames, smoke, not wearing a safety helmet, not wearing a reflective clothing, and bare soil, and generating corresponding hidden danger type labels to obtain a training set; S42, constructing a hidden danger classification model based on a DenseNet network according to the training set, setting the last layer of the hidden danger classification model to a fully connected layer, and the number of neurons in the fully connected layer is 5; S43, input the hidden danger target image and the corresponding absolute position coordinates output by S35 into the hidden danger classification model, perform feature extraction and classification prediction, and obtain the hidden danger target type label representing the hidden danger type and the position coordinates representing the hidden danger location.

7. The engineering quality management method based on AI big model according to claim 6 is characterized by: S5, according to the hidden danger target type label and location coordinates, collect the local image within the preset range of the hidden danger target, including: S51, determining the center position (x_c, y_c) of the hidden danger target in the panoramic image according to the hidden danger target type label and position coordinates; S52, determining the standard width w and height h of the corresponding hidden danger target according to the hidden danger target type label; Calculate the center position (x c ,y c ) is the width of the rectangular area centered on b and height h b ; With (x c ,y c ) is the center, w b is the major axis, h b As the short axis, a circumscribed ellipse is generated on the panoramic image, and the inner area of ​​the circumscribed ellipse is used as the preset range of the hidden danger target; S53, establishing a projection relationship between the panoramic image and the imaging plane of the panoramic device according to the intrinsic parameter matrix K and the extrinsic parameter matrix T of the panoramic device; Each pixel point (x, y) within the circumscribed ellipse is converted to a corresponding point (u, v) on the imaging plane of the panoramic device through a projection relationship; According to the corresponding point (u, v) on the imaging plane, the minimum circumscribed rectangular area is determined as the local imaging area. S54, using the panoramic device to collect images of the local imaging area, to obtain a local image within a preset range of the hidden danger target.

8. The engineering quality management method based on AI big model according to claim 7 is characterized by: S7, maps the fused image to the BIM model through 3D reconstruction to obtain a digital twin model of the construction site, including: S71, registering the fused image obtained in S6 with the panoramic image to obtain a mapping position and direction of the fused image in the panoramic image; S72, mapping the panoramic image into a three-dimensional space according to the position and posture parameters of the panoramic device, as well as the mapping position and direction, to obtain three-dimensional point cloud data; The fused image is mapped to the corresponding position in the three-dimensional point cloud data through spatial transformation to obtain a color point cloud with hidden danger target information; Filter, denoise and downsample the colored point cloud to obtain a sparse point cloud; S73, based on the sparse point cloud, using the incremental reconstruction algorithm PMVS to generate a dense point cloud as a three-dimensional real scene model of the engineering site; S74, registering the three-dimensional real scene model according to the BIM model of the project site, registering the point cloud with the BIM model through an ICP algorithm, and obtaining a mapping relationship between the three-dimensional real scene model and the BIM model; S75, according to the mapping relationship between the three-dimensional real scene model and the BIM model, the hidden danger target information is added to the corresponding component of the BIM model to obtain the BIM model integrated with the hidden danger information as the digital twin model of the engineering site; S75, according to the digital twin model, obtain the position of the hidden danger target in the BIM component and the type of the hidden danger target, and generate a hidden danger rectification task.

9. The engineering quality management method based on AI big model according to claim 8 is characterized by: S72, mapping the panoramic image into a three-dimensional space to obtain three-dimensional point cloud data, including: Obtain the position coordinates (x, y, z) and attitude angles (roll, pitch, yaw) of the panoramic device, and construct the external parameter matrix T of the panoramic device according to the position coordinates (x, y, z) and attitude angles (roll, pitch, yaw); According to the intrinsic parameter matrix K of the panoramic device, a mapping relationship between the panoramic image pixel coordinate system and the panoramic device camera coordinate system is established; According to the external parameter matrix T and the internal parameter matrix K, each pixel point (u, v) on the panoramic image is projected into the three-dimensional space through matrix multiplication [X, Y, Z, 1] = K × T × [u, v, 1] to obtain the corresponding three-dimensional point coordinates (X, Y, Z) to form three-dimensional point cloud data; where (u, v) is the pixel coordinate of the panoramic image, and (X, Y, Z) is the three-dimensional space coordinate; In the 3D point cloud data, the sub-point cloud corresponding to the fused image area is extracted, and the coordinate system of the sub-point cloud is aligned with the coordinate system of the panoramic image through rigid body transformation to achieve spatial registration of the fused image and the 3D point cloud data; According to the registered sub-point cloud, the pixel points in the fused image are mapped to the three-dimensional space to obtain the color point cloud data corresponding to the fused image. Each point contains RGB color information and three-dimensional coordinate information. The mapped color point cloud data is merged with the original three-dimensional point cloud data, and the color information of the corresponding points in the three-dimensional point cloud data is updated to obtain the fused color point cloud data with hidden danger target information.

10. An engineering quality management system based on AI big model, characterized in that: include: At least one processing unit; used to execute instructions to implement the engineering quality management method based on the AI ​​big model as described in any one of claims 1 to 9.

Images