An engineering quality management method and system based on an AI large model

Abstract

The application discloses an AI large model-based engineering quality management method and system, relates to the field of engineering quality management, and comprises the following steps: installing a panoramic device on a tower crane at an engineering site, collecting panoramic images through the panoramic device; performing semantic segmentation on the panoramic images by using a pre-constructed risk control model; checking hidden dangers in each grid area, collecting hidden danger target images of hidden danger targets through the panoramic device; inputting the hidden danger target images into a pre-trained hidden danger classification model to classify the hidden danger target images; collecting local images within a preset range of the hidden danger targets according to hidden danger target type labels and position coordinates; fusing the hidden danger target images and the corresponding local images; mapping the fused images to a BIM model through three-dimensional reconstruction to obtain a digital twin model of the engineering site, and performing engineering quality management by using the digital twin model. The application improves the hidden danger checking efficiency in view of the fact that the hidden danger checking in the prior art is not timely.

Description

TECHNICAL FIELD

[0001] The application relates to the field of engineering quality management, and in particular relates to an engineering quality management method and system based on an AI large model. BACKGROUND

[0002] With the rapid development and continuous expansion of the construction engineering industry, the construction environment of the engineering site is becoming increasingly complex, various safety hazards and quality problems occur frequently, and severe challenges are brought to engineering management. Traditional engineering quality management mainly relies on manual inspection and sampling inspection and other methods to check hidden dangers and monitor quality, which has problems such as low efficiency, poor accuracy, insufficient real-time performance, and is difficult to meet the needs of modern engineering management.

[0003] In recent years, digital technology has been widely applied in the engineering field, providing a new idea for solving the problem of engineering quality management. Among them, the BIM (Building Information Modeling) technology realizes the digital expression and integrated management of three-dimensional geometric information, physical information, functional characteristics and other information of the construction engineering, and provides a data basis and a collaborative platform for the whole life cycle management of the engineering. However, due to the lack of effective on-site data collection and fusion means, the BIM model is difficult to reflect the dynamic changes of the engineering site in real time, and cannot support the rapid discovery and disposal of hidden dangers and quality problems.

[0004] At the same time, the rapid development of artificial intelligence technology provides a new tool for solving the perception and decision-making problems of complex engineering scenes. Computer vision, deep learning and other AI technologies have made significant progress in target detection, semantic segmentation, three-dimensional reconstruction and other aspects, making intelligent perception and information extraction of complex environments possible. Combining artificial intelligence and BIM technology to build a digital twin of the engineering site, realizing real-time perception of the site state, automatic discovery and identification of hidden dangers, intelligent analysis and early warning of quality problems, has become an inevitable way to improve the level of engineering quality management.

[0005] However, existing researches are mainly limited to the application of local problems and single data, and there is still a lack of overall top-level design and systematic technical solutions. Engineering quality management involves panoramic perception, multi-source data fusion, hidden danger identification, positioning guidance, information integration and other links. How to build an overall solution for the whole scene, the whole process and the whole element of the engineering, break through the data barrier, realize seamless connection between links, and form a replicable and popularized intelligent quality management mode is a key problem that needs to be solved. SUMMARY

[0006] To solve the problem of not timely checking hidden dangers in the prior art, the application provides an engineering quality management method and system based on an AI large model, a risk control model is used to perform semantic segmentation on panoramic images to form grid areas, hidden danger target detection and positioning are performed in each grid area, hidden danger target images and their position coordinates are obtained, a hidden danger classification model is used to identify hidden danger types, image information of local environments around hidden danger targets is collected, hidden danger target images and local environment images are fused, finally, three-dimensional reconstruction technology is used to map the fused image information to a BIM model, a digital twin model of an engineering site is constructed, and the like, thereby improving the efficiency of hidden danger checking.

[0007] The purpose of the application is achieved by the following technical solutions.

[0008] One aspect of the application provides an engineering quality management method based on an AI large model, including: S1, installing a panoramic device on a tower crane in an engineering site, and collecting panoramic images through the panoramic device; S2, performing semantic segmentation on the panoramic images by using a pre-constructed risk control model to obtain panoramic images of multiple grid areas; S3, checking hidden dangers in each grid area to obtain position coordinates of hidden danger targets, and collecting hidden danger target images of the hidden danger targets through the panoramic device according to the position coordinates of the hidden danger targets; S4, inputting the hidden danger target images into a pre-trained hidden danger classification model to classify the hidden danger target images, and obtaining hidden danger target type labels and position coordinates; S5, collecting local images within a preset range of the hidden danger targets according to the hidden danger target type labels and the position coordinates; S6, fusing the hidden danger target images and the corresponding local images to obtain fused images containing semantic descriptions and environmental information; S7, mapping the fused images to a BIM model through three-dimensional reconstruction to obtain a digital twin model of the engineering site, and using the digital twin model to manage engineering quality.

[0009] Further, the types of hidden danger checking include fire monitoring, smoke monitoring, safety helmet monitoring, reflective clothing monitoring, and bare soil coverage monitoring.

[0010] Further, S2, using a pre-constructed risk control model to perform semantic segmentation on the panoramic image, obtaining a plurality of grid region panoramic images, comprising: S21, using PanoNet to construct a risk control model; S22, using the risk control model to perform feature extraction and pixel-level classification on the panoramic image, obtaining a semantic segmentation result of the panoramic image; in the feature extraction, the panoramic image is unfolded into a planar image according to the rectangular projection mode, and the planar image is feature-extracted using the SphericalConv convolution kernel; according to the pixel-level classification result, the panoramic image is divided into a foreground region and a background region; S23, according to the semantic segmentation result, the divided foreground region is divided into NxN grids; S24, output the divided grid region to obtain a panoramic image containing a grid image and corresponding position information; the grid image is a local image in each grid region, and the position information includes the coordinate position and size of each grid in the panoramic image.

[0011] Wherein, SphericalConv is a kind of spherical convolution kernel specially used for processing panoramic image. Unlike traditional two-dimensional convolution kernel, SphericalConv performs convolution operation on image in spherical coordinate system, which can effectively capture the spatial features in panoramic image. It unfolds the spherical image into a planar image by projection transformation, and then applies convolution operation in planar coordinate system. This way can well adapt to the geometric characteristics of panoramic image, and extract local and global feature representation of panoramic image.

[0012] Further, S3, according to the position coordinates of the hidden danger target, the hidden danger target image of the hidden danger target is collected by the panoramic device, comprising: S31, for each grid region panoramic image, using the improved target detection algorithm PanoLoc to detect and locate the hidden danger target, obtaining the boundary box coordinates of the hidden danger target; S32, according to the boundary box coordinates of the hidden danger target, using the spherical center projection to calculate the center point coordinates of the hidden danger target in the grid region panoramic image as the relative position coordinates of the hidden danger target in the corresponding grid; S33, mapping the relative position coordinates of the hidden danger target to the coordinate system of the grid region panoramic image, obtaining the absolute position coordinates of the hidden danger target in the panoramic image; S34, according to the absolute position coordinates of the hidden danger target, collecting the local image centered on the hidden danger target as the hidden danger target image through the panoramic device; S35, output the absolute position coordinates of the hidden danger target and the hidden danger target image as the input data of S4 hidden danger classification.

[0013] PanoLoc is a target detection algorithm specially designed for panoramic images. Unlike common target detection algorithms such as Faster R-CNN, YOLO, etc., PanoLoc fully considers the spherical geometric characteristics of panoramic images. It first extracts multi-scale features of panoramic images using SphericalConv, then generates potential target regions through a Region Proposal Network. After that, these regions are classified and position-regressed to obtain the final target bounding box. PanoLoc can accurately predict the position of the target in the spherical coordinate system while detecting.

[0014] Spherical central projection is a method of projecting points in the spherical coordinate system to the planar coordinate system. In panoramic image processing, it is often necessary to convert between spherical and planar coordinate systems. Spherical central projection can map points on the sphere (such as longitude and latitude) to pixel coordinates on a two-dimensional plane. Specifically, it calculates the intersection point of the line connecting the sphere center and the point on the sphere with the projection plane to determine the corresponding position of the sphere point on the plane. This projection method can expand the panoramic image into a rectangular planar image, facilitating subsequent processing and analysis.

[0015] Further, S31, an improved target detection algorithm PanoLoc is used for detection and positioning of hidden targets, obtaining the bounding box coordinates of hidden targets, including: inputting the panoramic image of the grid area into the feature extraction network of the target detection algorithm PanoLoc, extracting multi-scale features through the convolutional neural network to obtain the feature map of the grid image; inputting the feature map of the grid image into the region proposal network RPN of the target detection algorithm PanoLoc to generate candidate regions and calculate the confidence score and preliminary bounding box coordinates corresponding to the candidate regions; inputting the candidate regions and feature map into the detection and positioning branch of the target detection algorithm PanoLoc, classifying and regressing the candidate regions through region of interest ROI pooling and fully connected layer to obtain the class probability of hidden targets and the corrected bounding box coordinates; associating the corrected bounding box coordinates with the position information of the grid image in S24 to obtain the bounding box coordinates of hidden targets in the panoramic image of the grid area.

[0016] Further, S4, classifying the hidden danger target image to obtain a hidden danger target type label and a position coordinate, comprising: S41, collecting panoramic images of the construction site, and labeling the collected panoramic images, positioning and classifying the hidden danger targets in each image, labeling the open fire, smoke, no safety helmet, no reflective vest and bare soil coverage, and generating the corresponding hidden danger type label to obtain a training set; S42, constructing a hidden danger classification model based on the DenseNet network according to the training set, setting the last layer of the hidden danger classification model as a full connection layer, and the number of neurons of the full connection layer is 5; S43, inputting the hidden danger target image and the corresponding absolute position coordinate output by S35 into the hidden danger classification model to perform feature extraction and classification prediction, obtaining a hidden danger target type label representing a hidden danger type and a position coordinate representing a hidden danger position.

[0017] Further, S5, collecting a local image in a hidden danger target preset range according to the hidden danger target type label and the position coordinate, comprising: S51, determining the center position (x c ,y c ) of the hidden danger target in the panoramic image according to the hidden danger target type label and the position coordinate; S52, determining the standard width w and the height h of the corresponding hidden danger target according to the hidden danger target type label; calculating the width w b and the height h b of the rectangular region with the center position (x c ,y c ) as the center; generating an inscribed ellipse on the panoramic image with (x c ,y c ) as the center, w b as the major axis and h b as the minor axis, and taking the internal region of the inscribed ellipse as the hidden danger target preset range; S53, establishing a projection relationship between the panoramic image and the imaging plane of the panoramic device according to the internal parameter matrix K and the external parameter matrix T of the panoramic device; converting each pixel point (x, y) in the inscribed ellipse into the corresponding point (u, v) on the imaging plane of the panoramic device through the projection relationship; determining the minimum circumscribed rectangular region as the local imaging region according to the corresponding point (u, v) on the imaging plane; S54, collecting the image of the local imaging region by using the panoramic device to obtain the local image in the hidden danger target preset range.

[0018] S6, fusing the hidden danger target image and the corresponding local image to obtain a fusion image containing semantic description and environmental information, comprising: preprocessing the hidden danger target image and the local image, including size normalization, brightness adjustment, contrast enhancement and the like, so that the data distribution of the two images is more consistent; inputting the preprocessed hidden danger target image and the local image into a pre-trained image fusion model, extracting multi-scale features of the two images through a feature extraction network to obtain a feature spectrum F_t of the hidden danger target image and a feature spectrum F c .

[0019] generating an attention matrix A with the same size as F t and F c , each element a ij of the matrix representing the relevance of the i-th position in F t and the j-th position in F c ; calculating the weighted sum of F t and F c according to the attention matrix A to obtain a fusion feature spectrum F f , that is inputting the fusion feature spectrum F f into a decoding network to generate a fusion image with the same size as the input image through multi-layer deconvolution and upsampling operations; the fusion image contains not only high-resolution detailed features of the hidden danger target, but also surrounding environmental information of the hidden danger occurrence.

[0020] Further, S7, mapping the fusion image into the BIM model through three-dimensional reconstruction to obtain a digital twin model of the engineering site, comprising: S71, registering the fusion image obtained in S6 with the panoramic image to obtain the mapping position and direction of the fusion image in the panoramic image; S72, mapping the panoramic image to the three-dimensional space according to the position and posture parameters of the panoramic device to obtain three-dimensional point cloud data; mapping the fusion image to the corresponding position in the three-dimensional point cloud data through spatial transformation to obtain a color point cloud with hidden danger target information; filtering and denoising the color point cloud and downsampling to obtain a sparse point cloud; S73, generating a dense point cloud using an incremental reconstruction algorithm PMVS according to the sparse point cloud to obtain a three-dimensional real scene model of the engineering site; S74, registering the three-dimensional real scene model according to the BIM model of the engineering site, registering the point cloud and the BIM model through the ICP algorithm to obtain the mapping relationship between the three-dimensional real scene model and the BIM model; S75, adding the hidden danger target information to the corresponding component of the BIM model according to the mapping relationship between the three-dimensional real scene model and the BIM model to obtain a BIM model integrated with hidden danger information as a digital twin model of the engineering site; S75, obtaining the position of the hidden danger target in the BIM component and the type of the hidden danger target according to the digital twin model to generate a hidden danger rectification task.

[0021] Further, S72, map the panoramic image into the three-dimensional space to obtain three-dimensional point cloud data, comprising: obtaining the position coordinates (x, y, z) and the attitude angle (roll, pitch, yaw) of the panoramic device, constructing the extrinsic parameter matrix T of the panoramic device according to the position coordinates (x, y, z) and the attitude angle (roll, pitch, yaw); according to the intrinsic parameter matrix K of the panoramic device, the mapping relationship between the panoramic image pixel coordinate system and the panoramic device camera coordinate system is established; according to the extrinsic parameter matrix T and the intrinsic parameter matrix K, through the matrix multiplication [X, Y, Z, 1] = K x T x [u, v, 1], each pixel point (u, v) on the panoramic image is projected into the three-dimensional space to obtain the corresponding three-dimensional point coordinates (X, Y, Z), forming the three-dimensional point cloud data; wherein (u, v) is the pixel coordinate of the panoramic image, (X, Y, Z) is the three-dimensional space coordinate; in the three-dimensional point cloud data, the sub-point cloud corresponding to the fusion image region is extracted, the coordinate system of the sub-point cloud is aligned with the coordinate system of the panoramic image through the rigid body transformation, and the spatial registration of the fusion image and the three-dimensional point cloud data is realized; according to the registered sub-point cloud, the pixel points in the fusion image are mapped into the three-dimensional space to obtain the color point cloud data corresponding to the fusion image, each point contains RGB color information and three-dimensional coordinate information; the mapped color point cloud data and the original three-dimensional point cloud data are merged, the color information of the corresponding points in the three-dimensional point cloud data is updated, and the color point cloud data after fusion with hidden danger target information is obtained.

[0022] Another aspect of the present application also provides an AI large model-based engineering quality management system for executing an AI large model-based engineering quality management method of the present application.

[0023] Compared with the prior art, the present application has the following advantages:

[0024] By arranging panoramic image acquisition devices on the construction site, image data of the engineering site can be comprehensively acquired, and the risk control model is used to perform semantic segmentation on the panoramic image to form a grid region, so that the complex panoramic scene is divided into multiple local regions, and the difficulty of subsequent hidden danger target detection is simplified. Among them, the risk control model adopts the PanoNet network structure, introduces the SphericalConv convolution kernel to process the special data format of the panoramic image, extracts the multi-scale features of the panoramic image, and can accurately and efficiently realize the semantic segmentation of the panoramic image.

[0025] The improved target detection algorithm PanoLoc is used for hidden danger target detection and positioning in the grid area. The hidden danger target candidate area is generated through multi-scale feature extraction and region proposal network, and the candidate area is further classified and regressed by combining region interest pooling. The position and boundary box information of the hidden danger target can be accurately detected in the local image of the grid area. At the same time, the spherical central projection algorithm is introduced to map the detected hidden danger target position to the panoramic image coordinate system,

[0026] A special hidden danger classification model is constructed to classify and identify the detected hidden danger target image and determine the specific hidden danger type. The classification model adopts the DenseNet network structure, fully excavates and fuses the multi-scale features of the hidden danger target through the densely connected convolutional layer, and combines the fully connected layer to classify and judge the hidden danger type. At the same time, a large amount of construction site hidden danger data is collected and labeled to construct a special hidden danger classification data set. Through training, a hidden danger classification model with strong robustness and good generalization ability can accurately identify different types of hidden dangers.

[0027] For the local area around the hidden danger target, the panoramic device is used to supplement the collection of environmental information. By fusing the detected hidden danger target image with the local environment image, a fusion image containing more semantic information and scene information can be obtained, which can more comprehensively represent the hidden danger situation. Among them, the method adaptively determines the spatial range of the local area according to the hidden danger type, maps the panoramic image coordinates to the panoramic camera imaging plane coordinates through the projection relationship, and then obtains the accurate local imaging area, which improves the accuracy and controllability of local image acquisition. Image fusion makes full use of the environmental information of panoramic data, providing a more comprehensive data perspective for hidden danger analysis.

[0028] The fusion image is mapped to the BIM model through three-dimensional reconstruction technology to generate a digital twin model of the construction site, realizing the accurate association and visual expression of hidden danger information and BIM components. Through a series of processes such as registration of panoramic images and point cloud data, filtering and reconstruction of point cloud data, and registration of point cloud and BIM model, the hidden danger information in the image space is mapped to the three-dimensional space of the BIM model, so that the hidden danger target forms accurate spatial positioning and semantic association in the BIM component. Integrating hidden danger information into the BIM model forms an information-rich, intuitive and visual digital twin model. BRIEF DESCRIPTION OF DRAWINGS

[0029] The present application will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same reference numbers represent the same structures, wherein:

[0030] Figure 1is an example application scenario diagram of an AI large model-based engineering quality management method according to some embodiments of the present application;

[0031] Figure 2 is an example flowchart of an AI large model-based engineering quality management method according to some embodiments of the present application;

[0032] Figure 3 is an example flowchart of generating a hidden danger target image according to some embodiments of the present application;

[0033] Figure 4 is an example flowchart of generating a digital twin model of an engineering site according to some embodiments of the present application. DETAILED DESCRIPTION

[0034] The method and system provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0035] As shown in Figure 1 and Figure 2 , a panoramic device is installed on a tower crane in an engineering site, and panoramic images are collected by the panoramic device; a pre-constructed risk control model is used to perform semantic segmentation on the panoramic images to obtain panoramic images of multiple grid areas; hidden danger investigation is performed on each grid area to obtain the position coordinates of hidden danger targets; and hidden danger target images of the hidden danger targets are collected by the panoramic device according to the position coordinates of the hidden danger targets; the hidden danger target images are input into a pre-trained hidden danger classification model to classify the hidden danger target images, and hidden danger target type labels and position coordinates are obtained; local images within a preset range of the hidden danger targets are collected according to the hidden danger target type labels and position coordinates; the hidden danger target images and the corresponding local images are fused to obtain fusion images containing semantic descriptions and environmental information; the fusion images are mapped to a pre-established BIM model through three-dimensional reconstruction to obtain a digital twin model of the engineering site, and the digital twin model is used for engineering quality management.

[0036] Specifically, S1, considering that the tower crane can provide a high viewing angle and a wide coverage range, the panoramic device is selected to be installed on the tower crane, and 360-degree panoramic images of the engineering site are obtained by rotating and shooting. The panoramic device uses imaging methods such as multi-lens stitching or fisheye lens, and generates complete panoramic images by stitching images of different angles. In order to obtain high-quality panoramic images, the panoramic device needs to be accurately calibrated for internal and external parameters to ensure the stitching accuracy and color consistency between lenses. At the same time, the shooting parameters of the panoramic device such as exposure time and white balance need to be dynamically adjusted according to the site environment to adapt to different lighting conditions.

[0037] In order to realize large-scale and multi-angle data acquisition, multiple panoramic devices can be arranged at different heights and positions, the spatial coordinates of each device are determined through the positional relationship between the devices, and a panoramic image acquisition array is formed. Each device is connected to the central control unit through wired or wireless mode to realize synchronous acquisition and transmission of data.

[0038] The collected panoramic images need to be preprocessed, including image distortion correction, noise filtering, image enhancement, etc., to improve the image quality and provide more reliable input data for subsequent semantic segmentation and hidden danger detection. At the same time, in order to reduce the transmission and storage overhead, the original image can be compressed and encoded to reduce the data volume under the premise of ensuring the quality.

[0039] S2, a pre-constructed risk control model is used to perform semantic segmentation on the panoramic image, which divides the complex panoramic scene into multiple grid areas, simplifying the difficulty of subsequent hidden danger target detection. In specific implementation, the panoramic image is unfolded into a planar image according to the rectangular projection method, and PanoNet network structure and SphericalConv convolution kernel are used for feature extraction and pixel-level classification to obtain the semantic segmentation result of the panoramic image. Then, according to the segmentation result, the panoramic image is divided into foreground and background areas, and NxN grids are divided in the foreground area, finally outputting structured data containing grid images and corresponding position information.

[0040] S21, PanoNet is used as the basic network architecture of the risk control model. PanoNet is a convolutional neural network specially designed for panoramic images, which can effectively process the special data format of 360-degree panoramic images through convolution operation in the spherical coordinate system. PanoNet network includes multiple convolution layers and pooling layers, which can generate high-level semantic representation by continuously extracting and abstracting image features.

[0041] S22, the constructed PanoNet risk control model is used for end-to-end semantic segmentation of the panoramic image. First, the panoramic image is unfolded into a planar image according to the rectangular projection method, and the 360-degree spherical information is mapped into a 2D planar coordinate system, which is convenient for subsequent convolution operation. In the feature extraction process, the SphericalConv convolution kernel is used instead of the traditional rectangular convolution kernel to extract the features of the planar image. SphericalConv can better adapt to the data characteristics of the panoramic image by defining convolution operation on the spherical coordinate system, and extract the semantic information specific to the panoramic image. Through layer-by-layer convolution and downsampling, a multi-scale feature map of the image is generated. Finally, through upsampling and cascading operation, the feature maps of different scales are fused to generate a pixel-level segmentation result. The segmentation result divides each pixel of the image into different semantic categories, such as ground, building, mechanical equipment, construction materials, etc. According to the segmentation result, the image is divided into foreground area (construction scene of interest) and background area (sky, long-range view, etc. Non-key area).

[0042] S23, according to the foreground area generated by semantic segmentation, further divided into NxN grid. The choice of grid size needs to balance the computational complexity and the granularity of the area, generally according to the resolution of the panoramic image and the demand of target detection to set. Grid processing divides large-scale panoramic image into multiple small-scale local areas, each grid corresponds to a specific area of the construction site. Through gridding, the global detection task is transformed into multiple local detection tasks, reducing the complexity of the problem.

[0043] S24, the divided grid area is output to generate structured data containing grid image and corresponding position information. Among them, the grid image is the local image in each grid area cut from the original panoramic image, which represents a partial view of the construction site. The position information records the spatial position and size of each grid in the panoramic image, which is used for subsequent spatial mapping and three-dimensional reconstruction. Through grid output, the continuous panoramic image is discretized, which is convenient for parallel processing and data organization and management.

[0044] The present application converts complex panoramic images into structured local representations through semantic segmentation and gridding, which lays the foundation for subsequent hidden danger detection. Among them, the use of PanoNet and SphericalConv solves the problem of processing the special format of panoramic image, semantic segmentation realizes the automatic understanding and area division of image content, and gridding simplifies the scale of detection problem.

[0045] As Figure 3As shown, S3, on the basis of the meshed area generated in S2, the detection and positioning of hidden danger targets are performed on each meshed area, the position coordinates of the hidden danger targets are obtained, and the local images of the hidden danger targets are collected through the panoramic device according to the position coordinates, so as to provide input data for subsequent hidden danger classification.

[0046] S31, an improved target detection algorithm PanoLoc is adopted to perform hidden danger target detection and positioning on the panoramic image of each meshed area. Since the panoramic image has a large field of view and resolution, the size and proportion of hidden danger targets in the image differ greatly. In order to adapt to such scale variation, PanoLoc adopts a multi-scale feature extraction strategy. Specifically, the feature extraction network adopts a structure similar to FPN (Feature Pyramid Network), which generates multiple scale feature maps at different levels of the backbone network. Taking ResNet-50 as an example, PanoLoc selects the outputs of Conv2_x, Conv3_x, Conv4_x and Conv5_x four stages as multi-scale features, in which Conv2_x corresponds to a feature map size of 1 / 4 of the input image, Conv3_x is 1 / 8, Conv4_x is 1 / 16, and Conv5_x is 1 / 32. Through multi-scale feature extraction, PanoLoc can capture visual features of hidden danger targets at different receptive fields and abstraction levels, improving the robustness of detection.

[0047] In order to efficiently screen out candidate regions that may contain hidden danger targets from the image, PanoLoc adopts a region proposal network RPN. RPN is a two-stage target detection method based on sliding window, which generates candidate region proposals by densely sampling anchor boxes (AnchorBox) on the feature map with fixed step length, and performing binary classification (whether containing target) and boundary box regression on each anchor box. In PanoLoc, considering the perspective distortion characteristics of panoramic images, the design of anchor boxes is optimized, and SphericalAnchor defined in the spherical coordinate system is adopted. SphericalAnchor is generated by uniform sampling on the sphere, which can better adapt to the spatial distribution characteristics of panoramic data. At the same time, the sliding window step length and Anchor size of RPN are also adaptively adjusted according to the resolution and target size of the panoramic image, to balance the detection accuracy and efficiency.

[0048] For the RPN-generated candidate regions, due to their variable locations and sizes, they cannot be directly fed into the subsequent classification and regression branches. To this end, PanoLoc introduces a region of interest (ROI) pooling layer to map the candidate regions of different sizes into fixed-size feature representations. Unlike the traditional ROIPooling, PanoLoc adopts the ROIAlign operation to achieve more fine-grained feature alignment through bilinear interpolation, reducing feature distortion and information loss. In the implementation, PanoLoc samples a 7x7 grid of points on the feature map according to the bounding box coordinates of the candidate region, and calculates the feature values of each grid point through bilinear interpolation to obtain a fixed-size 7x7x256 ROI feature map as the input for classification and regression.

[0049] After obtaining the ROI feature map of the candidate region, PanoLoc classifies and regresses the candidate region through two parallel fully connected layer branches. The classification branch predicts the probability of the candidate region belonging to each class of hidden hazard target through the Softmax layer, and the regression branch corrects the location and size of the candidate region through the bounding box regressor, so that it can more accurately frame the hidden hazard target. In the training process, PanoLoc adopts the weighted combination of classification loss (such as cross-entropy loss) and regression loss (such as SmoothL1 loss) as the optimization target, and learns the network parameters through back propagation and gradient descent algorithm. In the inference stage, through setting the probability threshold and non-maximum suppression (NMS) post-processing operation, the final hidden hazard target detection result is obtained.

[0050] The bounding box coordinates of the hidden hazard target detected by PanoLoc are associated with the grid image position information of S24, and through coordinate conversion and mapping, the accurate coordinates of the hidden hazard target in the grid region panoramic image are obtained. This process realizes the spatial information association from local to global.

[0051] S32, in order to determine the relative position of the hidden hazard target in the grid, PanoLoc adopts the method of spherical center projection to convert the detected hidden hazard target bounding box from the plane coordinate system to the spherical coordinate system, and calculates the spherical center point coordinates as the relative position coordinates. Specifically, assuming that the four vertex coordinates of the hidden hazard target bounding box in the plane rectangular coordinate system (x, y) are (x1, y1), (x2, y2), (x3, y3), (x4, y4), where x represents the width direction of the image and y represents the height direction. PanoLoc first converts the vertex coordinates in the plane coordinate system to the plane coordinate system (x', y') with the center of the panoramic image as the origin according to the aspect ratio (such as 2:1) and the picture angle (such as 360°x180°) of the panoramic image, and then converts the vertex coordinates in the plane coordinate system to the spherical coordinate system (θ, φ) according to the spherical coordinate system correspondence between the planar coordinate system (x', y') and the spherical coordinate system, the coordinates of the four vertices on the sphere are calculated where θ represents the azimuth angle and φ represents the zenith angle, and the conversion formula is: where r represents the spherical radius, usually set to 1. According to the spherical coordinates of the four vertices, PanoLoc calculates the arithmetic mean The spherical center point coordinates of the hidden target, i.e.: Through the above calculation, PanoLoc obtains the relative position coordinates of the hidden target in the spherical coordinate system represent the azimuth angle and zenith angle offset of the hidden target relative to the center of the grid region. The relative position coordinates establish a local spatial correlation with the grid region, providing convenience for subsequent spatial information fusion.

[0052] S33, PanoLoc further maps the relative position coordinates of the hidden target to the global coordinate system of the panoramic image of the grid region, obtaining the absolute position coordinates of the hidden target. Specifically, assuming that the spherical coordinate range of the grid region in the panoramic image is By combining the relative position coordinates with the boundary coordinates of the grid region, the absolute spherical coordinates of the hidden target are obtained

[0053] where 0 < θ ≤ 2π and represent the value range of the azimuth angle and the zenith angle respectively. Through the above coordinate mapping, PanoLoc expands the position information of the hidden target from the grid local space to the global space of the panoramic image, establishing a spatial correlation between the hidden target and the entire image. The absolute position coordinates not only represent the precise position of the hidden target in the panoramic image, but also provide a spatial reference for subsequent three-dimensional reconstruction and visualization management.

[0054] S34, PanoLoc obtains the local image centered on the hidden target as the hidden target image according to the absolute position coordinates of the hidden target through directional acquisition of the panoramic device. Specifically, PanoLoc first converts the absolute spherical coordinates of the hidden target into a three-dimensional direction vector (x, y, z) in the camera coordinate system according to the internal and external parameters of the panoramic camera, and then controls the rotation angle (p a ,t a ) and zoom factor (z) of the pan-tilt head of the panoramic camera to align the camera field of view with the hidden target, where: p a = θ abs , wherein f represents the camera focal length, and fov represents the camera field of view. By setting appropriate shooting parameters, such as a field of view fov = 60° and a focal length f = 800 pixels, the PanoLoc controls the panoramic camera to capture a local image centered on the hidden danger target, obtaining a high-resolution image focused on the hidden danger target as the hidden danger target image. Compared with the panoramic image of the grid area, the hidden danger target image contains more abundant and detailed hidden danger feature information, such as the shape, texture, color, etc. of the hidden danger target, which provides targeted discrimination basis for subsequent hidden danger classification.

[0055] Finally, in S35, the absolute position coordinates of the hidden danger target and the collected hidden danger target image are packaged and output as input data for the S4 hidden danger classification step. The position coordinates are used to identify the spatial position information of the hidden danger target, and the hidden danger target image provides the visual feature information of the hidden danger target, which provides necessary prior knowledge and discrimination basis for hidden danger classification.

[0056] S4, the hidden danger target image is input into a pre-trained hidden danger classification model to classify the hidden danger target image, obtaining a hidden danger target type label and position coordinates, including: S41 in order to construct the hidden danger classification model, a certain amount of construction site panoramic images need to be collected as training data. In the data labeling stage, through manual inspection, the hidden danger targets appearing in each panoramic image are positioned and classified, and common hidden danger types such as open fire, smoke, no safety helmet, no reflective vest, and bare soil covering are labeled, and the corresponding hidden danger type label is generated. Taking 100 panoramic images of a construction site as an example, the position coordinates of the hidden danger target and the hidden danger type label are obtained through labeling, forming a training set containing 2000 hidden danger target samples. Among them, 500 cases of open fire hidden danger, 200 cases of smoke hidden danger, 800 cases of no safety helmet hidden danger, 300 cases of no reflective vest hidden danger, and 200 cases of bare soil covering hidden danger. The training set obtained by manual labeling provides high-quality supervision information for subsequent model training.

[0057] S42, based on the collected training set data, a hidden danger classification model based on DenseNet network is constructed. DenseNet is a kind of dense connection convolutional neural network structure, which realizes the reuse and fusion of features by introducing dense connection between different layers of the network, improves the expression ability and generalization performance of the model. In the hidden danger classification task, the last layer of DenseNet is set to a fully connected layer containing 5 neurons, each neuron corresponds to a hidden danger type, and the output of the neuron is converted into the probability distribution of the hidden danger type through the Softmax function. The input of the model is a 256x256 hidden danger target image, and the output is a 5-dimensional hidden danger type probability vector. In the training process, the cross entropy loss function and Adam optimizer are used, and the model parameters are learned through iterative optimization, so that the hidden danger target image can be accurately classified into the corresponding hidden danger type.

[0058] S43, the hidden danger target image and the corresponding absolute position coordinates output by S35 are input into the pre-trained hidden danger classification model, and the image features are extracted and classified by forward propagation. Specifically, the hidden danger classification model first extracts the features of the input 256x256 hidden danger target image, and obtains a high-dimensional feature representation vector through the multi-layer convolution and pooling operation of the DenseNet network. Then, the feature representation vector is input into the fully connected layer, and a 5-dimensional hidden danger type probability distribution vector is obtained through linear transformation and Softmax normalization, each element representing the confidence of the hidden danger target image belonging to a certain hidden danger type. By setting a confidence threshold (such as 0.5), the hidden danger type with the highest confidence and exceeding the threshold is taken as the final classification result, and the hidden danger target type label representing the hidden danger type is output. At the same time, the hidden danger classification model also directly outputs the hidden danger target absolute position coordinates obtained in S35, representing the spatial position of the hidden danger in the panoramic image.

[0059] S5, according to the type label and position coordinates of the hidden danger target, the preset range of the hidden danger target in the panoramic image is adaptively determined, and the local image in the range is collected to obtain the targeted hidden danger target region data, including: in S51, the center position (x c ,y c ) of the hidden danger target in the panoramic image is determined by using the hidden danger target type label and position coordinates output by S4. Specifically, the position coordinates are usually represented as the top-left and bottom-right coordinates of the hidden danger target bounding box (x min ,y min ,x max ,y max ), and the center point coordinates of the bounding box are calculated to obtain the center position of the hidden danger target in the panoramic image:

[0060] In S52, according to the type label of the hidden danger target, the standard width w and height h of the corresponding hidden danger target are determined. Different types of hidden danger targets present different sizes and appearance characteristics in the image. By statistical analysis of the geometric dimensions of a large number of hidden danger samples, the standard width-height ratio of each type of hidden danger is obtained. For example, the standard width-height ratio of the worker without safety helmet is 1:1.2, and the standard width-height ratio of the open fire is 1.5:1. Then, taking the center position (x c ,y c ) of the hidden danger target as the center, the width w b and height h b of the circumscribed rectangular region are calculated according to the standard width-height ratio: w b =k×w, h b =k×h, where k is a scale factor used to control the magnification of the circumscribed rectangular region relative to the hidden danger target, and is usually taken as 1.2-1.5. By reasonably setting the scale factor, the range of the region of interest can be appropriately expanded while covering the hidden danger target.

[0061] In order to better adapt to the irregular shape of the hidden danger target, S52 further generates a circumscribed ellipse on the panoramic image with (x c ,y c ) as the center, w b as the long axis and h b as the short axis, and takes the internal region of the ellipse as the preset range of the hidden danger target. Compared with the rectangular region, the elliptical region can more compactly cover the hidden danger target and reduce the interference of irrelevant background. The equation of the ellipse is:

[0062] By judging whether each pixel point (x, y) on the image satisfies the ellipse equation, the pixel point set belonging to the preset range of the hidden danger target is obtained.

[0063] In S53, in order to map the pixel points in the preset range of the hidden danger target to the imaging plane of the panoramic device, the projection relationship between the panoramic image and the imaging plane needs to be established. This process involves the internal parameter matrix K and the external parameter matrix T of the panoramic device. The internal parameter matrix K describes the internal imaging geometric characteristics of the panoramic device, including focal length, optical center and other parameters; the external parameter matrix T describes the pose transformation relationship of the panoramic device relative to the global coordinate system, including the rotation matrix and the translation vector. Through the joint action of the internal and external parameters, the conversion between the two-dimensional image coordinate system and the three-dimensional imaging plane coordinate system is realized.

[0064] For each pixel point (x, y) in the interior region of the ellipse, it is converted into the corresponding point (u, v) on the imaging plane through the internal parameter matrix K and the external parameter matrix T: [u, v, 1]' = K x T x [x, y, 1]', wherein [x, y, 1]' represents the homogeneous coordinates of the pixel point, and [u, v, 1]' represents the homogeneous coordinates of the corresponding point on the imaging plane. f x ,f y f represents the focal length, c x ,c y f represents the optical center coordinates, R represents the rotation matrix, and t represents the translation vector; by performing coordinate conversion on all pixel points in the ellipse, the preset range of the hidden danger target on the imaging plane is obtained. In order to facilitate the collection of local images, the minimum circumscribed rectangle of the preset range is further calculated as the local imaging area on the imaging plane.

[0065] In S54, the image of the local imaging area is collected by using the panoramic device, and the high-resolution local image in the preset range of the hidden danger target is obtained. Specifically, by controlling the rotation of the holder of the panoramic camera and the zoom parameter, the field of view of the camera is aligned with the local imaging area, and the focal length and resolution of the camera are dynamically adjusted according to the size of the area, so that clear and complete local close-up images of the hidden danger target are obtained. The resolution of the local image is usually higher than that of the panoramic image, which can provide more detailed and comprehensive feature information of the hidden danger target such as texture and shape, and provide more abundant basis for hidden danger analysis.

[0066] S6, the hidden danger target image and the corresponding local image are fused to obtain a fusion image containing semantic description and environmental information, including: in the data preprocessing stage, the size normalization, brightness adjustment and contrast enhancement of the hidden danger target image and the local image are performed, so that the data distribution of the two images is more consistent, and the difference between the images is reduced. Specifically, first, the two images are scaled to a uniform size (such as 256x256), and then the brightness distribution of the image is adjusted through the histogram equalization algorithm to make it more uniform in the gray level. At the same time, by stretching the pixel value range of the image, the contrast of the image is enhanced, and the texture details of the hidden danger target and the scene are highlighted. The preprocessed image pair is more suitable for feature extraction and fusion operation.

[0067] In the feature extraction stage, the preprocessed hazard target image and local image are input into the pre-trained image fusion model, and the multi-scale features of the two images are extracted through the feature extraction network. The image fusion model adopts an encoder-decoder architecture, and the encoder part is composed of multiple convolutional layers and pooling layers, which is responsible for extracting hierarchical features of the image. By setting different receptive fields and downsampling rates, the encoder generates multi-scale feature maps containing global information and local details of the image. Taking ResNet-50 as an example, feature maps of different sizes such as 64x64, 32x32, 16x16, and 8x8 are extracted, which represent the semantic information of the image at different abstraction levels. The highest layer feature maps of the hazard target image and the local image are denoted as F_t and F_c, which are used as input for the subsequent fusion operation.

[0068] In the feature fusion stage, an attention mechanism is introduced to adaptively adjust the weights of the two feature maps by calculating the relevance between F t and F c . First, an attention matrix A with the same size as F t and F c is generated, where each element a ij represents the relevance between the i-th position in F t and the j-th position in F c . The calculation of relevance is based on the similarity measure of feature maps, and common measure functions include dot product, cosine similarity, etc. Intuitively, each row of the attention matrix A represents the relevance of one position in F t to all positions in F c , and each column represents the relevance of one position in F c to all positions in F t . The rows and columns are normalized by the Softmax function to obtain the final value of the attention matrix A.

[0069] According to the attention matrix A, the weighted sum of F t and F c is calculated to obtain the fused feature map F f :

[0070] where, represents the weighted sum of each position in F t and F c , and the weight is determined by the attention matrix A; represents the weighted sum of each position in F c and F t , and the weight is 1-A. Through the control of the attention matrix, the fused feature map F f semantically realizes the fusion of F t and Fc Adaptive fusion, i.e., when F t and F c When the correlation at a certain location is high, the fusion result is more biased towards the feature value at that location; when the correlation is low, the fusion result considers the contributions of the two feature maps more evenly.

[0071] In the image reconstruction stage, the fused feature map F_f is input into the decoding network. Through multi-layer deconvolution and upsampling operations, the spatial resolution of the image is gradually restored, ultimately generating a fused image with the same size as the input image. The structure of the decoding network is the opposite of that of the encoder. By using skip connections, the multi-scale feature maps of the encoder are concatenated with the corresponding layers of the decoder, achieving cross-layer feature transfer and refinement. During the generation process, the fused image not only inherits the high-resolution detail features of the potential hazard image but also incorporates the surrounding environmental information provided by the local image, forming a comprehensive image representation that includes the potential hazard and the scene context.

[0072] This application introduces an attention mechanism to achieve adaptive fusion of hazard target features and scene information. Traditional image fusion methods, such as weighted averaging and pyramid fusion, typically employ fixed fusion rules, making it difficult to consider the semantic relevance of image content. The S6 scheme, however, dynamically adjusts the fusion weights through an attention matrix, enabling the fusion result to adaptively highlight features of relevant regions and suppress interference from irrelevant regions, generating a more semantically meaningful and information-rich fused image.

[0073] like Figure 4 As shown, S7 involves mapping the fused image to the BIM model through 3D reconstruction to obtain a digital twin model of the engineering site. This digital twin model is then used for engineering quality management. This includes: S71, registering the fused image generated in S6 with the original panoramic image to determine the mapping position and orientation of the fused image within the panoramic image. Specifically, a pair of corresponding feature points is extracted from the two images using a feature point matching algorithm (such as SIFT, SURF, etc.), and then the homography transformation matrix H between the two images is estimated using the RANSAC algorithm. The homography transformation matrix H describes the projection relationship from the fused image to the panoramic image. Matrix multiplication maps the pixel coordinates in the fused image to their corresponding positions in the panoramic image. Simultaneously, based on the decomposition of the homography transformation matrix H, the rotation angle and scale factor of the fused image relative to the panoramic image can be obtained, determining the orientation and size of the fused image within the panoramic image. The image registration results provide coordinate references for subsequent 3D mapping.

[0074] In S72, according to the position and attitude parameters of the panoramic device, the panoramic image is mapped into the three-dimensional space by using the multi-view geometry principle to obtain the three-dimensional point cloud data. First, the position coordinates (x, y, z) and the attitude angle (roll, pitch, yaw) of the panoramic device are obtained, and the extrinsic parameter matrix T of the panoramic device is constructed according to these parameters. The extrinsic parameter matrix T describes the rigid transformation relationship between the panoramic device camera coordinate system and the world coordinate system, including the rotation matrix R and the translation vector t. Then, according to the intrinsic parameter matrix K of the panoramic device, the mapping relationship between the panoramic image pixel coordinate system and the panoramic device camera coordinate system is established. The intrinsic parameter matrix K contains the focal length, optical center, distortion coefficient and other parameters of the camera, and describes the projection process of the three-dimensional space point on the camera imaging plane.

[0075] Through the joint action of the intrinsic and extrinsic parameters, each pixel point (u, v) on the panoramic image is projected into the three-dimensional space to obtain the corresponding three-dimensional point coordinates (X, Y, Z). The projection process is realized by matrix multiplication: [X, Y, Z, 1] = K x T x [u, v, 1], wherein [u, v, 1] represents the homogeneous coordinates of the pixel point, and [X, Y, Z, 1] represents the homogeneous coordinates of the three-dimensional space point. By projecting all the pixel points of the panoramic image, a dense three-dimensional point cloud data is obtained, each point containing three-dimensional coordinates and color information.

[0076] In order to integrate the hidden target information in the fusion image into the three-dimensional point cloud data, spatial transformation and mapping are needed. First, the sub-point cloud corresponding to the fusion image region is extracted from the three-dimensional point cloud data, and the coordinate system of the sub-point cloud is aligned with the coordinate system of the fusion image through rigid transformation (translation + rotation) to realize the spatial registration of the two. The registration process can be completed by manual calibration or automatic optimization. Then, according to the registered sub-point cloud, each pixel point in the fusion image is mapped into the three-dimensional space to obtain the color point cloud data corresponding to the fusion image. The mapping process is similar to the projection process of the panoramic image, which converts the pixel coordinates into three-dimensional coordinates by matrix multiplication, and assigns the pixel color to the corresponding three-dimensional point. Finally, the mapped color point cloud data is merged with the original three-dimensional point cloud data to update the color information of the corresponding points in the three-dimensional point cloud data, and the color point cloud data after fusion with hidden target information is obtained. In order to improve the efficiency and robustness of subsequent processing, filtering and denoising and downsampling operations are performed on the color point cloud data. Statistical filtering methods (such as voxel filtering, mean filtering, etc.) are used to remove outliers and noise in the point cloud, and voxel downsampling or uniform downsampling is used to reduce the data amount of the point cloud to obtain a sparse point cloud.

[0077] In S73, according to the sparse point cloud, a dense point cloud is generated by using the incremental reconstruction algorithm PMVS (Patch-based Multi-View Stereo), and a three-dimensional real scene model of the construction site is obtained. PMVS extracts features, matches feature points and performs triangulation in multiple views of images, and gradually incrementally recovers the three-dimensional structure of the scene. The algorithm first extracts a set of initial seed points in the sparse point cloud, and then performs local matching in the neighborhood of the seed points by using photometric consistency constraints and geometric constraints to obtain a set of locally reconstructed three-dimensional patches. By iteratively expanding and optimizing the three-dimensional patches, the density and coverage of the point cloud are continuously increased, and finally a dense three-dimensional point cloud model is obtained. The dense point cloud generated by PMVS not only contains the geometric structure of the scene, but also contains the texture details of the object surface, and is a high-precision and high-fidelity three-dimensional reconstruction method.

[0078] In S74, in order to associate the three-dimensional real scene model with the BIM model, spatial registration is needed for both. First, according to the BIM model of the construction site, the three-dimensional geometric information of the building components is extracted to obtain the point cloud representation of the BIM model. Then, the ICP (Iterative Closest Point) algorithm is used for point cloud registration of the three-dimensional real scene model and the BIM model. The ICP algorithm finds the optimal rigid transformation (rotation + translation) between the two point clouds through iterative optimization, so that the distance error between the registered point clouds is minimized. The registration process takes into account the geometric structure and semantic attributes of the point cloud, such as matching by extracting building edges, corner points and other features, or using component type information for semantic guided registration. Through ICP registration, the coordinate transformation relationship between the three-dimensional real scene model and the BIM model is obtained, and the geometric alignment of the two models is realized.

[0079] In S75, according to the mapping relationship between the three-dimensional real scene model and the BIM model, the hidden danger target information is added to the corresponding component of the BIM model, and a BIM model integrated with hidden danger information is obtained as the digital twin model of the construction site. Specifically, by performing a spatial intersection operation between the hidden danger target point cloud in the three-dimensional real scene model and the BIM model, the BIM component where the hidden danger target is located is determined, and the type, location, size and other attribute information of the hidden danger target are added to the parameter table of the component. At the same time, a three-dimensional model of the hidden danger target is generated at the corresponding position of the BIM model, and the fusion image is used as the texture map of the model to realize the visual display of the hidden danger target. In this way, the image space, point cloud space and BIM semantic space are associated and fused to form a multi-dimensional and multi-scale digital twin model of the construction site. The digital twin model not only reflects the actual progress and quality status of the project, but also integrates multi-source and heterogeneous data such as images, point clouds and BIM, providing a global and semantic digital means for project management.

[0080] In S76, based on the digital twin model, the spatial position of the hidden danger target in the BIM component and the type attribute of the hidden danger target are further obtained, and a hidden danger rectification task is generated to assist engineering quality management. Specifically, by positioning and querying the hidden danger target in the digital twin model, the component name, floor, spatial area, and other position information of the hidden danger occurrence, as well as the type, level, description, and other attribute information of the hidden danger are automatically extracted to form a structured hidden danger problem list. Then, according to the pre-set hidden danger processing rules and processes, a hidden danger rectification task sheet is automatically generated, including hidden danger description, responsible person, rectification measures, deadline requirements, and other contents, and is sent to the relevant responsible person through a mobile application or a workflow system. In the rectification process, the digital twin model is used to track and record the hidden danger rectification status to form a hidden danger closed-loop management. At the same time, through the visual interface of the digital twin model, the distribution and rectification progress of the hidden danger in the three-dimensional space are intuitively displayed to provide decision support for project managers.

[0081] The present application deeply integrates image semantic information and three-dimensional geometric information, and constructs an engineering site digital twin model integrating multi-source data. Traditional engineering quality management often relies on manual inspection and paper records, and there are problems of information silos and low efficiency. The S7 scheme realizes the mapping and transformation of hidden danger targets from image space to three-dimensional space and then to semantic space through the association and integration of images, point clouds, and BIM models, and corresponds the detected hidden danger problems with the actual scene and digital model of the project, making hidden danger management more intuitive, accurate, and comprehensive. The hidden danger positioning, tracing, rectification, and other management processes based on the digital twin model improve the efficiency of hidden danger discovery and handling.

Claims

1. A method for AI large model-based engineering quality management, characterized by, The method comprises the following steps: S1, installing a panoramic device on a tower crane at a construction site, and collecting panoramic images through the panoramic device; S2, performing semantic segmentation on the panoramic images by using a pre-constructed risk control model to obtain panoramic images of a plurality of grid regions; S3, performing hidden danger investigation on each grid region to obtain position coordinates of a hidden danger target; and collecting hidden danger target images of the hidden danger target through the panoramic device according to the position coordinates of the hidden danger target; S4, inputting the hidden danger target images into a pre-trained hidden danger classification model to classify the hidden danger target images, and obtaining a hidden danger target type label and position coordinates; S5, collecting local images within a preset range of the hidden danger target according to the hidden danger target type label and the position coordinates; S6, fusing the hidden danger target images and the corresponding local images to obtain fused images containing semantic descriptions and environmental information; S7, mapping the fused images to a pre-established BIM model through three-dimensional reconstruction to obtain a digital twin model of the construction site, and performing engineering quality management by using the digital twin model; S2, performing semantic segmentation on the panoramic images by using a pre-constructed risk control model to obtain panoramic images of a plurality of grid regions, comprising: S21, constructing a risk control model by using PanoNet; S22, performing feature extraction and pixel-level classification on the panoramic images by using the risk control model to obtain semantic segmentation results of the panoramic images; In the feature extraction, the panoramic images are unfolded into planar images according to a rectangular projection mode, and the planar images are subjected to feature extraction by using a SphericalConv convolution kernel to obtain pixel-level classification results; According to the pixel-level classification results, the panoramic images are divided into foreground regions and background regions to obtain the semantic segmentation results; S23, dividing the divided foreground regions into NxN grids according to the semantic segmentation results; S24, outputting the divided grid regions to obtain panoramic images of the grid regions containing grid images and corresponding position information; the panoramic images of the grid regions are local images in each grid region, and the position information contains coordinate positions and sizes of each grid in the panoramic images.

2. The engineering quality management method based on an AI large model according to claim 1, wherein: the types of hidden danger investigation include fire monitoring, smoke monitoring, safety helmet monitoring, reflective clothing monitoring, and bare soil coverage monitoring.

3. The engineering quality management method based on an AI large model according to claim 1, wherein: S3, collecting hidden danger target images of the hidden danger target through the panoramic device according to the position coordinates of the hidden danger target, comprising: S31, performing hidden danger target detection and positioning on the panoramic images of each grid region by using an improved target detection algorithm PanoLoc to obtain boundary box coordinates of the hidden danger target; performing hidden danger target detection and positioning by using the improved target detection algorithm PanoLoc to obtain boundary box coordinates of the hidden danger target, comprising: The panoramic image of the grid region is input into a feature extraction network of a target detection algorithm PanoLoc, multi-scale features are extracted through a convolutional neural network, and a feature map of the grid image is obtained; the feature extraction network adopts an FPN structure to generate feature maps of multiple scales on different levels of the backbone network; The feature map of the grid image is input into a region proposal network RPN of the target detection algorithm PanoLoc, candidate regions are generated, and confidence scores and preliminary bounding box coordinates corresponding to the candidate regions are calculated; the RPN adopts SphericalAnchor defined in a spherical coordinate system to generate anchor boxes through uniform sampling on the sphere, so as to adapt to the spatial distribution characteristics of the panoramic image; The candidate regions and the feature map are input into a detection and positioning branch of the target detection algorithm PanoLoc, the candidate regions are classified and regressed through region of interest ROIAlign pooling and a full connection layer, and class probability of the hidden danger target and corrected bounding box coordinates are obtained; the ROIAlign realizes feature alignment through bilinear interpolation, and reduces feature distortion caused by distortion of the panoramic image; The corrected bounding box coordinates are associated with the position information of the grid image in S24, and bounding box coordinates of the hidden danger target in the panoramic image of the grid region are obtained; S32, according to the bounding box coordinates of the hidden danger target, the center point coordinates of the hidden danger target in the panoramic image of the grid region are calculated by using the spherical center projection, as the relative position coordinates of the hidden danger target in the corresponding grid; S33, the relative position coordinates of the hidden danger target are mapped to the coordinate system of the panoramic image of the grid region, and absolute position coordinates of the hidden danger target in the panoramic image are obtained; S34, according to the absolute position coordinates of the hidden danger target, a local image centered on the hidden danger target is collected through the panoramic device, as a hidden danger target image; S35, the absolute position coordinates of the hidden danger target and the hidden danger target image are output as input data of S4 hidden danger classification.

4. The AI large model-based engineering quality management method according to claim 1, characterized in that: S4, classifying the hidden danger target image to obtain hidden danger target type labels and position coordinates, comprising: S41, collecting panoramic images of the construction site, and labeling the collected panoramic images, positioning and classifying hidden danger targets in each image, labeling open flames, smoke, no safety helmet, no reflective vest and bare soil coverage, and generating corresponding hidden danger type labels to obtain a training set; S42, constructing a hidden danger classification model based on a DenseNet network according to the training set, setting the last layer of the hidden danger classification model as a full connection layer, and the number of neurons of the full connection layer is 5; S43, inputting the hidden danger target image and the corresponding absolute position coordinates output by S35 into the hidden danger classification model for feature extraction and classification prediction to obtain hidden danger target type labels representing hidden danger types and position coordinates representing hidden danger positions.

5. The AI large model-based engineering quality management method according to claim 4, characterized in that: S5, according to the hidden danger target type label and the position coordinates, collect the local image in the preset range of the hidden danger target, comprising: S51, according to the hidden danger target type label and the position coordinates, determine the center position (x_c, y_c) of the hidden danger target in the panoramic image; S52, according to the hidden danger target type label, determine the standard width w and height h of the corresponding hidden danger target; Computing with central position Rectangular region width centered And height ; With as the center, as the long axis, as the short axis, an outer-ellipse is generated on the panoramic image, and the inner region of the outer-ellipse is taken as the hidden danger target preset range; S53, according to the internal parameter matrix K and the external parameter matrix T of the panoramic device, establish the projection relationship between the panoramic image and the imaging plane of the panoramic device; Convert each pixel point (x, y) in the circumscribed ellipse into the corresponding point (u, v) on the imaging plane of the panoramic device through the projection relationship; According to the corresponding point (u, v) on the imaging plane, determine the minimum circumscribed rectangle region as the local imaging region; S54, use the panoramic device to collect the image of the local imaging region, and obtain the local image in the preset range of the hidden danger target.

6. The AI large model-based engineering quality management method according to claim 5, characterized in that: S7, map the fusion image into the BIM model through three-dimensional reconstruction to obtain a digital twin model of the engineering site, comprising: S71, register the fusion image obtained in S6 with the panoramic image to obtain the mapping position and direction of the fusion image in the panoramic image; S72, according to the position and posture parameters of the panoramic device, and the mapping position and direction, map the panoramic image into the three-dimensional space to obtain three-dimensional point cloud data; Map the fusion image into the corresponding position in the three-dimensional point cloud data through spatial transformation to obtain a color point cloud with hidden danger target information; Filter and downsample the color point cloud to obtain a sparse point cloud; S73, according to the sparse point cloud, generate a dense point cloud using the incremental reconstruction algorithm PMVS as a three-dimensional real scene model of the engineering site; S74, according to the BIM model of the engineering site, register the three-dimensional real scene model, and register the point cloud and the BIM model through the ICP algorithm to obtain the mapping relationship between the three-dimensional real scene model and the BIM model; S75, according to the mapping relationship between the three-dimensional real scene model and the BIM model, add the hidden danger target information to the corresponding component of the BIM model to obtain a BIM model integrated with hidden danger information as a digital twin model of the engineering site; S75, according to the digital twin model, obtain the position of the hidden danger target in the BIM component and the type of the hidden danger target, and generate a hidden danger rectification task.

7. The AI large model-based engineering quality management method according to claim 6, characterized in that: S72, map the panoramic image into the three-dimensional space to obtain three-dimensional point cloud data, comprising: Obtain the position coordinates (x, y, z) and attitude angle (roll, pitch, yaw) of the panoramic device, and construct the external parameter matrix T of the panoramic device according to the position coordinates (x, y, z) and the attitude angle (roll, pitch, yaw); According to the internal parameter matrix K of the panoramic device, establish the mapping relationship between the panoramic image pixel coordinate system and the panoramic device camera coordinate system; According to the external parameter matrix T and the internal parameter matrix K, each pixel point (u, v) on the panoramic image is projected into a three-dimensional space through matrix multiplication to obtain corresponding three-dimensional point coordinates (X, Y, Z) to form three-dimensional point cloud data; wherein (u, v) is the pixel coordinates of the panoramic image, and (X, Y, Z) is the three-dimensional space coordinates. In the three-dimensional point cloud data, a sub-point cloud corresponding to a fusion image region is extracted, a coordinate system of the sub-point cloud is aligned with a coordinate system of the panoramic image through a rigid body transformation, and spatial registration of the fusion image and the three-dimensional point cloud data is realized; According to the registered sub-point cloud, a pixel point in the fusion image is mapped to a three-dimensional space to obtain color point cloud data corresponding to the fusion image, each point containing RGB color information and three-dimensional coordinate information; The mapped color point cloud data and the original three-dimensional point cloud data are merged, the color information of the corresponding points in the three-dimensional point cloud data is updated, and the color point cloud data with hidden danger target information after fusion is obtained.

8. An AI large model-based engineering quality management system, characterized in that it comprises: at least one processing unit for executing instructions to implement the AI large model-based engineering quality management method of any one of claims 1 to 7. ​

Images