A building appearance quality review system based on BIM and drone images

Abstract

The present invention belongs to the technical field of quality assessment. The present invention discloses a building appearance quality audit system based on BIM and drone images, comprising: constructing a BIM building model, and based on the BIM building model, performing image acquisition on the building appearance to be audited to obtain a corresponding appearance image sequence; performing image processing on the obtained appearance image sequence to obtain a corresponding first image sequence, and constructing a corresponding appearance panoramic image based on the first image sequence; based on the obtained appearance panoramic image and in combination with the constructed BIM building model, performing a quality audit on the target building appearance to obtain a corresponding audit result; the present invention has multiple beneficial effects such as high efficiency and accuracy, real-time dynamics, cost reduction, quality and safety improvement, and is of great significance for improving the efficiency and accuracy of building appearance quality audits.

Description

Technical Field

[0001] The present invention relates to the technical field of quality assessment, and more specifically, to a building appearance quality review system based on BIM and drone images. Background Art

[0002] In the construction industry, building appearance quality review is a key link to ensure the overall quality of the building. As the scale of buildings continues to expand and the building forms become increasingly complex, the traditional manual review method can no longer meet the requirements of efficiency and accuracy;

[0003] Traditional methods mainly rely on manual on-site inspections, which have many limitations. On the one hand, manual inspections are inefficient, especially for large buildings or complex building exteriors. They require a lot of manpower and time costs, and are limited by human visual range and fatigue, which may lead to the omission of some details.

[0004] On the other hand, manual judgment is highly subjective, and different auditors may have different assessments of the same defect, resulting in a lack of consistency and accuracy in the audit results. In addition, traditional methods make it difficult to comprehensively and systematically record and analyze the appearance of buildings, which is not conducive to subsequent quality traceability and management.

[0005] In view of this, the present invention proposes a building appearance quality review system based on BIM and drone images to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solutions:

[0007] A building appearance quality review system based on BIM and drone images, including:

[0008] The data acquisition module is used to construct a BIM building model and acquire images of the building appearance to be reviewed based on the model to obtain a corresponding appearance image sequence;

[0009] A data processing module, used for performing image processing on the obtained appearance image sequence to obtain a corresponding first image sequence, and constructing a corresponding appearance panoramic image based thereon;

[0010] The data evaluation module conducts a quality audit on the appearance of the target building based on the obtained panoramic image of the appearance and in combination with the constructed BIM building model to obtain the corresponding audit results.

[0011] Furthermore, the process of constructing a BIM building model includes:

[0012] Obtain the construction drawings corresponding to the buildings that need to undergo appearance quality review, and build the corresponding initial three-dimensional model based on them; assign material properties to each building structure in the corresponding initial three-dimensional model; after the assignment is completed, check according to the material properties and the size information of each structure in the initial three-dimensional model, generate a check report, and color the initial three-dimensional model according to the check report to obtain the corresponding BIM building model.

[0013] Furthermore, the process of obtaining the corresponding appearance image sequence includes:

[0014] Obtain the building appearance areas that need to be audited in the target building, and mark the areas in the corresponding BIM building model until the marking is completed; based on all the area marks in the corresponding BIM building model, build the corresponding appearance quality inspection route based on them; and send it to the drone group deployed in the data acquisition module;

[0015] The drone group collects images of the building appearance area that needs quality review based on the constructed appearance quality inspection route to obtain a corresponding appearance image sequence; the appearance image sequence is composed of building appearance images corresponding to several different time nodes.

[0016] Furthermore, the process of performing image processing on the obtained appearance image sequence to obtain a corresponding first image sequence includes:

[0017] Decompose the corresponding building appearance image to obtain the corresponding basic image and detail images ;

[0018] Performing adaptive logarithmic processing on the obtained basic image to obtain a corresponding enhanced basic image;

[0019] Based on the predefined scale mapping function, the obtained detail image is mapped to different image scales to obtain the corresponding scale mapping image; after the scale mapping is completed, the obtained scale mapping image is subjected to adaptive guided filtering to obtain the corresponding scale image matrix ; Based on the obtained scale image matrix, the obtained scale mapping image is scaled and fused to obtain the corresponding initial detail image ;

[0020] Performing detail enhancement processing on the obtained initial detail image to obtain a corresponding enhanced detail image; reconstructing the obtained enhanced basic image and the enhanced detail image to obtain a corresponding first appearance image;

[0021] Based on the acquisition process of the first appearance image, image processing is performed on all building appearance images in the corresponding appearance image sequence, and they are sorted according to time nodes; and the corresponding first image sequence is obtained.

[0022] Furthermore, the formula for image decomposition is: ; In the formula, Represents the building appearance image, represents a bilateral filter; Indicates size The sliding window is a constant; and Represent the Gaussian standard deviation of the metric space similarity and grayscale similarity respectively; ; Represents an image of a building's exterior;

[0023] The formula for adaptive logarithmic processing is: ; In the formula, Represents the pixel points in the enhanced base image The pixel value at ; Represents the pixel points in the base image The pixel value at ; is a constant, To adjust the parameters;

[0024] The formula of the scale mapping function is: ; Represents the pixel points in the detail image At image scale The gray value at the same position in the scale mapping image corresponding to the mapping; is a fixed constant; and denote the contrast gain coefficient and the brightness gain coefficient respectively; For detail images Inner pixel The gray value at ; Represents pixel The gray level corresponding to the gray value at , Used to indicate the mapping range; Indicates the dynamic range of the sliding window under the corresponding image scale change.

[0025] Furthermore, the process of constructing a corresponding appearance panoramic image based on the first image sequence includes:

[0026] Performing feature recognition on each first appearance image in the corresponding first image sequence based on the SIFT algorithm to obtain image feature points of the corresponding first appearance image;

[0027] Respectively obtain image feature points in adjacent first appearance images, and obtain image correlations between local areas to which corresponding image feature points belong, to obtain a corresponding feature confidence matrix;

[0028] Based on the feature confidence matrix, initial pixel-level matching is performed on the image feature points in the corresponding first appearance image. After the initial matching is completed, the local features of the local area to which the corresponding feature points belong are aligned with reference to the initial matching result, and corresponding matching feature point pairs are obtained based on the alignment;

[0029] Constructing a corresponding image stitching curve based on the obtained matching feature point pairs, and acquiring the image overlapped parts between the corresponding first appearance images based on the image stitching curve, and performing image overlap fusion on the parts; obtaining a corresponding first fused image;

[0030] Based on the acquisition process of the first fused image, all the first fused images are respectively fused for a second time, and so on, to obtain a corresponding appearance panoramic image.

[0031] Furthermore, the formula for obtaining the image stitching curve is: ; In the formula, is a natural number, Indicates the number index of matching feature point pairs; is the total number of matching feature point pairs; and Respectively represent the coordinate positions of the image feature points in the tth matching feature point pair; and Respectively represent the horizontal position and vertical position of the image stitching curve;

[0032] The formula for image overlap fusion is: ; In the formula, and They refer to the image overlapping areas corresponding to the image feature points in the corresponding matching feature point pairs, express Pixel values ​​of color channels; Refers to the horizontal and vertical coordinates of the pixels within the overlapping area of ​​the images; are the height and width of the overlapping area of ​​the corresponding images; Represents the image after the overlapping area of ​​the images is fused.

[0033] Furthermore, the process of conducting a quality review of the exterior appearance of the target building based on the obtained exterior panoramic image and in combination with the constructed BIM building model includes:

[0034] Input the obtained appearance panoramic image into a pre-built appearance recognition model to obtain a corresponding model recognition result; and determine whether there are defects in the corresponding building appearance area based on the model recognition result;

[0035] If there are no defects, indicating that the quality of the corresponding building appearance area is qualified, the review is passed;

[0036] If there are defects, indicating that the quality of the corresponding building appearance area is unqualified, then corresponding defect data is obtained based on the model output result, and the defect data includes defect type and defect range;

[0037] Map the corresponding appearance panoramic image to the corresponding BIM building model for model update to obtain the corresponding defective BIM model; compare the obtained defective BIM model with the model parameters of the corresponding BIM building model, and assign defect levels to the corresponding building appearance areas based on the comparison results; color-code the defective parts of the corresponding building appearance areas in the corresponding defective BIM model based on the assigned defect levels; feed it back to the reviewer; and issue a review failure notification.

[0038] Furthermore, the construction process of the appearance recognition model includes:

[0039] The backbone network of the appearance recognition model is an improved RetinaNet network, and the basic framework of the improved RetinaNet network is composed of an input layer, a feature extraction layer and an output layer;

[0040] The input layer is used to receive input image data and perform image conversion on it to different dimensions. Process it and convert it into the original dimension, and output it after being processed by the activation function to obtain the corresponding output image;

[0041] The feature extraction layer includes a dual feature branch and a feature fusion sublayer; the dual feature branch includes a first feature branch and a second feature branch; the first feature branch is used to extract color and texture related features of the output image of the input layer to obtain a corresponding first feature vector; the second feature branch is used to extract related geometric features of the output image of the input layer to obtain a corresponding second feature vector;

[0042] The feature fusion sublayer is used to perform feature fusion on the received feature extraction vectors to obtain a corresponding fused feature vector; the feature extraction vector includes a first feature vector and a second feature vector;

[0043] A fully connected layer is provided in the output layer, and the fully connected layer performs a global average pooling process on the input fused feature vector without dimensionality reduction, and selects a convolution kernel through an adaptive function, and performs a one-dimensional convolution operation on the fused feature vector after the fully connected layer processing based on the convolution kernel, so as to complete the information interaction across channels; further, based on the output layer, the fused feature vector after the one-dimensional convolution operation is mapped to the required model output result;

[0044] Construct a corresponding training data set, which consists of positive samples and negative samples; divide the obtained training data set into several training batches, and input them into the corresponding appearance recognition model for model training, and record the corresponding loss function. If the loss function area corresponding to several consecutive training batches converges, save the model parameters, that is, the model training is completed.

[0045] Furthermore, the formula for obtaining the output image corresponding to the input layer is: ; In the formula, Represents the output image of the input layer; and MLP() represent the pre-selected activation function operation and Processing operations; Refers to a shared multilayer perceptron; represents the simulation dimension transformation matrix, The image dimension representing the image transformation; is the input image data, The image dimension representing the input image data; Represents the bias term of the input layer;

[0046] The acquisition formula of the feature extraction vector is: ; In the formula, represents the obtained feature extraction vector; and Represent channel attention weight and spatial attention weight respectively; ; In the formula, avg() and max() refer to the average pooling operation and the maximum pooling operation respectively; Represents a convolution operation; cat() represents a stacking operation along the channel; Refers to the pre-selected activation function in the feature extraction layer;

[0047] The formula for obtaining the fused feature vector is: ; In the formula, represents the fused feature vector; and denote the first eigenvector and the second eigenvector respectively; represents the weight matrix; Represents the output result of the previous layer of the feature extraction sublayer;

[0048] The formula for selecting the convolution kernel by the adaptive function is: ; In the formula, and b are constants, Represents the number of channels in the fully connected layer; Indicates the convolution kernel size;

[0049] The loss function of the appearance recognition model is defined as ; In the formula, Represents a dynamic scaling factor used to balance the number of positive and negative samples in the training data set; Represents the balance factor, which is used to overcome the imbalance problem of positive and negative sample ratios; The predicted probability that the model output is a certain defect type; and Represent positive and negative samples respectively.

[0050] Technical effects and advantages of a building appearance quality review system based on BIM and drone images of the present invention:

[0051] 1. Process the image data collected by drones in real time and generate corresponding audit results; enable auditors to timely understand the quality status of the building appearance and promptly handle and rectify existing problems; in addition, it can also dynamically monitor the quality changes of the building appearance of the BIM building model, providing strong support for the maintenance and management of the building;

[0052] 2. The present invention greatly reduces the cost of manual review and improves the efficiency and accuracy of review through automated and intelligent review methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a building appearance quality review system based on BIM and drone images of the present invention;

[0054] Figure 2 Schematic diagram of a building appearance quality review method based on BIM and drone images according to the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Example 1

[0057] See also Figure 1As shown, the building appearance quality review system based on BIM and drone images described in this embodiment includes:

[0058] The data acquisition module is used to construct a BIM building model and acquire images of the building appearance to be reviewed based on the model to obtain a corresponding appearance image sequence;

[0059] A data processing module, used for performing image processing on the obtained appearance image sequence to obtain a corresponding first image sequence, and constructing a corresponding appearance panoramic image based thereon;

[0060] The data evaluation module conducts a quality review of the target building's appearance based on the obtained panoramic image of the appearance and in combination with the constructed BIM building model to obtain the corresponding review results;

[0061] The modules are connected to each other via wired and / or wireless means to achieve data transmission between modules.

[0062] It should be further explained that, in the specific implementation process, the process of constructing a BIM building model and collecting images of the building appearance to be reviewed based on the model to obtain the corresponding appearance image sequence includes:

[0063] Obtain the construction drawings corresponding to the building that needs to undergo appearance quality review, and build a corresponding initial three-dimensional model based on the drawings, wherein the initial three-dimensional model includes the size information, building structure and other information of the corresponding building; assign material properties to each building structure in the corresponding initial three-dimensional model, wherein the material properties include name, density and mechanical property information; after the assignment is completed, perform verification according to the material properties and the size information of each structure in the initial three-dimensional model, generate a verification report, and color the initial three-dimensional model according to the verification report to obtain the corresponding BIM building model;

[0064] Acquire the building appearance area that needs to be audited in the target building, and mark the area in the corresponding BIM building model until the marking is completed; based on all the area marks in the corresponding BIM building model, build a corresponding appearance quality inspection route based on them; and send it to the drone group deployed in the data acquisition module; then, the drone group collects images of the building appearance area that needs quality audit based on the constructed appearance quality inspection route to obtain a corresponding appearance image sequence; the appearance image sequence is composed of building appearance images corresponding to several different time nodes;

[0065] There is at least one drone in the drone group, and each drone is equipped with an image acquisition terminal or a video acquisition terminal; if a video acquisition terminal is carried, key frames will be extracted from the video stream collected by the video acquisition terminal to obtain a corresponding appearance image sequence.

[0066] It should be further explained that, in a specific implementation process, the process of performing image processing on the obtained appearance image sequence to obtain the corresponding first image sequence includes:

[0067] Take any building appearance image in the appearance image sequence as an example; perform image decomposition on the corresponding building appearance image to obtain the corresponding basic image and detail images ; Among them, the formula for image decomposition is: ; In the formula, Represents the building appearance image, represents a bilateral filter; Indicates size The sliding window is a constant; and Represent the Gaussian standard deviation of the metric space similarity and grayscale similarity respectively; ;

[0068] Then, the obtained basic image is subjected to adaptive logarithmic processing to obtain a corresponding enhanced basic image; wherein the formula for performing adaptive logarithmic processing is: ; In the formula, Represents the pixel points in the enhanced base image The pixel value at ; Represents the pixel points in the base image The pixel value at ; is a constant, To adjust the parameters;

[0069] At the same time, the obtained detail image is mapped to different image scales based on a predefined scale mapping function to obtain a corresponding scale mapping image; the scale mapping function is mainly used to adaptively adjust the contrast and brightness in the corresponding detail image to suppress the background noise in the corresponding detail image; wherein, the formula of the scale mapping function is: ; Represents the pixel points in the detail image At image scale The gray value at the same position in the scale mapping image corresponding to the mapping; is a fixed constant; and Respectively represent contrast gain coefficient and brightness gain coefficient; they are set by those skilled in the art based on their work experience; For detail images Inner pixel The gray value at ; Represents pixel The gray level corresponding to the gray value at , Used to indicate the mapping range; Represents the dynamic range of the sliding window under the corresponding image scale change; the size of the sliding window is determined by the image scale Determined;

[0070] After the scale mapping is completed, the obtained scale mapping image is processed by adaptive guided filtering to obtain the corresponding scale image matrix ; In the formula, Used to represent detail-based images Scale Map Image Perform guided filtering operation; represents the pre-selected filter used for guided filtering; and The guiding parameters used to determine the filter size and blur level in the guided filtering process respectively;

[0071] Based on the obtained scale image matrix and the obtained scale mapping image, scale image fusion is performed to obtain the corresponding initial detail image ; In the formula, Indicates the maximum image scale;

[0072] Since the image clarity and detail texture clarity of the original image will be partially lost during the scale fusion process, it is necessary to perform detail enhancement processing on the obtained initial detail image to obtain the corresponding enhanced detail image; the formula for detail enhancement processing is: ; In the formula, represents the enhanced detail image, ; represents the adaptive weight matrix, Indicates obtaining the corresponding Gaussian standard deviation is the pixel value at the center pixel of the corresponding Gaussian window;

[0073] Then, reconstructing the obtained enhanced basic image and enhanced detail image to obtain a corresponding first appearance image;

[0074] Based on the acquisition process of the first appearance image, image processing is performed on all building appearance images in the corresponding appearance image sequence, and they are sorted according to time nodes; and a corresponding first image sequence is obtained;

[0075] It should be further explained that, in a specific implementation process, the process of constructing a corresponding appearance panoramic image based on the first image sequence includes:

[0076] Performing feature recognition on each first appearance image in the corresponding first image sequence based on the SIFT algorithm to obtain image feature points of the corresponding first appearance image;

[0077] Respectively obtain image feature points in adjacent first appearance images, and obtain image correlations between local regions to which corresponding image feature points belong, to obtain a corresponding feature confidence matrix, wherein the matrix elements in the feature confidence matrix are probabilities that pixels at the same position in the local region to which the corresponding feature points belong are the same object;

[0078] Then, based on the feature confidence matrix, initial pixel-level matching is performed on the image feature points in the corresponding first appearance image. After the initial matching is completed, the local features of the local area to which the corresponding feature points belong are aligned with reference to the initial matching result, and corresponding matching feature point pairs are obtained based on the alignment.

[0079] Constructing a corresponding image stitching curve based on the obtained matching feature point pairs, and acquiring the image overlapped parts between the corresponding first appearance images based on the image stitching curve, and performing image overlap fusion on the parts; obtaining a corresponding first fused image;

[0080] Based on the acquisition process of the first fused image, performing secondary fusion on all the obtained first fused images respectively, and so on, until all the first appearance images in the corresponding first image sequence are fused, and marking all the fused first appearance images as an appearance panoramic image;

[0081] Among them, the formula for obtaining the image stitching curve is: ; In the formula, is a natural number, Indicates the number index of matching feature point pairs; is the total number of matching feature point pairs; and Respectively represent the coordinate positions of the image feature points in the tth matching feature point pair;

[0082] The formula for image overlap fusion is: ; In the formula, and They refer to the image overlapping areas corresponding to the image feature points in the corresponding matching feature point pairs, express Pixel values ​​of color channels; Refers to the horizontal and vertical coordinates of the pixels within the overlapping area of ​​the images; are the height and width of the overlapping area of ​​the corresponding images; Represents the image after the overlapping area of ​​the images is fused.

[0083] It should be further explained that, in the specific implementation process, the process of quality review of the appearance of the target building based on the obtained panoramic image of the appearance and combined with the constructed BIM building model includes:

[0084] Input the obtained appearance panoramic image into a pre-built appearance recognition model to obtain a corresponding model recognition result; and determine whether there are defects in the corresponding building appearance area based on the model recognition result;

[0085] If there are no defects, indicating that the quality of the corresponding building appearance area is qualified, the review is passed;

[0086] If there are defects, indicating that the quality of the corresponding building appearance area is unqualified, then corresponding defect data is obtained based on the model output result, and the defect data includes defect type and defect range;

[0087] Then, the corresponding appearance panoramic image is mapped to the corresponding BIM building model for model update to obtain the corresponding defective BIM model; the obtained defective BIM model is compared with the model parameters of the corresponding BIM building model, and the defect level is assigned to the corresponding building appearance area based on the comparison result; the defective part of the corresponding building appearance area is color-coded in the corresponding defective BIM model based on the assigned defect level; the defect is fed back to the auditor; and an audit failure notification is issued;

[0088] It should be further explained that, in the specific implementation process, the construction process of the appearance recognition model includes:

[0089] The backbone network of the appearance recognition model is an improved RetinaNet network, and the basic framework of the improved RetinaNet network is composed of an input layer, a feature extraction layer and an output layer;

[0090] The input layer is used to receive input image data and perform image conversion on it to different dimensions. Process it and convert it into the original dimension, and output it after being processed by the activation function to obtain the corresponding output image; wherein, the input layer is processed by image conversion and The processing realizes the fusion of information of different dimensions to enhance the recognition of image features; wherein the formula for obtaining the output image is: ; In the formula, Represents the output image of the input layer; and MLP() represent the pre-selected activation function operation and Processing operations; Refers to a shared multilayer perceptron; represents the simulation dimension transformation matrix, The image dimension representing the image transformation; is the input image data, The image dimension representing the input image data; Represents the bias term of the input layer;

[0091] The feature extraction layer includes a dual feature branch and a feature fusion sublayer; the dual feature branch includes a first feature branch and a second feature branch; the first feature branch is used to extract color and texture related features of the output image of the input layer to obtain a corresponding first feature vector; the second feature branch is used to extract related geometric features of the output image of the input layer to obtain a corresponding second feature vector;

[0092] The feature fusion sublayer is used to perform feature fusion on the received feature extraction vectors to obtain a corresponding fused feature vector; the feature extraction vector includes a first feature vector and a second feature vector;

[0093] A fully connected layer is provided in the output layer, and the fully connected layer performs a global average pooling process on the input fused feature vector without dimensionality reduction, and selects a convolution kernel through an adaptive function, and performs a one-dimensional convolution operation on the fused feature vector after the fully connected layer processing based on the convolution kernel, so as to complete the information interaction across channels; further, based on the output layer, the fused feature vector after the one-dimensional convolution operation is mapped to the required model output result;

[0094] The formula for obtaining the feature extraction vector is: ; In the formula, represents the obtained feature extraction vector; and Represent channel attention weight and spatial attention weight respectively; ; In the formula, avg() and max() refer to the average pooling operation and the maximum pooling operation respectively; Represents a convolution operation; cat() represents a stacking operation along the channel; Refers to the pre-selected activation function in the feature extraction layer;

[0095] Among them, the formula for obtaining the fused feature vector is: ; In the formula, represents the fused feature vector; and denote the first eigenvector and the second eigenvector respectively; represents the weight matrix; Represents the output result of the previous layer of the feature extraction sublayer;

[0096] The formula for selecting the convolution kernel by the adaptive function is: ; In the formula, and b are constants, Represents the number of channels in the fully connected layer; Indicates the convolution kernel size;

[0097] The loss function of the appearance recognition model is defined as ; In the formula, Represents a dynamic scaling factor used to balance the number of positive and negative samples in the training data set; Represents the balance factor, which is used to overcome the imbalance problem of positive and negative sample ratios; The predicted probability that the model output is a certain defect type; and Represent positive and negative samples respectively;

[0098] Acquire a number of audited historical building appearance images, and construct a corresponding training data set based on the images, wherein the training data set consists of positive samples and negative samples; the positive samples refer to historical building appearance images marked as having qualified appearance quality based on the audit results; the negative samples refer to historical building appearance images marked as having appearance quality defects based on the audit results; wherein the historical building appearance images of the negative samples also have corresponding defect annotations, and the defect annotations include defect types and defect ranges;

[0099] The obtained training data set is divided into several training batches, and input into the corresponding appearance recognition model for model training, and the corresponding loss function is recorded. If the loss function area corresponding to several consecutive training batches converges, the model parameters are saved, that is, the model training is completed.

[0100] The present invention realizes a comprehensive and efficient review of the building appearance quality by combining the BIM building model and the image data collected by the drone; the drone can collect images of the building appearance according to the preset inspection route, covering all areas that need to be reviewed, ensuring the integrity of the review. At the same time, by performing image processing on the collected images, the key features of the building appearance are extracted, further improving the accuracy of the review.

[0101] Example 2

[0102] See also Figure 2 As shown, the part not described in detail in this embodiment is described in Example 1, which provides a method for reviewing the quality of building appearance based on BIM and drone images, including:

[0103] Step 1: Build a BIM building model, and based on it, collect images of the building appearance to be reviewed to obtain the corresponding appearance image sequence;

[0104] Step 2: performing image processing on the obtained appearance image sequence to obtain a corresponding first image sequence, and constructing a corresponding appearance panoramic image based on the first image sequence;

[0105] Step 3: Based on the obtained panoramic image of the exterior and combined with the constructed BIM building model, conduct a quality audit on the exterior of the target building to obtain the corresponding audit results.

[0106] Example 3

[0107] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the building appearance quality review system based on BIM and drone images provided above is implemented.

[0108] Since the electronic device introduced in this embodiment is an electronic device used to implement a building appearance quality review system based on BIM and drone images in the embodiment of this application, based on the building appearance quality review system based on BIM and drone images introduced in the embodiment of this application, the technical personnel of this field can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application will not be described in detail here. As long as the technical personnel of this field implement the electronic device used in the building appearance quality review system based on BIM and drone images in the embodiment of this application, it belongs to the scope of protection of this application.

[0109] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0110] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A building appearance quality review system based on BIM and drone images, characterized in that: include: The data acquisition module is used to construct a BIM building model and acquire images of the building appearance to be reviewed based on the model to obtain a corresponding appearance image sequence; A data processing module, used for performing image processing on the obtained appearance image sequence to obtain a corresponding first image sequence, and constructing a corresponding appearance panoramic image based thereon; The data evaluation module conducts a quality review of the target building's appearance based on the obtained panoramic image of the appearance and in combination with the constructed BIM building model to obtain the corresponding review results; The process of performing image processing on the obtained appearance image sequence to obtain a corresponding first image sequence includes: Decompose the corresponding building appearance image to obtain the corresponding basic image and detail images ; Performing adaptive logarithmic processing on the obtained basic image to obtain a corresponding enhanced basic image; At the same time, the obtained detail image is mapped to different image scales based on a predefined scale mapping function to obtain a corresponding scale mapping image; The formula of the scale mapping function is: ; Represents the pixel points in the detail image At image scale The gray value at the same position in the scale mapping image corresponding to the mapping; is a fixed constant; and denote the contrast gain coefficient and the brightness gain coefficient respectively; For detail images Inner pixel The gray value at ; Represents pixel The gray level corresponding to the gray value at , Used to indicate the mapping range; Indicates the dynamic range of the sliding window under the corresponding image scale change; After the scale mapping is completed, the obtained scale mapping image is processed by adaptive guided filtering to obtain the corresponding scale image matrix ; Based on the obtained scale image matrix and the obtained scale mapping image, scale image fusion is performed to obtain the corresponding initial detail image ; Perform detail enhancement processing on the obtained initial detail image; obtain a corresponding enhanced detail image; the formula for performing detail enhancement processing is: ; In the formula, represents the enhanced detail image, ; represents the adaptive weight matrix, Indicates obtaining the corresponding Gaussian standard deviation is the pixel value at the center pixel of the corresponding Gaussian window; Reconstructing the obtained enhanced basic image and enhanced detail image to obtain a corresponding first appearance image; Based on the acquisition process of the first appearance image, image processing is performed on all building appearance images in the corresponding appearance image sequence, and they are sorted according to time nodes; and the corresponding first image sequence is obtained.

2. The building appearance quality review system based on BIM and drone images according to claim 1 is characterized in that: The process of building a BIM building model includes: Obtain the construction drawings corresponding to the buildings that need to undergo appearance quality review, and build the corresponding initial three-dimensional model based on them; assign material properties to each building structure in the corresponding initial three-dimensional model; after the assignment is completed, check according to the material properties and the size information of each structure in the initial three-dimensional model, generate a check report, and color the initial three-dimensional model according to the check report to obtain the corresponding BIM building model.

3. The building appearance quality review system based on BIM and drone images according to claim 2 is characterized in that: The process of obtaining the corresponding appearance image sequence includes: Obtain the building appearance areas that need to be audited in the target building, and mark the areas in the corresponding BIM building model until the marking is completed; based on all the area marks in the corresponding BIM building model, build the corresponding appearance quality inspection route based on them; and send it to the drone group deployed in the data acquisition module; The drone group collects images of the building appearance area that needs quality review based on the constructed appearance quality inspection route to obtain a corresponding appearance image sequence; the appearance image sequence is composed of building appearance images corresponding to several different time nodes.

4. The building appearance quality review system based on BIM and drone images according to claim 3 is characterized in that: The formula for image decomposition is: ; In the formula, Represents the building appearance image, represents a bilateral filter; Indicates size The sliding window is a constant; and Represent the Gaussian standard deviation of the metric space similarity and grayscale similarity respectively; ; Represents an image of a building's exterior; The formula for adaptive logarithmic processing is: ; In the formula, Represents the pixel points in the enhanced base image The pixel value at ; Represents the pixel points in the base image The pixel value at ; is a constant, To adjust the parameters.

5. The building appearance quality review system based on BIM and drone images according to claim 4 is characterized in that: The process of constructing a corresponding appearance panoramic image based on the first image sequence includes: Performing feature recognition on each first appearance image in the corresponding first image sequence based on the SIFT algorithm to obtain image feature points of the corresponding first appearance image; Respectively obtain image feature points in adjacent first appearance images, and obtain image correlations between local areas to which corresponding image feature points belong, to obtain a corresponding feature confidence matrix; Based on the feature confidence matrix, initial pixel-level matching is performed on the image feature points in the corresponding first appearance image. After the initial matching is completed, the local features of the local area to which the corresponding feature points belong are aligned with reference to the initial matching result, and corresponding matching feature point pairs are obtained based on the alignment; Constructing a corresponding image stitching curve based on the obtained matching feature point pairs, and acquiring the image overlapped parts between the corresponding first appearance images based on the image stitching curve, and performing image overlap fusion on the parts; obtaining a corresponding first fused image; Based on the acquisition process of the first fused image, all the first fused images are respectively fused for a second time, and so on, to obtain a corresponding appearance panoramic image.

6. The building appearance quality review system based on BIM and drone images according to claim 5 is characterized in that: The formula for obtaining the image stitching curve is: ; In the formula, is a natural number, Indicates the number index of matching feature point pairs; is the total number of matching feature point pairs; and Respectively represent the coordinate positions of the image feature points in the tth matching feature point pair; and Respectively represent the horizontal position and vertical position of the image stitching curve; The formula for image overlap fusion is: ; In the formula, and They refer to the image overlapping areas corresponding to the image feature points in the corresponding matching feature point pairs, express Pixel values ​​of color channels; Refers to the horizontal and vertical coordinates of the pixels within the overlapping area of ​​the images; are the height and width of the overlapping area of ​​the corresponding images; Represents the image after the overlapping area of ​​the images is fused.

7. The building appearance quality review system based on BIM and drone images according to claim 6 is characterized in that: The process of quality review of the target building appearance based on the obtained panoramic image of the appearance and combined with the constructed BIM building model includes: Input the obtained appearance panoramic image into a pre-built appearance recognition model to obtain a corresponding model recognition result; and determine whether there are defects in the corresponding building appearance area based on the model recognition result; If there are no defects, indicating that the quality of the corresponding building appearance area is qualified, the review is passed; If there are defects, indicating that the quality of the corresponding building appearance area is unqualified, then corresponding defect data is obtained based on the model output result, and the defect data includes defect type and defect range; Map the corresponding appearance panoramic image to the corresponding BIM building model for model update to obtain the corresponding defective BIM model; compare the obtained defective BIM model with the model parameters of the corresponding BIM building model, and assign defect levels to the corresponding building appearance areas based on the comparison results; color-code the defective parts of the corresponding building appearance areas in the corresponding defective BIM model based on the assigned defect levels; feed it back to the reviewer; and issue a review failure notification.

8. The building appearance quality review system based on BIM and drone images according to claim 7 is characterized in that: The construction process of the appearance recognition model includes: The backbone network of the appearance recognition model is an improved RetinaNet network, and the basic framework of the improved RetinaNet network is composed of an input layer, a feature extraction layer and an output layer; The input layer is used to receive input image data and perform image conversion on it to different dimensions. Process it and convert it into the original dimension, and output it after being processed by the activation function to obtain the corresponding output image; The feature extraction layer includes a dual feature branch and a feature fusion sublayer; the dual feature branch includes a first feature branch and a second feature branch; the first feature branch is used to extract color and texture related features of the output image of the input layer to obtain a corresponding first feature vector; the second feature branch is used to extract related geometric features of the output image of the input layer to obtain a corresponding second feature vector; The feature fusion sublayer is used to perform feature fusion on the received feature extraction vectors to obtain a corresponding fused feature vector; the feature extraction vector includes a first feature vector and a second feature vector; A fully connected layer is provided in the output layer, and the fully connected layer performs global average pooling processing on the input fused feature vector without dimensionality reduction, and selects a convolution kernel through an adaptive function, and performs a one-dimensional convolution operation on the fused feature vector after the fully connected layer processing based on the convolution kernel, so as to complete the information interaction across channels; based on the output layer, the fused feature vector after the one-dimensional convolution operation is mapped to the required model output result; Construct a corresponding training data set, which consists of positive samples and negative samples; divide the obtained training data set into several training batches, and input them into the corresponding appearance recognition model for model training, and record the corresponding loss function. If the loss function area corresponding to several consecutive training batches converges, save the model parameters, that is, the model training is completed.

9. The building appearance quality review system based on BIM and drone images according to claim 8 is characterized in that: The formula for obtaining the output image corresponding to the input layer is: ; In the formula, Represents the output image of the input layer; and MLP() represent the pre-selected activation function operation and Processing operations; Refers to a shared multilayer perceptron; represents the simulation dimension transformation matrix, The image dimension representing the image transformation; is the input image data, The image dimension representing the input image data; Represents the bias term of the input layer; The acquisition formula of the feature extraction vector is: ; In the formula, represents the obtained feature extraction vector; and Represent channel attention weight and spatial attention weight respectively; ; In the formula, avg() and max() refer to the average pooling operation and the maximum pooling operation respectively; Represents a convolution operation; cat() represents a stacking operation along the channel; Refers to the pre-selected activation function in the feature extraction layer; The formula for obtaining the fused feature vector is: ; In the formula, represents the fused feature vector; and denote the first eigenvector and the second eigenvector respectively; represents the weight matrix; Represents the output result of the previous layer of the feature extraction sublayer; The formula for selecting the convolution kernel by the adaptive function is: ; In the formula, and b are constants, Represents the number of channels in the fully connected layer; Indicates the convolution kernel size; The loss function of the appearance recognition model is defined as ; In the formula, Represents a dynamic scaling factor used to balance the number of positive and negative samples in the training data set; Represents the balance factor, which is used to overcome the imbalance problem of positive and negative sample ratios; The predicted probability that the model output is a certain defect type; and Represent positive and negative samples respectively.

Images