A building indoor positioning method and system based on BIM cross-domain image retrieval

Abstract

The application belongs to the field of building indoor navigation, and provides a building indoor positioning method and system based on BIM cross-domain image retrieval, which comprises BIM model rendering video acquisition, BIM model rendering image set extraction, rendering image position data set acquisition, BIM model rendering image set and rendering image position data set fusion, indoor image set acquisition, feature extraction, k-d tree construction, to-be-retrieved feature acquisition and target position data matching. The application realizes the migration of image style through a CycleGAN style migration model, solves the problem of low recognition accuracy caused by large style difference in the prior art, and improves the recognition accuracy and reliability of the model; through BIM model rendering video acquisition, the construction of a large database is realized, the positioning accuracy and efficiency of the model are improved, and the demand for computing power is reduced.

Description

Technical Field

[0001] This invention relates to the field of building interior navigation, and in particular to a building interior positioning method and system based on BIM cross-domain image retrieval. Background Technology

[0002] Indoor positioning is a key technology for realizing location-based indoor services, widely used in the construction, engineering, and construction industries for monitoring construction progress, planning emergency evacuation routes in critical situations, and automatic maintenance of intelligent building facilities. The signals used in indoor positioning technologies include external signal sources based on artificially generated signals and natural signal sources based on signals generated by natural physical processes. Indoor positioning technologies based on external signal sources mainly include RFID, Bluetooth, infrared technology, wireless LAN, and ultra-wideband technology. These methods are significantly affected by noise interference in complex spatial environments, and the energy consumption and overall cost of the locators are relatively high. Sensors based on natural signal sources mainly include the Earth's magnetic field, inertial navigation, and vision. These rely less on external equipment and are therefore more versatile. Among them, vision-based indoor positioning has low cost, strong visualization capabilities, and broader application prospects. Visual positioning systems typically consist of two stages: offline database construction and online retrieval. Offline database construction in visual indoor positioning requires a large number of geotagged images, limiting its practical application. Acquiring geotagged images requires the use of acquisition equipment such as laser scanners to obtain a large number of indoor images or point clouds, constructing an indoor environment map model. The acquisition process is cumbersome and complex, requiring the inclusion of every location within the indoor environment. Positioning accuracy is affected by image density; the smaller the image acquisition interval, the more accurate the location positioning during the online retrieval stage. As the positioning area expands, the offline database data volume grows synchronously, leading to increased storage space requirements and a lack of scalability, failing to meet the positioning needs in indoor scenarios. With the increasing digitalization and informatization of construction project management, Building Information Modeling (BIM) is widely used in various fields of the construction industry. Compared with traditional data acquisition and mapping methods, extracting information from BIM is more convenient, eliminating the need for pre-constructing map models and directly obtaining location information at any location. Automatically extracting datasets from existing models is faster and more efficient than manually acquiring datasets.

[0003] However, the huge visual difference between the rendered images of the Building Information Model (BIM) and the indoor images can lead to a sharp drop in positioning performance. Compared with the high-accuracy and high-performance indoor positioning method using geotagged images, this severely limits the feasibility of building indoor positioning based on BIM. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a building interior positioning method and system based on BIM cross-domain image retrieval. By using the CycleGAN style transfer model to achieve image style transfer, the invention solves the problem of low recognition accuracy caused by large style differences in existing solutions, thereby improving the recognition accuracy and reliability of the model. Furthermore, by performing BIM model rendering video acquisition, a large database is constructed, which improves the model's positioning accuracy and efficiency while reducing computing power requirements.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A building interior positioning method based on BIM cross-domain image retrieval includes:

[0007] Construct a BIM 3D model, set up a virtual camera and several positioning reference points, and use the virtual camera to capture video based on the BIM 3D model and the positioning reference points to obtain a BIM model rendering video;

[0008] Based on the VideoCapture function, the BIM model rendering video is sampled according to a preset frame interval to obtain a BIM model rendering image set.

[0009] The rendered image location dataset is obtained based on the BIM model rendered image set and the BIM model rendered video;

[0010] The BIM model rendered image set and the rendered image location dataset are fused to obtain the BIM location image set;

[0011] The BIM location image set was converted into an indoor image set using a pre-trained CycleGAN style transfer model.

[0012] Based on MobileNetV3 small, feature extraction is performed on the indoor image set using a global average pooling layer and a pre-network layer to obtain a basic feature dataset;

[0013] A kd-tree is constructed based on the aforementioned basic feature dataset, and the partitioning dimension and partitioning points are set;

[0014] The features of the indoor images to be retrieved are obtained to obtain the features to be retrieved;

[0015] Based on the division dimension and division point, the subtrees of the kd-tree are traversed according to the features to be retrieved to obtain the most matching BIM location image, and the location information of the most matching BIM location image is extracted to obtain the target location data.

[0016] Preferably, a BIM 3D model is constructed, a virtual camera and several positioning reference points are set up, and video is captured using the virtual camera based on the BIM 3D model and the positioning reference points to obtain a BIM model rendering video, including:

[0017] Load the BIM 3D model into the pre-designed Revit plugin, and set the shading parameters, consistent color parameters, and realistic parameters for the BIM 3D model;

[0018] The positioning reference points and sampling paths are set according to user requirements;

[0019] Based on the Walkthrough mode, the virtual camera records the interior of the BIM 3D model along the sampling path to obtain a rendered video of the BIM model.

[0020] Preferably, obtaining the rendered image location dataset based on the BIM model rendered image set and the BIM model rendered video includes:

[0021] Based on the BIM model rendering video, obtain the camera position and orientation vector attributes corresponding to each image in the BIM model rendering image set;

[0022] The direction vector attribute is converted to radians to obtain the included angle in radians;

[0023] The camera position and the included angle of the radian are matched and combined to obtain the position information;

[0024] All the location information is integrated to obtain the rendered image location dataset.

[0025] Preferably, the CycleGAN style transfer model includes: a generator, a discriminator, and an FID score verification module; the generator includes: a CBAM attention mechanism module; the discriminator has the architecture of a PatchGAN structure.

[0026] Preferably, the resolution of the images in the BIM model rendering image set is 256×256 pixels.

[0027] Preferably, the plugins used by the Revit platform include RevitLookup and Add-inManager.

[0028] Preferably, a building interior positioning system based on BIM cross-domain image retrieval includes: a data acquisition module, a style transfer module, a retrieval and positioning module, and a visualization interface module;

[0029] The data acquisition module is used to acquire a BIM location image set; the style transfer module is used to perform style conversion on the BIM location image set to obtain the indoor image set; the retrieval and positioning module is used to match the indoor image set with the kd-tree of the indoor image set to obtain the target location data; the visualization interface module is used to display offline database creation and online retrieval content.

[0030] The present invention discloses the following technical effects:

[0031] This invention provides a building interior positioning method and system based on BIM cross-domain image retrieval. It achieves image style transfer through CycleGAN style transfer model, solving the problem of low recognition accuracy caused by large style differences in existing solutions, and improving the recognition accuracy and reliability of the model. By performing BIM model rendering video acquisition, it solves the problem of poor positioning accuracy caused by low data volume and realizes the construction of a large database. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of the building interior positioning process based on BIM cross-domain image retrieval provided in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the implementation process of building indoor positioning based on BIM cross-domain image retrieval provided in an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the style transfer module provided in an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of an image retrieval module provided in an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the overall process provided for an embodiment of the present invention;

[0038] Figure 6 This is a user interface diagram for offline database creation provided in an embodiment of the present invention;

[0039] Figure 7 A user interface diagram for the online retrieval stage provided in an embodiment of the present invention;

[0040] Figure 8This is a system module diagram provided for an embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] The purpose of this invention is to provide a building interior positioning method and system based on BIM cross-domain image retrieval. By using the CycleGAN style transfer model to achieve image style transfer, the problem of low recognition accuracy caused by large style differences in existing solutions is solved, thereby improving the recognition accuracy and reliability of the model. By performing BIM model rendering video acquisition, a large database is constructed, which improves the positioning accuracy and efficiency of the model and reduces the computing power requirements.

[0043] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Figure 1 This is a schematic diagram of the building interior positioning process based on BIM (Building Information Modeling) cross-domain image retrieval, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the implementation process of building indoor positioning based on BIM cross-domain image retrieval provided in an embodiment of the present invention, such as... Figure 1 and Figure 2 As shown, this invention provides a building interior positioning method based on BIM cross-domain image retrieval, including:

[0045] Step 100: Construct a BIM 3D model, set up a virtual camera and several positioning reference points, and use the virtual camera to capture video based on the BIM 3D model and positioning reference points to obtain a BIM model rendering video;

[0046] Step 20: Using the VideoCapture function on the OpenCV platform, sample the BIM model rendering video according to the preset frame interval to obtain the BIM model rendering image set;

[0047] Step 300: Obtain the rendered image location dataset based on the BIM model rendered image set and the BIM model rendered video;

[0048] Step 400: Merge the BIM model rendered image set and the rendered image location dataset to obtain the BIM location image set;

[0049] Step 500: Use the pre-trained CycleGAN style transfer model to convert the BIM location image set into an indoor image set;

[0050] Step 600: Based on MobileNetV3 small, use global average pooling layers and pre-network layers to extract features from the indoor image set to obtain the basic feature dataset;

[0051] Step 700: Construct a kd-tree based on the basic feature dataset, and set the partition dimension and partition points;

[0052] Step 800: Obtain the features of the indoor image to be retrieved to obtain the features to be retrieved;

[0053] Step 900: Based on the partitioning dimension and partitioning point, traverse the subtrees of the kd-tree according to the features to be retrieved to obtain the most matching BIM location image, and extract the location information of the most matching BIM location image to obtain the target location data.

[0054] Preferably, a BIM 3D model is constructed, a virtual camera and several positioning reference points are set up, and video is captured using the virtual camera based on the BIM 3D model and positioning reference points to obtain a BIM model rendering video, including:

[0055] Load the BIM 3D model into a pre-designed Revit plugin and set the shading parameters, consistent color parameters, and realistic parameters for the BIM 3D model;

[0056] Set the positioning reference points and sampling path according to user needs;

[0057] Based on the Walkthrough mode, a virtual camera is used to record the interior of the BIM 3D model along the sampling path to obtain a BIM model rendering video.

[0058] Preferably, the rendered image location dataset is obtained based on the BIM model rendered image set and the BIM model rendered video, including:

[0059] Obtain the camera position and orientation vector attributes corresponding to each image in the BIM model rendering image set based on the BIM model rendering video;

[0060] Convert the direction vector attribute to radians to obtain the included angle in radians;

[0061] The position information is obtained by matching and combining the camera position and the included angle of the radian.

[0062] By integrating all the location information, a dataset of rendered image locations is obtained.

[0063] Specifically, the Revit-based location information acquisition plugin consists of three parts: the main implementation code, the front-end interface, and the back-end interface.

[0064] The implementation code primarily uses the ActiveView property of the Document property in the Revit Document class to obtain the currently active view, which is the 3D view of the BIM model. It then uses the GetSavedOrientation method in the View3D method of the View3D class to obtain the orientation vector property of the 3D view of the BIM model, while simultaneously recording the current frame information of the virtual camera. Finally, it uses the EyePosition property of the ViewOrientation3D class to obtain the camera coordinates in the 3D view, thus obtaining the 3D coordinate information of the geographical location.

[0065] The virtual camera's orientation is obtained through the `ForwardDirection` property. Since the positive X-axis is 0 degrees, the rotation angle in radians of the current virtual camera's orientation is obtained using the `AngleTo` method within the `XYZ` class. This radian value is then converted back to an angle to obtain the geographic location angle information. `α` represents the return value obtained using `XYZ.BasisX.AngleTo(vector)`, which represents the angle between the virtual camera's orientation and the positive X-axis obtained through the `AngleTo` method.

[0066] Preferably, the CycleGAN style transfer model includes: a generator, a discriminator, and an FID score verification module; the generator includes: a CBAM attention mechanism module; the discriminator has the architecture of a PatchGAN structure.

[0067] Preferably, the resolution of the images in the BIM model rendering image set is 256×256 pixels.

[0068] Preferably, the plugins used by the Revit platform include Revit Lookup and Add-inManager.

[0069] Preferably, a building interior positioning system based on BIM cross-domain image retrieval includes: a data acquisition module, a style transfer module, a retrieval and positioning module, and a visualization interface module;

[0070] The data acquisition module is used to acquire BIM location image sets; the style transfer module is used to perform style conversion on the BIM location image sets to obtain indoor image sets; the retrieval and positioning module is used to match the indoor image sets with kd-trees to obtain target location data; and the visualization interface module is used to display offline database creation and online retrieval content.

[0071] Specifically, the Revit-based location information acquisition plugin was developed on a DELLT3600 graphics workstation with a Windows 10 system environment, a Visual Studio 2022 development environment, and the Revit 2022 SDK as the reference document. It uses the Revit Lookup and Add-inManager plugins.

[0072] Preferably, the location information acquisition process includes: acquiring the orientation of the 3D view of the BIM model and recording camera frame information. Obtaining the geographic location coordinates and angles through the camera position and orientation. Calculating the inter-frame angle change and saving it as XML data to record the rendered image location information. Assuming positive X is 0 degrees (XYZ corresponds to the BIM model; when adding a location label plugin for Revit secondary development, the XYZ axes and origin correspond one-to-one with the building BIM model), the rotation angle in radians of the current virtual camera orientation is obtained through the AngleTo method of the XYZ class, and then converted to an angle value to obtain the geographic location angle information. The following formula exists:

[0073]

[0074] Here, α represents the return value obtained using the code XYZ.BasisX.AngleTo(vector), that is, the angle between the virtual camera orientation and the positive X-axis obtained by the AngleTo method.

[0075] refer to Figure 3 This paper improves the CycleGAN training framework by reconstructing the generator and discriminator structures to achieve style transfer from BIM model rendered images to indoor photographed images. A CBAM attention mechanism module is added to the residual block of the CycleGAN generator. After the convolutional layer output, channel attention weighting is applied first, followed by spatial attention weighting, ultimately resulting in enhanced feature representations. The discriminator adopts a PatchGAN structure, adding a spectral normalization layer after the convolutional layers of the discriminator network. An FID score verification module is constructed and incorporated into the training framework. The training samples of the BIM model rendered images and indoor photographed images are denoted as a and b, respectively. The generator G... AB The generator G is responsible for converting rendered BIM model images into composite images in the style of indoor photographs. BA Then the reverse process is executed, discriminator D B This is used to distinguish between composite images and indoor photographic images generated by the generator, which are styled like indoor photographic images. The cycle consistency process involves the generator taking a BIM model rendered image as input and generating a composite image in the style of an indoor photographic image. The input to the generator G is... AB Generate reconstructed image G BA (G AB (a)). Figure 3 In this context, FID (Fixed Similarity Index) is a metric used to measure the quality and diversity of style-synthesized images generated by a generative model from indoor photographed images. It calculates the difference between the generated image and the indoor photographed image by comparing their feature distributions. Higher similarity results in a lower FID score; when the datasets are completely identical, the FID score is 0. The formula for calculating the FID score is:

[0076] FID = ||μ r -μ g || 2 +Tr(∑r+∑g-2(∑r∑g) 1 / 2 )

[0077] In the formula, r and g represent the indoor image and the composite image in the style of the indoor image, respectively, and μ r and μ g Let ∑r and ∑g represent the average eigenvectors of the two images, respectively, while ∑r and ∑g represent the covariance matrices of the two images, respectively. The trace is denoted by Tr.

[0078] InceptionV3 is a pre-trained feature extractor that can extract feature vectors from samples of indoor image datasets and synthetic image datasets with indoor image styles. Lipschitz continuity is controlled by the maximum singular value, allowing the parameter matrix to generate diverse images. Regarding training stability, the CycleGAN discriminator network consists of multiple convolutional and activation layers, equivalent to a composite function. For a composite function to satisfy 1-Lipschitz continuity, each function composing the composite function must also satisfy 1-Lipschitz continuity. Since the Leaky ReLU activation function naturally satisfies 1-Lipschitz continuity, ensuring that the convolutional layers satisfy 1-Lipschitz continuity is sufficient for the CycleGAN discriminator network to satisfy 1-Lipschitz continuity. The spectral norm of matrix A is defined as follows:

[0079]

[0080] In the formula, h represents the input, σ(A) represents the maximum singular value of matrix A, and σ(W) represents the spectral norm of the convolutional layer parameter matrix, W. SN This represents the parameter matrix of the convolutional layer after SN processing. W SN The calculation formula is:

[0081] W SN =W / σ(A)

[0082] The maximum singular value of the convolutional layer using the CycleGAN-SN algorithm is always 1, and the input-output characteristics of the entire discriminant network also conform to the 1-Lipschitz condition.

[0083] refer to Figure 4 The steps for cross-domain image retrieval and positioning based on MobileNetV3 small image feature extraction, kd-tree-based image feature retrieval, and mapping of location information include: using MobileNetV3 small as the basis for image feature extraction, and using global average pooling layers and previous network layers for feature extraction; constructing a kd-tree, selecting partition dimensions and partition points, and iteratively selecting dimensions according to the order of feature values; starting from the root node of the kd-tree, comparing the feature values ​​of the indoor image to be retrieved with the median of the node, selecting the corresponding subtree for traversal, until the leaf node is reached; using the BIM model to render the image to obtain the corresponding location information and the location of the floor plan in the BIM model, and using the frame value correspondence relationship to match and store the acquired rendered image and location information based on the location information collected by Revit.

[0084] Furthermore, the specific steps of the location positioning method based on BIM rendered images are as follows: obtain the BIM model rendered image before CycleGAN style transfer; after style transfer using CycleGAN, use Python's replace function to remove the _translated suffix from the image name to match the BIM model rendered image; use os.listdir to obtain the original image file name; combine the location information of the BIM model rendered image; and obtain the 3D position, angle, and floor plan address through regular expressions and XML parsing. By combining indoor photographed images with the BIM model, the user's position in the model can be accurately displayed.

[0085] Specifically, the building interior positioning system based on BIM cross-domain image retrieval includes: a data acquisition module, which acquires BIM model rendered images and corresponding location information, specifically including the acquisition and display of rendered image data and BIM data location information; a style transfer module, which trains an improved CycleGAN to transfer the style of the acquired BIM model rendered images to synthetic images with an interior shooting style; a retrieval and positioning module, which retrieves the generated image most similar to the image to be retrieved and retrieves the BIM model rendered image before style transfer, and positions the image based on tag information; and a visualization interface module, which provides users with a graphical interface for intuitive operation of the entire system. The graphical interface is developed based on the Tkinter library and is used to display offline database creation and online retrieval content.

[0086] refer to Figure 5 The specific implementation process of the building indoor positioning system based on BIM cross-domain image retrieval is as follows:

[0087] A building interior positioning system based on BIM cross-domain image retrieval includes an offline database construction phase and an online retrieval phase.

[0088] The offline database construction phase includes a data acquisition module and a style transfer module. The data acquisition module includes uploading captured videos, BIM data acquisition functions, and displaying BIM data location information.

[0089] Define the select_file function, use the filedialog.askopenfilename function to open the file dialog box and return the path of the selected video file, thereby completing the video recording and uploading;

[0090] Define the select_bimfile function, use the subprocess.Popen function to open the Revit software, and perform data acquisition based on the Revit plugin developed in this paper to complete the acquisition of BIM data;

[0091] Define the function get_coordinates_and_angles_from_xml, use the ET.parse and getroot functions to parse the XML file and root element saved based on the Revit plugin, traverse the carmeaInfo elements, use the get function to obtain the three-dimensional position information and angle information, and complete the display of BIM data position information.

[0092] The style transfer module includes setting video frame cropping, displaying the number of images, saving the image location, selecting the style transfer model, setting training hyperparameters, a style transfer button, the number of training rounds, the loss function, the iteration training time, real-time FID, displaying real-time style transfer images, and evaluation metrics.

[0093] Configure video frame capture, number of displayed images, and save location: Define the `capture_frames` function, which sets the frame interval for capturing images by setting the `frame_interval` value, and prints `save_frames` to display the number of images. Define the `save_path` function and use the `os.path.join` function to construct the complete save path;

[0094] Selecting a style transfer model and setting training hyperparameters: Define the change_config function, use the subprocess.Popen function to open the CycleGAN config file, and modify the model and loss function in model, train_pipeline to modify the training preprocessing, optimizer to modify the optimizer, lr_config to modify the starting point of the second-stage learning rate, total_iters to modify the total number of iterations, etc.

[0095] The system includes a style transfer button, training epochs, loss function, iteration training time, real-time FID, and display of real-time style transfer images. It defines the `run_translated` function, which runs the `slurm_train` script using the `subprocess.run` function. It defines the `extract_numbers` function, which extracts the epoch, loss, time, and FID keyword values ​​using the `re.search` function and adds them to the display list. It defines the `get_image` function, which lists all saved files and filters out image files using the `os.listdir` function, sorts them using the `max` function to obtain the latest epoch's style transfer image, and updates it in real-time using the `time.sleep` function.

[0096] Define the functions calculate_fid, calculate_psrn, calculate_ssim, and calculate_css for evaluation.

[0097] Figure 5 In the MobileNetV3 architecture, to meet the needs of different scenarios, the model is divided into two versions: large and small. The small version is designed to be more lightweight, with fewer parameters and faster computation speed. Given the functionality required in this embodiment, MobileNetV3small is chosen as the basis for image feature extraction, using a global average pooling layer and the preceding network layers for feature extraction. Specifically, the global average pooling layer of MobileNetV3 small is truncated to retain its output, resulting in a three-dimensional tensor with a height of 1, a width of 1, and a depth of 576 as the image feature vector. Through a flattening operation, the 1×1×576 three-dimensional tensor feature is flattened into a 576 one-dimensional feature vector. By flattening the image feature vector into a one-dimensional array, similarity calculation and retrieval operations are performed more efficiently, achieving the required functionality. Table 1 shows the detailed information of the MobileNetV3 small framework.

[0098] Table 1

[0099]

[0100] Specifically, this embodiment uses MobileNetV3 small to extract image features before the global pooling layer from indoor style-synthesized images collected in the offline dataset acquisition stage, storing them as an image feature database in the offline dataset. In the online image retrieval and localization stage, image features are extracted from the indoor images to be retrieved. The purpose of these feature extractions is to quantify the similarity and differences between images, providing similarity calculation parameters for subsequent retrieval and localization.

[0101] refer to Figure 6 A graphical interface was developed based on the Tkinter library to display offline database creation content. The graphical interface is as follows: Figure 6 As shown, the offline database creation interface, from top to bottom on the left, includes the style transfer button, upload captured video, BIM data acquisition function, style transfer model, settings for video frames, number of images, save location, and display of BIM data location information. The right side includes hyperparameter settings, rounds, loss function, iteration time, FID, save location, detection metrics, and results. The offline database creation stage includes a data acquisition module and a style transfer module. The data acquisition module includes uploading captured video, BIM data acquisition function, and displaying BIM data location information. The data acquisition module collects indoor captured images and BIM model rendered images, along with their corresponding location information. Specifically, it includes uploading video, BIM data acquisition function, and displaying BIM data location information. The functions of each button are defined as follows: Upload actual video button: opens a file dialog box and returns the path to the selected captured video file; BIM extract graphics button: opens Revit software to acquire BIM rendered images; Display BIM data location information button: obtains the 3D location and angle information of the selected image and adds it to the display list.

[0102] Specifically, the style transfer module trains an improved CycleGAN and transfers the style of the collected BIM model rendered images to a composite image with an indoor shooting style. The settings include: a video frame capture button (sets the frame interval for capturing images, and the video will capture images at preset intervals to form a rendered image dataset); a save location button (selects the save path for the captured rendered image dataset); a style transfer model button (selects the model to use for style transfer); a training hyperparameter button (opens the config file of the selected model, allowing modification of parameters such as the optimizer, learning rate, and number of iterations, and displays the loss function, iteration time, and real-time FID obtained from the current parameter configuration); and a style transfer button (runs the pre-set script to perform style transfer on the BIM rendered images, obtaining a style-transferred composite image of the uploaded image).

[0103] Furthermore, the online search stage mainly consists of the search and positioning module, which includes uploading the image to be searched, search methods, search buttons, search time, displaying the best matching image, the original image before style transfer, displaying 3D coordinates and angle position information, BIM model floor plan, etc.

[0104] Define the function select_file, which uses the filedialog.askopenfilename function to open a file dialog box and return the path of the selected image file to be retrieved;

[0105] Define functions such as save_features_LBP, save_features_SIFT, and save_features_RGB to perform image feature retrieval based on kd-tree; define the run_retrieval function, use the subprocess.run function to run the slurm_retrieval script, use the time.time function to calculate the retrieval time, use the os.path.basename function to get the base name, and add the best matching image to the display column;

[0106] Define the find_original function, use the replace function to remove the _translated suffix after CycleGAN style transfer, use the os.listdir function to get the original image file name, and retrieve the original image before style transfer and add it to the display column;

[0107] Define the extract_info function, use the re.compile function to match attributes using regular expressions, use the ET.parse and getroot functions to parse the XML file and root element saved based on the Revit plugin, use the get function to obtain the 3D position, angle information, and BIM model floor plan address, search and match the Name keyword, and use the re.match function to extract information and add it to the display column;

[0108] The online search interface, from top to bottom on the left, displays the search button, upload actual image, best matching image, original image before style transfer, search method, search time, and usage weight. The right side displays 3D coordinates and angle position information, and BIM model floor plans, such as... Figure 7 As shown. The online search phase mainly consists of the search and location module, including uploading the image to be searched, search method, search button, search time, displaying the best matching image, the original image before style transfer, displaying 3D coordinates and angle position information, and BIM model floor plan, etc. The "Upload Image to Search" button opens the file dialog box and returns the file path of the selected image; the "Search Method" button selects the image retrieval model to be used; the "Search" button runs the pre-set script to search for indoor images, obtaining the specific location information of the building where the image is located, the search time, and the weight used.

[0109] refer to Figure 8The system comprises several modules: The data acquisition module obtains datasets based on BIM models, including acquiring BIM model information, utilizing Revit software and its API for secondary development, and obtaining rendered images and location information of the BIM model. The image style transfer module adds a CBAM attention mechanism to the generator residual block, introduces the PReLU activation function in the convolutional layer, uses a label smoothing strategy in the discriminator, and introduces spectral normalization in the convolutional layer. The real-time FID module is merged into the training framework to obtain the optimal number of training epochs. The acquired dataset is used to improve the CycleGAN network to obtain synthetic images with an indoor shooting style. The retrieval and localization module is a cross-domain image retrieval and localization module based on MobileNetV3 small-kd trees. It extracts image features based on MobileNetV3 small, performs feature retrieval based on kd trees, obtains the image with the highest matching degree based on cosine similarity, and acquires the location information of that image to achieve localization.

[0110] Specifically, the BIM cross-domain image retrieval building indoor positioning system developed in this embodiment consists of a DELL T3660 graphics workstation server, a Jestson Nano embedded system, a CSI camera, a Bull power bank, and a personal computer. The visualization interface module is deployed on the personal computer, with a Secure Shell (SSH) client installed. An SSH server is installed on the data acquisition module, the DELL T3660 graphics workstation server, and the Jestson Nano embedded system. The data acquisition module, style transfer module, and retrieval and positioning module are connected via SSH commands and SSH-based Secure Copy (SCP) commands. SSH is an encrypted network protocol that replaces Tenlnet, preventing information leakage during transmission. It is used to securely access remote computers on networks with potential for data breaches. It consists of a transport layer protocol, a user authentication protocol, and a connection protocol. The SSH service is started in the terminal using the `ssh start` command. Compared to password login (entering username, IP address, and password), key-based login eliminates the need to enter a password each time and is more difficult to steal. Specifically, the `ssh-keygen` command generates a key pair, `ssh-copy-id` copies the public key `id_ras.pub` to the server, and `id_rsa` is used to connect to the server using the private key. SCP uses SSH to complete network transmission based on the BSDRCP protocol, ensuring the authenticity and confidentiality of file transfers.

[0111] Furthermore, the data acquisition module uses a tripod and smartphone to capture indoor video and acquires rendered images of the BIM model based on Revit. It receives SSH commands from the visualization interface module to upload both to the DELL T3660 graphics workstation server, uploads the Revit-based location information to the JestsonNano embedded system, and returns the acquisition results to the visualization interface module via SCP commands.

[0112] Preferably, the DELLT3660 graphics workstation server in the style transfer module receives SSH commands from the visualization interface module to set video frame capture images, number of images, save location, select style transfer model, and set training hyperparameters. During style transfer training, training results such as the number of training rounds, loss function, iteration training time, real-time FID, style transfer images, and evaluation metrics are transmitted back to the visualization interface module via SCP, and the style transfer images are uploaded to the retrieval and localization module.

[0113] Specifically, the DELLT3660 graphics workstation server in the retrieval and positioning module receives SSH commands from the visualization interface module and extracts style transfer image features using an extraction method. The extracted image feature dataset is uploaded to the JestsonNano embedded system. The JestsonNano embedded system receives SSH commands from the visualization interface module, captures and uploads the image to be retrieved, selects a retrieval method, and uses image feature retrieval and location positioning methods combined with the uploaded image feature dataset and BIM model location information to perform retrieval and positioning. The data is then transmitted back to the visualization interface module via SCP, displaying the retrieval time, the best matching image, the original image before style transfer, 3D coordinates and angle position information, and the BIM model floor plan.

[0114] Preferably, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements any step of this embodiment.

[0115] The beneficial effects of this invention are as follows:

[0116] This invention achieves image style transfer through the CycleGAN style transfer model, solving the problem of low recognition accuracy caused by large style differences in existing solutions, and improving the recognition accuracy and reliability of the model; by performing BIM model rendering video acquisition, a large database is constructed, which improves the model's positioning accuracy and efficiency, and reduces computing power requirements.

[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0118] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A building interior positioning method based on BIM cross-domain image retrieval, characterized in that, include: Construct a BIM 3D model, set up a virtual camera and several positioning reference points, and use the virtual camera to capture video based on the BIM 3D model and the positioning reference points to obtain a BIM model rendering video; Based on the VideoCapture function, the BIM model rendering video is sampled according to a preset frame interval to obtain a BIM model rendering image set. The rendered image location dataset is obtained based on the BIM model rendered image set and the BIM model rendered video; The BIM model rendered image set and the rendered image location dataset are fused to obtain the BIM location image set; The BIM location image set was converted into an indoor image set using a pre-trained CycleGAN style transfer model. Based on MobileNetV3 small, feature extraction is performed on the indoor image set using a global average pooling layer and a pre-network layer to obtain a basic feature dataset; A kd-tree is constructed based on the aforementioned basic feature dataset, and the partitioning dimension and partitioning points are set; The features of the indoor images to be retrieved are obtained to obtain the features to be retrieved; Based on the division dimension and division point, the subtrees of the kd-tree are traversed according to the features to be retrieved to obtain the most matching BIM location image, and the location information of the most matching BIM location image is extracted to obtain the target location data.

2. The building interior positioning method based on BIM cross-domain image retrieval according to claim 1, characterized in that, A BIM 3D model is constructed, a virtual camera and several positioning reference points are set up, and video is captured using the virtual camera based on the BIM 3D model and the positioning reference points to obtain a BIM model rendering video, including: Load the BIM 3D model into the pre-designed Revit plugin, and set the shading parameters, consistent color parameters, and realistic parameters for the BIM 3D model; The positioning reference points and sampling paths are set according to user requirements; Based on the Walkthrough mode, the virtual camera records the interior of the BIM 3D model along the sampling path to obtain a rendered video of the BIM model.

3. The building interior positioning method based on BIM cross-domain image retrieval according to claim 1, characterized in that, The rendered image location dataset is obtained based on the BIM model rendered image set and the BIM model rendered video, including: Based on the BIM model rendering video, obtain the camera position and orientation vector attributes corresponding to each image in the BIM model rendering image set; The direction vector attribute is converted to radians to obtain the included angle in radians; The camera position and the included angle of the radian are matched and combined to obtain the position information; All the location information is integrated to obtain the rendered image location dataset.

4. The building interior positioning method based on BIM cross-domain image retrieval according to claim 1, characterized in that, The CycleGAN style transfer model includes a generator, a discriminator, and an FID score verification module; the generator includes a CBAM attention mechanism module; the discriminator has a PatchGAN architecture.

5. A building interior positioning method based on BIM cross-domain image retrieval according to claim 1, characterized in that, The resolution of the images in the BIM model rendering image set is 256×256 pixels.

6. A building interior positioning method based on BIM cross-domain image retrieval according to claim 2, characterized in that, The plugins used by the Revit platform include Revit Lookup and Add-in Manager.

7. A building indoor positioning system based on BIM cross-domain image retrieval, characterized in that, The building interior positioning method based on BIM cross-domain image retrieval as described in claim 1 includes: a data acquisition module, a style transfer module, a retrieval and positioning module, and a visualization interface module; The data acquisition module is used to acquire a BIM location image set; the style transfer module is used to perform style conversion on the BIM location image set to obtain the indoor image set; the retrieval and positioning module is used to match the indoor image set with the kd-tree of the indoor image set to obtain the target location data; the visualization interface module is used to display offline database creation and online retrieval content.

Images