Building indoor positioning method and system based on BIM cross-domain image retrieval

Abstract

The invention belongs to the field of building indoor navigation, and provides a building indoor positioning method and system based on BIM cross-domain image retrieval. Comprising the steps of BIM model rendering video acquisition, BIM model rendering image set extraction, rendering image position data set acquisition, BIM model rendering image set and rendering image position data set fusion, indoor image set acquisition, feature extraction, k-d tree construction, to-be-retrieved feature acquisition and target position data matching. According to the method, image style migration is realized through the CycleGAN style migration model, the problem of relatively low recognition precision caused by large style difference in an existing scheme is solved, and the recognition precision and reliability of the model are improved; through BIM model rendering video acquisition, the construction of a large database is realized, the positioning accuracy and efficiency of the model are improved, and the computing power demand is reduced.

Description

Technical Field

[0001] The present invention relates to the field of building indoor navigation, and in particular to a building indoor positioning method and system based on BIM cross-domain image retrieval. Background Art

[0002] Indoor positioning is a key technology for realizing location-based indoor services. It is widely used in construction progress monitoring, emergency evacuation route planning in critical situations, and automatic maintenance of intelligent building facilities in the construction, engineering and construction industries. The signals used in indoor positioning technology include external sources based on artificially generated signals and natural sources based on signals generated by physical processes in nature. Indoor positioning technologies based on external sources mainly include radio frequency identification, Bluetooth, infrared technology, wireless local area network and ultra-wideband technology. These methods are more obviously interfered by noise signals in a more complex spatial environment, and the energy consumption and overall cost of the locator are high. Sensors based on natural sources are mainly the earth's magnetic field, inertial navigation and vision. They rely on fewer external devices and are therefore more applicable. Among them, indoor positioning based on vision has low cost, strong visualization function and broader application prospects. Visual positioning systems are usually divided into two stages: offline database construction and online retrieval. Offline database construction requires a large number of geo-tagged images in visual indoor positioning, which limits its practical application. To obtain geo-tagged images, it is necessary to use acquisition equipment such as laser scanners to obtain a large number of indoor images or point clouds and build indoor environment map models. The acquisition process is cumbersome and complex and needs to include every location of indoor positioning. Positioning accuracy is affected by the density of images. The smaller the image acquisition interval, the more accurate the location positioning in the online retrieval stage. As the positioning area expands, the amount of data in the offline database grows synchronously, resulting in an increase in storage space requirements and a lack of scalability, which cannot meet the needs of positioning in indoor scenes. As construction project management gradually develops towards digitalization and informatization, building information models are widely used in many fields of the construction industry. Compared with traditional data collection and mapping methods, extracting information from building information models is more convenient. There is no need to pre-build a map model and directly obtain location information at any location. Automatically extracting data sets from existing models is faster and more efficient than manually collecting data sets.

[0003] However, the huge visual difference between the rendered images of the building information model and the indoor captured images will cause a sharp drop in positioning performance. Compared with the high-precision and good-effect indoor positioning method directly using geo-tagged images, the feasibility of indoor positioning of buildings based on building information models is seriously limited. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method and system for indoor building positioning based on BIM cross-domain image retrieval, realize image style transfer through the CycleGAN style transfer model, solve the problem of low recognition accuracy caused by large style differences in the existing scheme, and improve the recognition accuracy and reliability of the model; by collecting BIM model rendering video, a large database can be built to improve the positioning accuracy and efficiency of the model and reduce computing power requirements.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A building indoor positioning method based on BIM cross-domain image retrieval, comprising:

[0007] Constructing a BIM stereo model, setting a virtual camera and a number of positioning reference points, and using the virtual camera to perform video acquisition based on the BIM stereo model and the positioning reference points to obtain a BIM model rendering video;

[0008] Based on the VideoCapture function, the BIM model rendering video is sampled according to a preset frame number interval to obtain a BIM model rendering image set;

[0009] Acquire a rendering image position data set according to the BIM model rendering image set and the BIM model rendering video;

[0010] The BIM model rendering image set and the rendering image location data set are merged to obtain a BIM location image set;

[0011] Using a pre-trained CycleGAN style transfer model to convert the BIM location image set into an indoor image set;

[0012] Based on MobileNetV3 small, the global average pooling layer and the front network layer are used to extract features of the indoor image set to obtain a basic feature data set;

[0013] Constructing a kd tree according to the basic feature data set, and setting the partitioning dimension and partitioning point;

[0014] Acquire features of the indoor image to be retrieved to obtain features to be retrieved;

[0015] Based on the division dimension and division point, the subtree of the kd tree is traversed according to the feature to be retrieved to obtain the best matching BIM location image, and the location information of the best matching BIM location image is extracted to obtain the target location data.

[0016] Preferably, constructing a BIM stereo model, setting a virtual camera and a plurality of positioning reference points, and using the virtual camera to perform video acquisition based on the BIM stereo model and the positioning reference points to obtain a BIM model rendering video includes:

[0017] Loading the BIM three-dimensional model into a pre-designed Revit plug-in, and setting shading parameters, consistent color parameters, and real parameters for the BIM three-dimensional model;

[0018] Setting the positioning reference point and sampling path according to user requirements;

[0019] Based on the Walk through mode, the virtual camera is used to record the interior of the BIM three-dimensional model along the sampling path to obtain the BIM model rendering video.

[0020] Preferably, acquiring a rendered image position data set according to the BIM model rendered image set and the BIM model rendered video includes:

[0021] Obtaining camera position and direction vector attributes corresponding to each image in the BIM model rendering image set according to the BIM model rendering video;

[0022] Convert the direction vector attribute into radians to obtain an angle in radians;

[0023] Matching and combining the camera position and the arc angle to obtain position information;

[0024] All the position information is integrated to obtain the rendered image position data set.

[0025] Preferably, the CycleGAN style transfer model includes: a generator, a discriminator and a FID score verification module; the generator includes: a CBAM attention mechanism module; the architecture of the discriminator is: a PatchGAN structure.

[0026] Preferably, the resolution of the images in the BIM model rendering image set is 256×256 pixels.

[0027] Preferably, the plug-ins used by the Revit platform include: RevitLookup and Add-inManager.

[0028] Preferably, a building indoor positioning system based on BIM cross-domain image retrieval comprises: a data acquisition module, a style transfer module, a retrieval and positioning module, and a visualization interface module;

[0029] The data acquisition module is used to obtain a BIM location image set; the style transfer module is used to perform style conversion on the BIM location image set to obtain the indoor image set; the retrieval positioning module is used to match the indoor captured images to be retrieved using a kd tree according to the indoor image set to obtain target location data; the visualization interface module is used to display offline library building and online retrieval content.

[0030] The present invention discloses the following technical effects:

[0031] The present invention provides a method and system for indoor building positioning based on BIM cross-domain image retrieval, which realizes image style migration through the CycleGAN style migration model, solves the problem of low recognition accuracy caused by large style differences in the existing solution, and improves the recognition accuracy and reliability of the model; by performing BIM model rendering video acquisition, the problem of poor positioning accuracy caused by low data volume is solved, and the construction of a large database is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 A schematic diagram of a building indoor positioning process based on BIM cross-domain image retrieval provided by an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of an implementation process of building indoor positioning based on BIM cross-domain image retrieval provided by an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of a style transfer module provided by an embodiment of the present invention;

[0036] Figure 4 A schematic diagram of an image retrieval module provided by an embodiment of the present invention;

[0037] Figure 5 A schematic diagram of the overall process provided by an embodiment of the present invention;

[0038] Figure 6 This is a diagram of the offline database building user interface provided by an embodiment of the present invention;

[0039] Figure 7 A user interface diagram of the online search phase provided by an embodiment of the present invention;

[0040] Figure 8A system module diagram provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The purpose of the present invention is to provide a method and system for indoor positioning of buildings based on BIM cross-domain image retrieval, realize image style migration through the CycleGAN style migration model, solve the problem of low recognition accuracy caused by large style differences in existing solutions, and improve the recognition accuracy and reliability of the model; by collecting BIM model rendering videos, a large database can be built to improve the positioning accuracy and efficiency of the model and reduce computing power requirements.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Figure 1 A schematic diagram of a building indoor positioning process based on BIM (Building Information Model) cross-domain image retrieval provided by an embodiment of the present invention, Figure 2 A schematic diagram of the implementation process of building indoor positioning based on BIM cross-domain image retrieval provided by an embodiment of the present invention is shown in FIG. Figure 1 and Figure 2 As shown, the present invention provides a building indoor positioning method based on BIM cross-domain image retrieval, comprising:

[0045] Step 100: construct a BIM stereo model, set a virtual camera and several positioning reference points, and use the virtual camera to collect video based on the BIM stereo model and the positioning reference points to obtain a BIM model rendering video;

[0046] Step 20: Based on the VideoCapture function of the OpenCV platform, the BIM model rendering video is sampled according to a preset frame interval to obtain a BIM model rendering image set;

[0047] Step 300: Acquire a rendering image position data set according to a BIM model rendering image set and a BIM model rendering video;

[0048] Step 400: Fusing the BIM model rendering image set and the rendering image location data set to obtain a BIM location image set;

[0049] Step 500: Convert the BIM location image set into an indoor image set using a pre-trained CycleGAN style transfer model;

[0050] Step 600: Based on MobileNetV3 small, use the global average pooling layer and the previous network layer to extract features from the indoor image set to obtain a basic feature data set;

[0051] Step 700: construct a kd tree according to the basic feature data set, and set the partitioning dimension and partitioning point;

[0052] Step 800: Acquire features of the indoor image to be retrieved, and obtain features to be retrieved;

[0053] Step 900: Based on the partitioning dimension and the partitioning point, the subtree of the kd tree is traversed according to the features to be retrieved to obtain the best matching BIM location image, and the location information of the best matching BIM location image is extracted to obtain the target location data.

[0054] Preferably, a BIM stereo model is constructed, a virtual camera and a number of positioning reference points are set, and video is captured using the virtual camera based on the BIM stereo model and the positioning reference points to obtain a BIM model rendering video, including:

[0055] Load the BIM solid model into the pre-designed Revit plug-in and set the shading parameters, consistent color parameters and realistic parameters for the BIM solid model;

[0056] Set positioning reference points and sampling paths according to user needs;

[0057] Based on the Walk-through mode, a virtual camera is used to record the interior of the BIM three-dimensional model along the sampling path to obtain a BIM model rendering video.

[0058] Preferably, obtaining a rendered image position dataset according to a BIM model rendered image set and a BIM model rendered video includes:

[0059] Obtain the camera position and direction vector attributes corresponding to each image in the BIM model rendering image set according to the BIM model rendering video;

[0060] Convert the direction vector attribute into radians to obtain the angle in radians;

[0061] The camera position and the arc angle are matched and combined to obtain the position information;

[0062] All position information is integrated to obtain a rendered image position dataset.

[0063] Specifically, the Revit-based location information collection plug-in includes three parts, namely the main implementation code part, the interface foreground, and the interface background.

[0064] The implementation code mainly obtains the currently active view as the 3D view of the BIM model according to the ActiveView property in the Document property of the Revit Document class. The direction vector property of the 3D view of the BIM model is obtained according to the GetSavedOrientation method in the View3D method of the View3D class, and the current frame information of the virtual camera is recorded at the same time. The camera coordinates in the 3D view are obtained through the EyePosition property in the ViewOrientation3D property of the ViewOrientation3D class, and the 3D coordinate information of the geographic location is obtained.

[0065] Get the virtual camera direction through the ForwardDirection property. According to the positive X direction of 0 degrees, the AngleTo method in the XYZ method in the XYZ class is used to get the rotation angle of the current virtual camera direction in radians, and then the radians are converted into angle values ​​to get the geographic location angle information. α represents the return value obtained by using the XYZ.BasisX.AngleTo(vector) code, that is, the angle between the virtual camera direction and the positive X axis is obtained through the AngleTo method.

[0066] Preferably, the CycleGAN style transfer model includes: a generator, a discriminator and a FID score verification module; the generator includes: a CBAM attention mechanism module; the architecture of the discriminator is: a PatchGAN structure.

[0067] Preferably, the resolution of the images in the BIM model rendering image set is 256×256 pixels.

[0068] Preferably, the plug-ins used in the Revit platform include: Revit Lookup and Add-inManager.

[0069] Preferably, a building indoor positioning system based on BIM cross-domain image retrieval comprises: a data acquisition module, a style transfer module, a retrieval and positioning module, and a visualization interface module;

[0070] The data acquisition module is used to obtain the BIM location image set; the style transfer module is used to perform style conversion on the BIM location image set to obtain the indoor image set; the retrieval and positioning module is used to match the indoor images to be retrieved based on the indoor image set using the kd tree to obtain the target location data; the visualization interface module is used to display the offline database construction and online retrieval content.

[0071] Specifically, the plug-in based on Revit collecting location information was developed on the DELLT3600 graphics workstation. The computer system environment is Windows 10, the development environment is Visual Studio 2022, the reference document is REVIT 2022 SDK, and the Revit Lookup and Add-in Manager plug-ins are used.

[0072] Preferably, the location information acquisition process includes: obtaining the three-dimensional view direction of the BIM model and recording the camera frame information. The geographic location coordinates and angle are obtained through the camera position and orientation. The angle change between frames is calculated and saved as XML data to record the rendered image location information. Assuming that positive X is the 0 degree direction (XYZ and BIM models correspond to each other, when adding a location tag plug-in for Revit secondary development, the XYZ axis and the origin correspond one-to-one to the building BIM model), the rotation angle of the current virtual camera orientation is obtained in radians through the Angle To method of the XYZ class, and then the radians are converted into angle values ​​to obtain the geographic location angle information. Then, the following formula exists:

[0073]

[0074] Among them, α represents the return value obtained by using the XYZ.BasisX.AngleTo(vector) code, that is, the angle between the virtual camera direction and the positive X-axis is obtained by the AngleTo method.

[0075] refer to Figure 3 , the training framework of CycleGAN is improved through the structure of generator and discriminator to realize the style transfer from BIM model rendering image to indoor shooting image; the CBAM attention mechanism module is added to the residual block of CycleGAN generator, and after the output of the convolution layer, the channel attention weighting is performed first, and then the spatial attention weighting is performed, and finally the enhanced feature representation is obtained; the discriminator adopts the PatchGAN structure, and the spectral normalization layer is added after the convolution layer of the discriminant network; the FID score verification module is constructed and merged into the training framework. The training samples of BIM model rendering image and indoor shooting image are denoted as a and b respectively; the generator G AB The generator G is responsible for converting the BIM model rendering image into a synthetic image in the style of an indoor shooting image. BA The opposite process is performed, and the discriminator D B It is used to distinguish between the synthetic images of the indoor shooting style generated by the generator and the indoor shooting images. The cycle consistency process is that the generator takes the BIM model rendering image as input, generates the synthetic image of the indoor shooting style, and inputs the generator G AB Generate the reconstructed image G BA (G AB (a)). Figure 3 In the , FID is an indicator used to measure the quality and diversity of the indoor image style synthesis images generated by the generative model. The difference between the generated image and the indoor image is calculated by comparing the feature distribution of the generated image and the indoor image. The higher the similarity, the lower the FID score. When the data sets are exactly the same, the FID score is 0. The calculation formula of the FID score is:

[0076] FID=||μ r -μ g || 2 +Tr(∑r+∑g-2(∑r∑g) 1 / 2 )

[0077] Where r and g represent the indoor image and the synthetic image of the indoor image style, respectively, and μ r and μ g Denote the mean values ​​of the feature vectors of the two images, while ∑r and ∑g denote the covariance matrices of the two images. The trace is denoted by Tr.

[0078] InceptionV3 is a pre-trained feature extractor that can extract feature vectors from samples of indoor image datasets and synthetic image datasets in the style of indoor image datasets. Lipschitz continuity is controlled by the maximum singular value, so that the parameter matrix can generate diverse images. In terms of training stability, the discriminator network of CycleGAN consists of multiple convolutional layers and activation layers, which is equivalent to a composite function. In order for a composite function to satisfy 1-Lipschitz continuity, each function that makes up the composite function needs to satisfy 1-Lipschitz continuity. Since the Leaky ReLU activation function naturally satisfies 1-Lipschitz continuity, it is only necessary to ensure that the convolutional layer satisfies 1-Lipschitz continuity, and the discriminator network of CycleGAN satisfies 1-Lipschitz continuity. For the spectral norm of matrix A, it is defined as follows:

[0079]

[0080] In the formula, h represents the input, σ(A) represents the maximum singular value of the matrix A. σ(W) represents the spectral norm of the convolutional layer parameter matrix, W SN W represents the convolutional layer parameter matrix after SN processing. SN The calculation formula is:

[0081] W SN =W / σ(A)

[0082] The maximum singular value of the convolutional layer using the CycleGAN-SN algorithm is constant at 1, and the input and output characteristics of the entire discriminant network also meet the 1-Lipschitz condition.

[0083] refer to Figure 4 , image feature extraction based on MobileNetV3 small, image feature retrieval based on kd tree, mapping location information, and the steps of realizing cross-domain image retrieval and positioning include: using MobileNetV3small as the basis for extracting image features, using the global average pooling layer and the previous network layer for feature extraction; constructing a kd tree, selecting the partitioning dimension and partitioning point, and cyclically selecting the dimension in the order of the feature value; starting the query from the root node of the kd tree, comparing the feature value of the indoor image to be retrieved with the median value of the node, selecting the corresponding subtree for traversal until reaching the leaf node; using the BIM model to render the image to obtain the corresponding location information and the location of the BIM model floor plan, and using the frame value correspondence relationship based on the Revit acquisition location information to correspond and store the acquired rendered image and location information.

[0084] Furthermore, the specific steps of the position positioning method based on BIM rendered images are as follows: obtain the corresponding BIM model rendering image before CycleGAN style transfer, use Python's replace function to remove the _translated suffix in the image name after style transfer through CycleGAN to match the BIM model rendering image, use os.listdir to obtain the original image file name, combine the location information of the BIM model rendering image, obtain the three-dimensional position, angle and floor plan address through regular expressions and XML parsing, combine the indoor captured images with the BIM model, and accurately display the user's position in the model.

[0085] Specifically, the building indoor positioning system based on BIM cross-domain image retrieval includes: a data acquisition module, which collects BIM model rendering images and corresponding location information, specifically including rendering image data acquisition and displaying BIM data location information; a style transfer module, which trains the improved CycleGAN to transfer the style of the collected BIM model rendering images to a synthetic image of the indoor shooting style; a retrieval and positioning module, which retrieves the generated image most similar to the image to be retrieved and retrieves the BIM model rendering image before style transfer, and locates according to the label information; a visualization interface module, which provides users with a graphical interface of the entire system for intuitive operation, and develops a graphical interface based on the Tkinter library to display offline database building and online retrieval content.

[0086] refer to Figure 5 ,The specific implementation process of the building indoor positioning system based on BIM cross-domain image retrieval is as follows:

[0087] The building indoor positioning system based on BIM cross-domain image retrieval includes offline database construction stage and online retrieval stage:

[0088] The offline database construction stage includes data collection module and style transfer module. The data collection module includes uploading shooting videos, BIM data collection function, and displaying BIM data location information.

[0089] Define the select_file function, use the filedialog.askopenfilename function to open the file dialog box and return the selected video file path to complete the video shooting and uploading;

[0090] Define the select_bimfile function, use the subprocess.Popen function to open the Revit software, and collect data based on the Revit plug-in developed in this article to complete the collection of BIM data;

[0091] Define the get_coordinates_and_angles_from_xml function, use ET.parse and getroot functions to parse the XML file and root element saved based on the Revit plug-in, traverse the carmeaInfo element, use the get function to obtain the three-dimensional position information and angle information, and complete the display of BIM data location information.

[0092] The style transfer module includes setting the video frame capture image, displaying the number of images, saving location, selecting the style transfer model, training hyperparameter settings, style transfer button, rounds in the training process, loss function, iterative training time, real-time FID, displaying real-time style transfer images, evaluation indicators, etc.

[0093] Set the video frame capture image, display image number, and save location: define the capture_frames function, set the capture image interval frame by setting the frame_interval value, and print out save_frames to display the image number. Define save_path and use the os.path.join function to construct a complete save path;

[0094] Select the style transfer model and set the training hyperparameters: define the change_config function, use the subprocess.Popen function to open the CycleGAN config file, modify the model and loss function, modify the training preprocessing in train_pipeline, modify the optimizer in optimizer, modify the starting point of the second-stage learning rate in lr_config, modify the total number of iterations in total_iters, etc.

[0095] Style transfer button, rounds in the training process, loss function, iterative training time, real-time FID, display real-time style transfer images: define the run_translated function, use the subprocess.run function to run the slurm_train script. Define the extract_numbers function, use the re.search function to extract the epoch, loss, time, and FID keyword values, and add them to the display list. Define the get_image function, use the os.listdir function to list all saved files and filter out image files, use the max function to sort to get the latest round of style transfer images, and use the time.sleep function to update in real time;

[0096] Define calculate_fid, calculate_psrn, calculate_ssim, and calculate_css functions for evaluation.

[0097] Figure 5 In the MobileNetV3 architecture, in order to meet the needs of different scenarios, the model is divided into two versions: large and small. The small version is designed to be more lightweight, with fewer parameters and faster computing speed. In view of the functions required to be implemented in this embodiment, MobileNetV3small is selected as the basis for extracting image features, and the global average pooling layer and its previous network layer are used for feature extraction. The specific method is to truncate the global average pooling layer of MobileNetV3 small to retain its output results, and obtain a three-dimensional tensor with a height of 1, a width of 1, and a depth of 576 as the feature vector of the image. Through the flattening operation, the three-dimensional tensor feature of 1×1×576 is flattened into a one-dimensional feature vector of 576. By flattening the image feature vector into a one-dimensional array, similarity calculation and retrieval operations are performed more efficiently to achieve the required functions. Table 1 is the detailed information of the MobileNetV3 small framework.

[0098] Table 1

[0099]

[0100] Specifically, this embodiment uses MobileNetV3 small to extract image features before the global pooling layer from the indoor style synthetic images in the offline dataset collection phase, and stores them as the image feature database in the offline dataset. In the online image retrieval and positioning phase, image features of the indoor images to be retrieved are extracted. The extraction of these features is intended to quantify the similarities and differences between images and provide similarity calculation parameters for subsequent retrieval and positioning.

[0101] refer to Figure 6 , based on the Tkinter library, a graphical interface is developed to display the offline database content. Figure 6 As shown in the figure, the left side of the offline database building interface is from top to bottom: style transfer button, upload video, BIM data collection function, style transfer model, set video frame, number of images, save location, and display BIM data location information. The right side is hyperparameter setting, round, loss function, iteration time, FID, save location, detection index and result. The offline database building stage includes data collection module and style transfer module. The data collection module includes uploading video, BIM data collection function, and displaying BIM data location information. The data collection module collects indoor images and BIM model rendering images and corresponding location information. Specifically, it includes uploading video, BIM data collection function, and displaying BIM data location information. The functions of each button are limited to: upload actual video button: open the file dialog box and return the selected video file path; perform BIM extraction graphic button: open Revit software for BIM rendering image collection; display BIM data location information button: obtain the three-dimensional location information and angle information of the selected image and add it to the display list.

[0102] Specifically, the style transfer module trains the improved CycleGAN and transfers the style of the collected BIM model rendering image to a synthetic image of the indoor shooting style. Set the video frame capture image button: set the capture image interval frame, the video will capture the image according to the preset interval frame to form a rendered image dataset; Save location button: select the save path of the captured rendered image dataset; Style transfer model button: select the model used for style transfer; Training hyperparameter button: open the config file of the selected model, you can modify the model's optimizer, learning rate, number of iterations and other parameters, and display the loss function, iteration time and real-time FID obtained by the current parameter configuration. Style transfer button: run the set script to perform style transfer of BIM rendered images, and obtain a synthetic image after the uploaded image style transfer.

[0103] Furthermore, the online retrieval stage mainly consists of the retrieval positioning module, including uploading the image to be retrieved, the retrieval method, the retrieval button, the retrieval time, displaying the best matching image, the original image before style transfer, displaying the three-dimensional coordinate and angle position information, and the BIM model floor plan;

[0104] Define the select_file function, use the filedialog.askopenfilename function to open the file dialog box and return the path of the selected image file to be retrieved;

[0105] Define functions such as save_features_LBP, save_features_SIFT, save_features_RGB, etc. to perform image feature retrieval based on kd tree; define run_retrieval function, use subprocess.run function to run slurm_retrieval script, use time.time function to calculate retrieval time, use os.path.basename function to get the base name and add the best matching image to the display column;

[0106] Define the find_original function, use the replace function to remove the _translated suffix after the CycleGAN style transfer, use the os.listdir function to get the original image file name, retrieve the original image before style transfer and add it to the display column;

[0107] Define the extract_info function, use the re.compile function to match the regular expression attributes, use the ET.parse and getroot functions to parse the XML file and root element saved based on the Revit plug-in, use the get function to obtain the 3D position, angle information, and BIM model floor plan address, search and match the Name keyword, and use the re.match function to extract information and add it to the display column;

[0108] From top to bottom on the left side of the online search interface are the search button, upload the actual image, display the best matching image, the original image before style transfer, the search method, the search time, the use weight, and the right side displays the three-dimensional coordinate and angle position information, and the BIM model floor plan, such as Figure 7 As shown. The online retrieval stage is mainly the retrieval positioning module, including uploading the image to be retrieved, retrieval method, retrieval button, retrieval time, displaying the best matching image, the original image before style transfer, displaying the three-dimensional coordinates and angle position information, BIM model floor plan, etc. Upload image to be retrieved button: open the file dialog box and return the selected image to be retrieved file path; retrieval method button: select the image retrieval model used for retrieval and the trained image retrieval model; retrieval button: run the set script to retrieve indoor images, and obtain the specific location information of the building where the image is located, the retrieval time and the usage weight.

[0109] refer to Figure 8, where the data acquisition module is to obtain the data set based on the BIM model, including obtaining BIM model information, using Revit software and performing secondary development based on Revit's API to obtain BIM model rendering images and location information. In the image style transfer module, the CBAM attention mechanism is added to the generator residual block, the PReLU activation function is introduced in the convolution layer, the label smoothing strategy is used in the discriminator, and spectral normalization is introduced in the convolution layer. The real-time FID module is merged into the training framework to obtain the optimal number of training rounds. The collected data set will improve the CycleGAN network to obtain synthetic images of indoor shooting styles. The retrieval and positioning module is a cross-domain image retrieval and positioning based on the MobileNetV3 small-kd tree, image feature extraction based on MobileNetV3small, feature retrieval based on the kd tree, the image with the highest matching degree is obtained according to the cosine similarity, and the location information of the image is obtained to achieve positioning.

[0110] Specifically, the building indoor positioning system for BIM cross-domain image retrieval developed in this embodiment is composed of a DELL T3660 graphics workstation server, a Jestson Nano embedded system, a CSI camera, a Bull mobile power supply, a personal computer, etc. The visualization interface module is deployed on a personal computer, a secure shell protocol (Secure Shell, SSH) client is installed on the personal computer, and an SSH server is installed on the data acquisition module, the DELL T3660 graphics workstation server, and the Jestson Nano embedded system. The data acquisition module, the style transfer module, and the retrieval and positioning module are connected through SSH instructions and SSH-based secure copy (Secure copy, SCP) instructions. SSH is an encrypted network protocol that replaces Tenlnet. It prevents information leakage during transmission and is used to securely access remote computers on networks with leakage risks. It consists of a transport layer protocol, a user authentication protocol, and a connection protocol. Use the ssh start command in the terminal to start the SSH service. Compared with the password login with the username, IP address, and password, the key login does not require a password every time and is more difficult to be stolen. The specific method is to use the ssh-keygen command to generate a pair of keys, use ssh-copy-id to copy the public key id_ras.pub to the server, and use the id_rsa private key to connect to the server. SCP uses SSH to complete the network transmission protocol based on the BSDRCP protocol, ensuring the authenticity and confidentiality of file transmission.

[0111] Furthermore, the data acquisition in the data acquisition module uses a tripod and a smartphone to shoot indoor video, and collects BIM model rendering images based on Revit. The SSH command received from the visualization interface module uploads both to the DELLT3660 graphics workstation server, uploads the location information based on Revit to the JestsonNano embedded system, and returns the collection results to the visualization interface module through the SCP command.

[0112] Preferably, the DELLT3660 graphics workstation server in the style transfer module receives the SSH instruction of the visualization interface module, sets the video frame capture image, the number of images, the save location, selects the style transfer model, sets the training hyperparameters, etc. During the style transfer training, the training results such as the round, loss function, iterative training time, real-time FID, style transfer image, and evaluation index in the training process are transmitted back to the visualization interface module through SCP, and the style transfer image is uploaded to the retrieval and positioning module.

[0113] Specifically, the DELLT3660 graphics workstation server in the retrieval and positioning module receives the SSH command of the visualization interface module and extracts the style migration image features through the extraction method. The extracted image feature data set is uploaded to the JestsonNano embedded system, which receives the SSH command of the visualization interface module, shoots and uploads the image to be retrieved, selects the retrieval method, and uses the image feature retrieval and position positioning method to combine the uploaded image feature data set and the BIM model location information for retrieval and positioning, and transmits the retrieval time, the best matching image, the original image before style migration, the three-dimensional coordinates and angle position information, and the BIM model floor plan back to the visualization interface module through scp.

[0114] Preferably, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any step of this embodiment is implemented.

[0115] The beneficial effects of the present invention are as follows:

[0116] The present invention realizes the transfer of image style through the CycleGAN style transfer model, solves the problem of low recognition accuracy caused by large style differences in the existing scheme, and improves the recognition accuracy and reliability of the model; by collecting BIM model rendering videos, the construction of a large database is realized, the positioning accuracy and efficiency of the model are improved, and the computing power requirement is reduced.

[0117] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0118] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A building indoor positioning method based on BIM cross-domain image retrieval, characterized in that: include: Constructing a BIM stereo model, setting a virtual camera and a number of positioning reference points, and using the virtual camera to perform video acquisition based on the BIM stereo model and the positioning reference points to obtain a BIM model rendering video; Based on the VideoCapture function, the BIM model rendering video is sampled according to a preset frame number interval to obtain a BIM model rendering image set; Acquire a rendering image position data set according to the BIM model rendering image set and the BIM model rendering video; The BIM model rendering image set and the rendering image location data set are merged to obtain a BIM location image set; Using a pre-trained CycleGAN style transfer model to convert the BIM location image set into an indoor image set; Based on MobileNetV3 small, the global average pooling layer and the front network layer are used to extract features of the indoor image set to obtain a basic feature data set; Constructing a kd tree according to the basic feature data set, and setting the partitioning dimension and partitioning point; Acquire features of the indoor image to be retrieved to obtain features to be retrieved; Based on the division dimension and division point, the subtree of the kd tree is traversed according to the feature to be retrieved to obtain the best matching BIM location image, and the location information of the best matching BIM location image is extracted to obtain the target location data.

2. The building indoor positioning method based on BIM cross-domain image retrieval according to claim 1 is characterized in that: Constructing a BIM stereo model, setting a virtual camera and a plurality of positioning reference points, and using the virtual camera to perform video acquisition based on the BIM stereo model and the positioning reference points to obtain a BIM model rendering video, including: Loading the BIM three-dimensional model into a pre-designed Revit plug-in, and setting shading parameters, consistent color parameters, and real parameters for the BIM three-dimensional model; Setting the positioning reference point and sampling path according to user requirements; Based on the Walk through mode, the virtual camera is used to record the interior of the BIM three-dimensional model along the sampling path to obtain the BIM model rendering video.

3. The building indoor positioning method based on BIM cross-domain image retrieval according to claim 1 is characterized in that: Acquiring a rendering image position data set according to the BIM model rendering image set and the BIM model rendering video, including: Obtaining camera position and direction vector attributes corresponding to each image in the BIM model rendering image set according to the BIM model rendering video; Convert the direction vector attribute into radians to obtain an angle in radians; Matching and combining the camera position and the arc angle to obtain position information; All the position information is integrated to obtain the rendered image position data set.

4. The building indoor positioning method based on BIM cross-domain image retrieval according to claim 1 is characterized in that: The CycleGAN style transfer model includes: a generator, a discriminator and a FID score verification module; the generator includes: a CBAM attention mechanism module; the architecture of the discriminator is: a PatchGAN structure.

5. The building indoor positioning method based on BIM cross-domain image retrieval according to claim 1 is characterized in that: The resolution of the images in the BIM model rendering image set is 256×256 pixels.

6. The building indoor positioning method based on BIM cross-domain image retrieval according to claim 2 is characterized in that: The plug-ins used in the Revit platform include: Revit Lookup and Add-in Manager.

7. A building indoor positioning system based on BIM cross-domain image retrieval, characterized in that: include: Data collection module, style transfer module, retrieval and positioning module, and visualization interface module; The data acquisition module is used to obtain a BIM location image set; The style transfer module is used to perform style conversion on the BIM location image set to obtain the indoor image set; The retrieval and positioning module is used to match the indoor shooting images to be retrieved using the kd tree according to the indoor image set to obtain the target location data; the visual interface module is used to display the offline database building and online retrieval content.

Images