Image-based building model data search method, device, equipment and medium

Abstract

The application discloses an image-based building model data search method, device, equipment and medium, first obtaining an initial local image of a building to be searched and then obtaining current multi-modal features thereof; current building positioning data is obtained, and a target building model database is obtained in a building model data set database; a building sub-model corresponding to current multi-modal features and sub-model multi-modal features in the target building model database having the maximum feature similarity with the current multi-modal features is obtained as a target building sub-model, and a target BIM model corresponding thereto is obtained. The embodiment of the application can first extract current multi-modal features from the initial local image of the building to be searched based on a multi-modal model, and then obtain a target building sub-model having the maximum feature similarity with the current multi-modal features in the target building model database determined based on the current building positioning data, so that the calculation process of searching for a three-dimensional model according to an image is simplified, and the efficiency of the image search result is improved.

Description

Technical Field

[0001] This invention relates to the field of smart building technology, and in particular to image-based building model data search methods, devices, equipment, and media. Background Technology

[0002] With the increasingly widespread application of smart building technologies in the construction field, 3D model data of buildings can be utilized from initial construction to subsequent maintenance. For example, in the initial construction phase, BIM technology (Building Information Modeling) can be used to create 3D models and guide the actual construction process. Of course, throughout the building's operation and its entire lifecycle, the BIM model can be continuously used to acquire and apply relevant data.

[0003] Currently, image-based retrieval technology has been widely applied in other technological fields. For example, it can be used in search engines to quickly obtain image search results related to the initially uploaded image. However, in the field of architecture, the method of retrieving the corresponding architectural model from a photograph taken by a user using a camera is not yet widely used. Moreover, the drawings corresponding to the 3D model data of buildings are scattered, and even after knowing the actual location of the building structure to be viewed, there are still problems with low efficiency and incomplete image viewing. Although the patent application number CN202311755642.8 describes a method that can process the image to be retrieved using a trained image-based 3D model retrieval network to obtain the retrieved 3D model from a 3D model database. When processing the image-based 3D model retrieval network after training based on the image to be queried, after obtaining the features of the image to be queried, it is necessary to input the features of each 3D model in the 3D model database into the attention mechanism network and output the weighted 3D model feature vector. This method makes the calculation process of obtaining the 3D model corresponding to the image to be queried very complicated, resulting in low efficiency of image retrieval results and also making it impossible to quickly obtain design drawing data related to the 3D model. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for searching architectural model data based on images. It aims to solve the problem that in the prior art, retrieving 3D models from a 3D model database based on a query image involves inputting the features of the query image and each 3D model in the database into an attention mechanism network to output a weighted 3D model feature vector. This method results in a very complex calculation process for obtaining the 3D model corresponding to the query image, leading to low efficiency in image retrieval and an inability to quickly obtain design drawing data related to the 3D model.

[0005] In a first aspect, embodiments of the present invention provide an image-based building model data search method, comprising:

[0006] In response to an image search command sent by the requesting end, an initial partial image of the building to be searched corresponding to the image search command is obtained; wherein, the partial image of the building to be searched is a real-scene photograph, a hand-drawn sketch, a modified drawing, or a two-dimensional structural drawing of the building;

[0007] The current multimodal features corresponding to the initial local image of the building to be retrieved are obtained by a pre-trained multimodal model; wherein the current multimodal features are obtained by fusing at least the building image features and the 3D model features;

[0008] Obtain the current building location data corresponding to the initial partial image of the building to be retrieved, and obtain the target building model database corresponding to the current building location data from the building model dataset database;

[0009] Obtain the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database, and obtain the building sub-model corresponding to the sub-model multimodal feature with the maximum feature similarity to the current multimodal feature, as the target building sub-model;

[0010] Obtain the target BIM model and target 2D drawing file corresponding to the target building sub-model, and send the target BIM model and target 2D drawing file to the requesting end.

[0011] Secondly, embodiments of the present invention also provide an image-based building model data search device, comprising:

[0012] The building image acquisition unit is used to acquire an initial partial image of the building to be retrieved in response to an image search command sent by the requesting end; wherein the partial image of the building to be retrieved is a real-scene photograph, a hand-drawn sketch, a modified drawing, or a two-dimensional structural drawing of the building;

[0013] A multimodal feature acquisition unit is used to acquire current multimodal features corresponding to the initial local image of the building to be retrieved through a pre-trained multimodal model; wherein, the current multimodal features are obtained by fusing at least building image features and 3D model features;

[0014] The target building model database acquisition unit is used to acquire the current building location data corresponding to the initial partial image of the building to be retrieved, and to acquire the target building model database corresponding to the current building location data from the building model dataset database;

[0015] The target building sub-model acquisition unit is used to acquire the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database, and to acquire the building sub-model corresponding to the sub-model multimodal feature with the maximum feature similarity to the current multimodal feature, as the target building sub-model.

[0016] The target BIM model sending unit is used to obtain the target BIM model and target two-dimensional drawing file corresponding to the target building sub-model, and send the target BIM model and the target two-dimensional drawing file to the requesting end.

[0017] Thirdly, embodiments of the present invention also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect above.

[0018] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, can implement the method described in the first aspect above.

[0019] This invention provides an image-based building model data search method, apparatus, device, and medium. The method includes: responding to an image search command sent by a requesting end, acquiring an initial partial image of a building to be searched corresponding to the image search command; acquiring current multimodal features corresponding to the initial partial image of the building to be searched through a pre-trained multimodal model; wherein the current multimodal features are obtained by fusing at least building image features and 3D model features; acquiring current building location data corresponding to the initial partial image of the building to be searched, and acquiring a target building model database corresponding to the current building location data from a building model dataset database; acquiring the feature similarity between the current multimodal features and the sub-model multimodal features of each building sub-model in the target building model database, and acquiring the building sub-model corresponding to the sub-model multimodal feature with the maximum feature similarity to the current multimodal features, as the target building sub-model; acquiring the target BIM model corresponding to the target building sub-model, and sending the target BIM model to the requesting end. The embodiments of the present invention can first extract the current multimodal features from the initial partial image of the building to be retrieved based on the multimodal model, and then obtain the target building sub-model with the maximum feature similarity to the current multimodal features from the target building model database determined based on the current building location data. This simplifies the calculation process of retrieving the three-dimensional model from the image, not only improving the efficiency of image retrieval results, but also allowing simultaneous viewing of the design drawing data related to the three-dimensional model. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of the image-based building model data search method provided in an embodiment of the present invention.

[0022] Figure 2 A flowchart illustrating the image-based building model data search method provided in an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a sub-process of the image-based building model data search method provided in an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of another sub-process of the image-based building model data search method provided in an embodiment of the present invention;

[0025] Figure 5This is a schematic diagram of another sub-process of the image-based building model data search method provided in an embodiment of the present invention;

[0026] Figure 6 A schematic block diagram of an image-based building model data search device provided in an embodiment of the present invention;

[0027] Figure 7 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0030] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0031] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] Please also refer to Figure 1 and Figure 2 ,in Figure 1 This is a schematic diagram illustrating a scenario of the image-based building model data search method according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating the image-based architectural model data search method provided in an embodiment of the present invention. Figure 1 As shown, the image-based building model data search method provided in this embodiment of the invention is applied to server 10, and the requesting end 20 is connected to server 10. Figure 2 As shown, the method includes the following steps S110-S150.

[0033] S110. In response to the image search command sent by the requesting end, obtain the initial partial image of the building to be searched corresponding to the image search command.

[0034] The partial images of the building to be retrieved can be actual photos taken on-site, hand-drawn on-site, modified drawings, or two-dimensional structural drawings of the building.

[0035] In this embodiment, the technical solution is described with the server as the execution entity. A building model dataset database is deployed on the server, including building model databases corresponding to multiple buildings. When the requesting end (such as a desktop computer, laptop, tablet, or smartphone used by the user) establishes a communication connection with the server, it can send initial partial images of the building to be searched, obtained directly or indirectly, to the server. For example, if the requesting end has its own image acquisition device, when the user walks inside a building (which can be under construction or already completed and in use), they can take a picture of a specific building structure (such as a ceiling, load-bearing beam, or load-bearing column) to obtain an initial partial image of the building to be searched. This initial partial image can be used as the initial search condition for subsequent building model data searches. Of course, the aforementioned initial partial image of the building to be searched can be obtained not only by the image acquisition device of the requesting end capturing images of the actual building, but also by a hand-drawn sketch drawn by the user directly on paper or on the interface of the requesting end, a modified drawing selected by the user from multiple building drawing files, or a two-dimensional structural drawing of the building.

[0036] S120. Obtain the current multimodal features corresponding to the initial local image of the building to be retrieved through a pre-trained multimodal model.

[0037] The current multimodal features are obtained by fusing architectural image features and 3D model features.

[0038] In this embodiment, after the server obtains the initial partial image of the building to be retrieved, it can extract and fuse multimodal features from the initial partial image based on a pre-trained multimodal model to obtain the corresponding current multimodal features. At this time, the server only needs to process the initial partial image of the building to be retrieved in conjunction with the multimodal model, without needing to calculate the fusion features of each building sub-model and the current multimodal features in the local building model dataset database of the server during this round of retrieval. This effectively reduces the amount of data processing in this round of retrieval.

[0039] In one embodiment, such as Figure 3 As shown, step S120 includes:

[0040] S121. The initial local image of the building to be retrieved is used to extract features through the ViT encoder in the multimodal model to obtain the current building image features;

[0041] S122. Obtain the current building geometric features of the initial building local image to be retrieved through the random sampling model in the multimodal model, and obtain the current building topological features of the initial building local image to be retrieved through the Reeb graph model in the multimodal model; wherein, the current building geometric features and the current building topological features constitute the three-dimensional model features corresponding to the initial building local image to be retrieved.

[0042] S123. The current building image features, the current building geometric features, and the current building topological features are fused using the cross-attention mechanism in the multimodal model to obtain the current multimodal features.

[0043] In this embodiment, a ViT encoder can be used to extract image features from the initial partial image of the building to be retrieved. ViT stands for Vision Transformer (a visual Transformer model that directly captures the global semantic and structural features of an image through a self-attention mechanism). The ViT encoder converts the initial partial image of the building to be retrieved into sequential data, thereby obtaining a multi-dimensional vector form of the current building image features. More specifically, these current building image features include at least global semantic features (such as architectural style, material, and spatial layout) and local detail features (such as door and window shapes, decorative patterns, etc.).

[0044] When extracting 3D model features from the initial partial image of the building to be retrieved, it is necessary to combine a random sampling model and a Reeb graph model.

[0045] When obtaining the current building geometry features of the initial local image of the building to be retrieved through the random sampling model in the multimodal model, the random sampling model calculates the vertices identified in the initial local image of the building to be retrieved, and calculates the statistical distribution of the Euclidean distance between the vertices, thereby constructing a global shape histogram, and extracting the local curvature features to describe the surface details. For example, the curvature distribution of the interior wall surface in the initial local image of the building to be retrieved is used as a distinguishing feature.

[0046] When obtaining the current building topology features of the initial building local image to be retrieved through the Reeb graph model in the multimodal model, the three-dimensional structure can be abstracted into a graph model through the Reeb graph model. The main processing steps include: A1) converting the building local image to be retrieved into a triangular mesh model through Poisson reconstruction; A2) applying Gaussian filtering to the triangular mesh model to remove sampling noise and obtain the filtered triangular mesh model; A3) using the Gaussian curvature function as a scalar function to perform critical point detection (such as minimum points, maximum points, and saddle points) and feature enhancement (such as removing low-persistence critical points based on persistent homology, and retaining the critical points that affect the model topology) on the filtered triangular mesh model. Key points that make significant contributions, such as setting a threshold to filter out small extreme points caused by noise, to obtain significant critical point screening results; A4) Threshold segmentation of the scalar function corresponding to the significant critical point screening results to generate a series of isosurfaces, and then using a graph traversal algorithm to identify the connected regions in each isosurface, thereby obtaining the level set corresponding to the significant critical point screening results; A5) Map each connected component corresponding to the level set to a node in the Reeb graph. If two connected components are adjacent in the direction of change of the scalar function (i.e., there is an inclusion relationship), add an edge in the Reeb graph. After completing the above processing, the current building topology features of the initial local image of the building to be retrieved can be obtained.

[0047] After extracting the multidimensional features of the current building image, geometric features, and topological features corresponding to the initial local image of the building to be retrieved, these three features can be fused using a cross-attention mechanism to obtain the current multimodal features. The cross-attention mechanism establishes dependencies between multiple different sequences (or modalities), enabling interaction and fusion of features from different sources. Its core processing involves mapping multiple different sequences (or modalities) to the same dimension through linear projection. Therefore, the current multimodal features of the initial local image of the building to be retrieved can be obtained quickly using this method.

[0048] S130. Obtain the current building location data corresponding to the initial partial image of the building to be retrieved, and obtain the target building model database corresponding to the current building location data from the building model dataset database.

[0049] In this embodiment, to reduce the amount of subsequent retrieval data processing, the current building location data corresponding to the initial partial image of the building to be retrieved can be further obtained. This current building location data is then used as a retrieval condition to retrieve the target building model database corresponding to the current building location data from the local building model dataset database on the server. For example, if the specific building where the user is currently located can be determined through the current building location data, the target building model data corresponding to that specific building can be directly retrieved from the building model dataset database without needing to perform subsequent image retrieval processing across the entire building model dataset database.

[0050] In one embodiment, as a first embodiment of step S130, step S130 includes:

[0051] If the current building location data corresponding to the initial partial image of the building to be retrieved includes the current building name and the current building number, then the current building name and the current building number are used as the current search criteria to retrieve the target building model database from the building model dataset database.

[0052] In this embodiment, if the current building location data corresponding to the initial partial image of the building to be retrieved includes the current building name and the current building number, it indicates that the current building location data is extremely accurate. At this time, the current building name and the current building number are used to form the current search conditions, and the target building model database is retrieved from the building model dataset database using the current building name and the current building number as the current search conditions.

[0053] It is important to note that the building model dataset database includes multiple building model databases, each corresponding to a single building or a cluster of buildings, and each database contains unique building location data. Once the current building name and building number are obtained from the current building location data and used as search criteria, the target building model database can be retrieved from the building model dataset database. Therefore, the above method allows for the rapid retrieval of the target building model database for the user's current location from the building model dataset database.

[0054] In one embodiment, as a second embodiment of step S130, step S130 includes:

[0055] If it is determined that the current building location data corresponding to the initial partial image of the building to be retrieved does not include the current building name and the current building number, then the current network communication information corresponding to the requesting end is obtained;

[0056] Based on the unique identifier data of the current building corresponding to the current network communication information, the target building model database corresponding to the current building location data is obtained from the building model dataset database.

[0057] In this embodiment, if it is determined that the current building location data corresponding to the initial partial image of the building to be retrieved does not include the current building name and the current building number, it indicates that the current building location data is fuzzy location data (e.g., only the current building name is known) or location data not represented by the building name and building number (e.g., latitude and longitude location data or base station communication location data). In this case, the current network communication information corresponding to the communication terminal currently connected to the requesting end, such as a communication base station or wireless LAN, can be obtained. Then, the unique identifier data of the current building corresponding to the deployment location of the communication base station or wireless LAN can be obtained. Finally, the unique identifier data of the current building can be used as the search condition to obtain the target building model database corresponding to the current building location data from the building model dataset database. It can be seen that the target building model database of the current user's building can also be quickly obtained from the building model dataset database through the above method.

[0058] S140. Obtain the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database, and obtain the building sub-model corresponding to the sub-model multimodal feature with the maximum feature similarity to the current multimodal feature, as the target building sub-model.

[0059] In this embodiment, to quickly identify the building sub-model corresponding to the sub-model multimodal feature with the highest feature similarity to the current multimodal feature in the server, each building sub-model in the target building model database and the complete building model dataset database stored in the server has pre-extracted building topology features that integrate building image features, building geometric features, and building topology features. After calculating the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database, the building sub-model corresponding to the sub-model multimodal feature with the highest feature similarity to the current multimodal feature is selected as the target building sub-model. The obtained target building sub-model is the model that the requesting end needs to view, and this target building sub-model can then be fed back to the requesting end.

[0060] In one embodiment, such as Figure 4 As shown, step S140 includes:

[0061] S141. Obtain the current hash code of the current multimodal feature based on the deep cross-modal hashing model;

[0062] S142. Obtain the sub-model hash code corresponding to the sub-model multimodal features of each building sub-model in the target building model database;

[0063] S143. Obtain the Hamming distance between the current hash code and the sub-model hash codes of each building sub-model in the target building model database, and use it as the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database.

[0064] In this embodiment, to improve data retrieval efficiency, the current hash code of the current multimodal feature can be obtained first based on a deep cross-modal hashing model (DCMH model, DCMH stands for Deep Cross-Modal Hashing, which is a cross-modal retrieval model combining deep learning and hashing technology, capable of mapping different modal data to a unified binary hash code space). Then, the sub-model hash codes corresponding to the sub-model multimodal features of each building sub-model in the target building model database are obtained (the sub-model hash codes corresponding to the sub-model multimodal features of each building sub-model are also obtained on the server, not during the current retrieval process). Finally, the Hamming distance between the current hash code and the sub-model hash codes of each building sub-model in the target building model database is obtained, serving as the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database. It is evident that by comparing the Hamming distance between hash codes as described above, the retrieval speed can be effectively improved, and the time required to obtain retrieval results can be reduced.

[0065] In steps S120 to S140, the initial local image of the building to be retrieved is obtained, and the current multimodal features are extracted from it. Then, the target building sub-model is matched from the building model dataset database.

[0066] As another implementation of steps S120-S140, the current multimodal features corresponding to the initial local image of the building to be retrieved can be obtained first through a pre-trained multimodal model; then, the current multimodal features are input into a preset combined model composed of a neural radiation field (NeRF) and a diffusion model, or the current multimodal features are input into a structured 3D generation model to generate a 3D model of the building to be retrieved corresponding to the initial local image of the building to be retrieved; then, the 3D model similarity between the 3D model of the building to be retrieved and each building sub-model in the target building model database is obtained (when calculating the similarity between two 3D models, the vector similarity between the current building topological features of the two 3D models can be calculated), and the building sub-model with the maximum feature similarity to the 3D model of the building to be retrieved is obtained as the target building sub-model.

[0067] S150. Obtain the target BIM model and target 2D drawing file corresponding to the target building sub-model, and send the target BIM model and target 2D drawing file to the requesting end.

[0068] In this embodiment, when the target building sub-model is obtained from the server, in order to improve the result feedback speed, a lightweight target BIM model can be obtained and the target BIM model can be sent to the requesting end.

[0069] Of course, the server can also obtain the target two-dimensional drawing file corresponding to the building sub-model (such as the target CAD file and related information such as detailed drawings and design specifications related to the target CAD file), and send it to the requesting end to open and view it. The target CAD file includes CAD drawings of all disciplines (architecture, structure, water, heating, electricity, etc.) and related information such as detailed drawings and design specifications related to the target CAD file. Users can open it and view it in time, realizing intelligent matching and intelligent display based on images, which can prevent users from missing relevant information.

[0070] In one embodiment, such as Figure 5 As shown, step S150 includes:

[0071] S151. Obtain the current user type corresponding to the requesting end; the current user type includes at least production management personnel, technical personnel, quality management personnel, and construction workers.

[0072] S152. Obtain the unique number of the target BIM model corresponding to the target building sub-model;

[0073] S153. Obtain the original BIM model of the target building model from the target building model database according to the unique number of the target BIM model and the current user type;

[0074] S154. The target original BIM model is subjected to model lightweighting processing through a preset lightweight data conversion model to obtain the target BIM model;

[0075] S155. Based on the current user type, retrieve the target two-dimensional drawing file corresponding to the target building sub-model from the target building model database.

[0076] In this embodiment, the target building sub-model attribute data obtained from the server includes the unique ID of the target BIM model and the current user type. Using the unique ID and current user type as search criteria, the original target BIM model can be retrieved from the target building model database. Then, a lightweight data conversion model can be used to convert the original target BIM model from its original IFC format (IFC stands for Industry Foundation Classes) to a preset format (such as OBJ, where OBJ is a geometry-oriented text format), thereby achieving lightweight data processing. The resulting target BIM model, as lightweight data, can be sent to the requesting end for viewing more quickly.

[0077] The current user types include at least production management personnel, technical personnel, quality management personnel, and construction workers. To better understand the data viewed by each type of personnel, the following explanation is provided in conjunction with Table 1:

[0078] Table 1

[0079]

[0080] As can be seen, by using the above-mentioned method of allowing different types of users to view the corresponding target BIM model and target 2D drawing files, users of all types can quickly obtain the relevant data corresponding to their responsibilities, and it is also convenient for them to understand the relevant data and perform corresponding operations. This avoids the situation of opening the full model data and drawing files of the target BIM model without considering the user type. This reduces the number of models and files that need to be opened, and the speed of opening models and files is improved due to the reduction in the amount of data required.

[0081] In this application, the initial partial image of the building to be searched is used as the initial search condition to retrieve building-related data. However, in specific implementation, keywords, axis positioning, etc. can also be used as search conditions to retrieve building-related data. Specifically, keywords, axis positioning, etc. can be used as search conditions without image processing and image feature extraction. Instead, the matching results can be obtained directly using text features.

[0082] In one embodiment, the method further includes the following step after step S150:

[0083] If a building maintenance request corresponding to the target BIM model is detected by the requesting end, then target maintenance information corresponding to the target BIM model is obtained and the target maintenance information is sent to the requesting end; wherein, the target maintenance information includes at least one of maintenance text information and maintenance video information.

[0084] In this embodiment, after the requesting end receives the target BIM model sent by the server, if the user on the requesting end discovers that the location corresponding to the initially captured partial image of the building to be retrieved needs maintenance but is unsure how to perform the maintenance, they can send a building maintenance request to the server for the target BIM model. The server can preset maintenance information for each building sub-model in the building model dataset database. Furthermore, it can obtain target maintenance information (including at least one of maintenance text information and maintenance video information) corresponding to the target BIM model and send this target maintenance information to the requesting end. The requesting end can then perform timely on-site maintenance on the location corresponding to the initially captured partial image of the building based on the target maintenance information.

[0085] As can be seen, the implementation of this method can first extract the current multimodal features from the initial local image of the building to be retrieved based on the multimodal model, and then obtain the target building sub-model with the maximum feature similarity to the current multimodal features from the target building model database determined based on the current building location data. This simplifies the calculation process of retrieving the 3D model from the image, not only improving the efficiency of image retrieval results, but also allowing simultaneous viewing of the 3D model's related design drawings.

[0086] Figure 6 This is a schematic block diagram of an image-based architectural model data search device provided in an embodiment of the present invention. Figure 6 As shown, corresponding to the above-described image-based building model data search method, the present invention also provides an image-based building model data search apparatus 100. This image-based building model data search apparatus 100 includes a unit for performing the above-described image-based building model data search method. Please refer to... Figure 6 The image-based building model data search device 100 includes: a building image acquisition unit 110, a multimodal feature acquisition unit 120, a target building model database acquisition unit 130, a target building sub-model acquisition unit 140, and a target BIM model sending unit 150.

[0087] The building image acquisition unit 110 is used to acquire an initial partial image of the building to be retrieved corresponding to the image search instruction sent by the requesting end.

[0088] The partial images of the building to be retrieved can be actual photos taken on-site, hand-drawn on-site, modified drawings, or two-dimensional structural drawings of the building.

[0089] In this embodiment, the technical solution is described with the server as the execution entity. A building model dataset database is deployed on the server, including building model databases corresponding to multiple buildings. Once the requesting device (such as a desktop computer, laptop, tablet, or smartphone used by the user) establishes a communication connection with the server, it can send initial partial images of the building to be searched, obtained directly or indirectly, to the server. For example, if the requesting device has its own image acquisition device, the user can take a picture of a building structure (such as a ceiling, load-bearing beam, or load-bearing column) while walking inside a building, obtaining an initial partial image of the building to be searched. This initial partial image can be used as the initial search condition for subsequent building model data searches. Of course, the aforementioned initial partial image can be obtained not only from the image acquisition device of the requesting device capturing images of the actual building, but also from a hand-drawn sketch by the user on paper or the interface of the requesting device, a modified drawing selected by the user from multiple building drawing files, or a two-dimensional structural drawing of the building.

[0090] The multimodal feature acquisition unit 120 is used to acquire the current multimodal features corresponding to the initial local image of the building to be retrieved through a pre-trained multimodal model.

[0091] The current multimodal features are obtained by fusing architectural image features and 3D model features.

[0092] In this embodiment, after the server obtains the initial partial image of the building to be retrieved, it can extract and fuse multimodal features from the initial partial image based on a pre-trained multimodal model to obtain the corresponding current multimodal features. At this time, the server only needs to process the initial partial image of the building to be retrieved in conjunction with the multimodal model, without needing to calculate the fusion features of each building sub-model and the current multimodal features in the local building model dataset database of the server during this round of retrieval. This effectively reduces the amount of data processing in this round of retrieval.

[0093] In one embodiment, the multimodal feature acquisition unit 120 is specifically used for:

[0094] The ViT encoder in the multimodal model is used to extract features from the initial local image of the building to be retrieved, thereby obtaining the features of the current building image;

[0095] The current building geometric features of the initial building local image to be retrieved are obtained through the random sampling model in the multimodal model, and the current building topological features of the initial building local image to be retrieved are obtained through the Reeb graph model in the multimodal model; wherein, the current building geometric features and the current building topological features constitute the three-dimensional model features corresponding to the initial building local image to be retrieved;

[0096] The current building image features, the current building geometric features, and the current building topological features are fused using the cross-attention mechanism in the multimodal model to obtain the current multimodal features.

[0097] In this embodiment, a ViT encoder can be used to extract image features from the initial partial image of the building to be retrieved. ViT stands for Vision Transformer (a visual Transformer model that directly captures the global semantic and structural features of an image through a self-attention mechanism). The ViT encoder converts the initial partial image of the building to be retrieved into sequential data, thereby obtaining a multi-dimensional vector form of the current building image features. More specifically, these current building image features include at least global semantic features (such as architectural style, material, and spatial layout) and local detail features (such as door and window shapes, decorative patterns, etc.).

[0098] When extracting 3D model features from the initial partial image of the building to be retrieved, it is necessary to combine a random sampling model and a Reeb graph model.

[0099] When obtaining the current building geometry features of the initial local image of the building to be retrieved through the random sampling model in the multimodal model, the random sampling model calculates the vertices identified in the initial local image of the building to be retrieved, and calculates the statistical distribution of the Euclidean distance between the vertices, thereby constructing a global shape histogram, and extracting the local curvature features to describe the surface details. For example, the curvature distribution of the interior wall surface in the initial local image of the building to be retrieved is used as a distinguishing feature.

[0100] When obtaining the current building topological features of the initial building local image to be retrieved using the Reeb graph model in the multimodal model, the three-dimensional structure can be abstracted into a graph model. After completing the multidimensional feature extraction of the current building image features, current building geometric features, and current building topological features corresponding to the initial building local image to be retrieved, the above three features can be fused through a cross-attention mechanism to obtain the current multimodal features. The cross-attention mechanism can establish dependencies between multiple different sequences (or modalities), realizing the interaction and fusion of cross-source features. Its core processing procedure is to map multiple different sequences (or modalities) to the same dimension through linear projection. Therefore, the current multimodal features of the initial building local image to be retrieved can be obtained quickly through the above method.

[0101] The target building model database acquisition unit 130 is used to acquire the current building location data corresponding to the initial partial image of the building to be retrieved, and to acquire the target building model database corresponding to the current building location data in the building model dataset database.

[0102] In this embodiment, to reduce the amount of subsequent retrieval data processing, the current building location data corresponding to the initial partial image of the building to be retrieved can be further obtained. This current building location data is then used as a retrieval condition to retrieve the target building model database corresponding to the current building location data from the local building model dataset database on the server. For example, if the specific building where the user is currently located can be determined through the current building location data, the target building model data corresponding to that specific building can be directly retrieved from the building model dataset database without needing to perform subsequent image retrieval processing across the entire building model dataset database.

[0103] In one embodiment, as a first embodiment of the target building model database acquisition unit 130, the target building model database acquisition unit 130 is specifically used for:

[0104] If the current building location data corresponding to the initial partial image of the building to be retrieved includes the current building name and the current building number, then the current building name and the current building number are used as the current search criteria to retrieve the target building model database from the building model dataset database.

[0105] In this embodiment, if the current building location data corresponding to the initial partial image of the building to be retrieved includes the current building name and the current building number, it indicates that the current building location data is extremely accurate. At this time, the current building name and the current building number are used to form the current search conditions, and the target building model database is retrieved from the building model dataset database using the current building name and the current building number as the current search conditions.

[0106] It is important to note that the building model dataset database includes multiple building model databases, each corresponding to a single building or a cluster of buildings, and each database contains unique building location data. Once the current building name and building number are obtained from the current building location data and used as search criteria, the target building model database can be retrieved from the building model dataset database. Therefore, the above method allows for the rapid retrieval of the target building model database for the user's current location from the building model dataset database.

[0107] In one embodiment, as a second embodiment of the target building model database acquisition unit 130, the target building model database acquisition unit 130 is specifically used for:

[0108] If it is determined that the current building location data corresponding to the initial partial image of the building to be retrieved does not include the current building name and the current building number, then the current network communication information corresponding to the requesting end is obtained;

[0109] Based on the unique identifier data of the current building corresponding to the current network communication information, the target building model database corresponding to the current building location data is obtained from the building model dataset database.

[0110] In this embodiment, if it is determined that the current building location data corresponding to the initial partial image of the building to be retrieved does not include the current building name and the current building number, it indicates that the current building location data is fuzzy location data (e.g., only the current building name is known) or location data not represented by the building name and building number (e.g., latitude and longitude location data or base station communication location data). In this case, the current network communication information corresponding to the communication terminal currently connected to the requesting end, such as a communication base station or wireless LAN, can be obtained. Then, the unique identifier data of the current building corresponding to the deployment location of the communication base station or wireless LAN can be obtained. Finally, the unique identifier data of the current building can be used as the search condition to obtain the target building model database corresponding to the current building location data from the building model dataset database. It can be seen that the target building model database of the current user's building can also be quickly obtained from the building model dataset database through the above method.

[0111] The target building sub-model acquisition unit 140 is used to acquire the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database, and to acquire the building sub-model corresponding to the sub-model multimodal feature with the maximum feature similarity to the current multimodal feature, as the target building sub-model.

[0112] In this embodiment, to quickly identify the building sub-model corresponding to the sub-model multimodal feature with the highest feature similarity to the current multimodal feature in the server, each building sub-model in the target building model database and the complete building model dataset database stored in the server has pre-extracted building topology features that integrate building image features, building geometric features, and building topology features. After calculating the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database, the building sub-model corresponding to the sub-model multimodal feature with the highest feature similarity to the current multimodal feature is selected as the target building sub-model. The obtained target building sub-model is the model that the requesting end needs to view, and this target building sub-model can then be fed back to the requesting end.

[0113] In one embodiment, the target building sub-model acquisition unit 140 is specifically used for:

[0114] The current hash code of the current multimodal feature is obtained based on the deep cross-modal hashing model;

[0115] Obtain the sub-model hash code corresponding to the sub-model multimodal features of each building sub-model in the target building model database;

[0116] Obtain the Hamming distance between the current hash code and the sub-model hash codes of each building sub-model in the target building model database, and use it as the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database.

[0117] In this embodiment, to improve data retrieval efficiency, the current hash code of the current multimodal feature can be obtained first based on a deep cross-modal hashing model (DCMH model, DCMH stands for Deep Cross-Modal Hashing, which is a cross-modal retrieval model combining deep learning and hashing technology, capable of mapping different modal data to a unified binary hash code space). Then, the sub-model hash codes corresponding to the sub-model multimodal features of each building sub-model in the target building model database are obtained (the sub-model hash codes corresponding to the sub-model multimodal features of each building sub-model are also obtained on the server, not during the current retrieval process). Finally, the Hamming distance between the current hash code and the sub-model hash codes of each building sub-model in the target building model database is obtained, serving as the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database. It is evident that by comparing the Hamming distance between hash codes as described above, the retrieval speed can be effectively improved, and the time required to obtain retrieval results can be reduced.

[0118] In the data processing of the multimodal feature acquisition unit 120, the target building model database acquisition unit 130, and the target building sub-model acquisition unit 140, the initial local image of the building to be retrieved is acquired, and the current multimodal features are extracted from it. Then, the target building sub-model is matched from the building model dataset database.

[0119] As another implementation of the data processing procedure corresponding to the multimodal feature acquisition unit 120, the target building model database acquisition unit 130, and the target building sub-model acquisition unit 140, it can be that the current multimodal features corresponding to the initial local image of the building to be retrieved are first acquired through a pre-trained multimodal model; then, the current multimodal features are input into a preset combined model composed of a neural radiation field (NeRF) and a diffusion model, or the current multimodal features are input into a structured 3D generation model to generate a 3D model of the building to be retrieved corresponding to the initial local image of the building to be retrieved; then, the 3D model similarity between the 3D model of the building to be retrieved and each building sub-model in the target building model database is acquired (when calculating the similarity between two 3D models, the vector similarity between the current building topological features of the two 3D models can be calculated), and the building sub-model with the maximum feature similarity to the 3D model of the building to be retrieved is acquired as the target building sub-model.

[0120] The target BIM model sending unit 150 is used to obtain the target BIM model and target two-dimensional drawing file corresponding to the target building sub-model, and send the target BIM model and the target two-dimensional drawing file to the requesting end.

[0121] In this embodiment, when the target building sub-model is obtained from the server, in order to improve the result feedback speed, a lightweight target BIM model can be obtained and the target BIM model can be sent to the requesting end.

[0122] Of course, the server can also obtain the target two-dimensional drawing file corresponding to the building sub-model (such as the target CAD file and related information such as detailed drawings and design specifications related to the target CAD file), and send it to the requesting end to open and view. The target CAD file includes CAD drawings of all disciplines (architecture, structure, water, heating, electricity, etc.) and related information such as detailed drawings and design specifications related to the target CAD file. Users can open it and view it in time, realizing intelligent matching and intelligent display based on images, which can prevent users from missing relevant information.

[0123] In one embodiment, the target BIM model sending unit 150 is specifically used for:

[0124] Obtain the current user type corresponding to the requesting end; the current user type includes at least production management personnel, technical personnel, quality management personnel, and construction workers.

[0125] Obtain the unique number of the target BIM model corresponding to the target building sub-model;

[0126] The original BIM model of the target is obtained from the target building model database based on the unique number of the target BIM model and the current user type.

[0127] The target original BIM model is subjected to lightweight data conversion processing using a preset lightweight data conversion model to obtain the target BIM model;

[0128] Based on the current user type, retrieve the target 2D drawing file corresponding to the target building sub-model from the target building model database.

[0129] In this embodiment, one of the attribute data of the target building sub-model obtained from the server is the unique ID of the target BIM model. Using the unique ID of the target BIM model as a search condition, the original target BIM model can be obtained from the target building model database. Then, a lightweight data conversion model can be used to convert the original target BIM model from the original IFC format (IFC stands for Industry Foundation Classes) to a preset format (such as OBJ, where OBJ is a geometry-oriented text format), thereby achieving lightweight data processing. The obtained target BIM model, as lightweight data, can be sent to the requesting end for viewing more quickly.

[0130] As can be seen, by using the above-mentioned method of allowing different types of users to view the corresponding target BIM model and target 2D drawing files, users of all types can quickly obtain the relevant data corresponding to their responsibilities, and it is also convenient for them to understand the relevant data and perform corresponding operations. This avoids the situation of opening the full model data and drawing files of the target BIM model without considering the user type. This reduces the number of models and files that need to be opened, and the speed of opening models and files is improved due to the reduction in the amount of data required.

[0131] In this application, the initial partial image of the building to be searched is used as the initial search condition to retrieve building-related data. However, in specific implementation, keywords, axis positioning, etc. can also be used as search conditions to retrieve building-related data. Specifically, keywords, axis positioning, etc. can be used as search conditions without image processing and image feature extraction. Instead, the matching results can be obtained directly using text features.

[0132] In one embodiment, the image-based building model data search device 100 further includes:

[0133] The maintenance information sending unit is configured to, if a building maintenance request corresponding to the target BIM model is detected by the requesting end, obtain target maintenance information corresponding to the target BIM model and send the target maintenance information to the requesting end; wherein the target maintenance information includes at least one of maintenance text information and maintenance video information.

[0134] In this embodiment, after the requesting end receives the target BIM model sent by the server, if the user on the requesting end discovers that the location corresponding to the initially captured partial image of the building to be retrieved needs maintenance but is unsure how to perform the maintenance, they can send a building maintenance request to the server for the target BIM model. The server can preset maintenance information for each building sub-model in the building model dataset database. Furthermore, it can obtain target maintenance information (including at least one of maintenance text information and maintenance video information) corresponding to the target BIM model and send this target maintenance information to the requesting end. The requesting end can then perform timely on-site maintenance on the location corresponding to the initially captured partial image of the building based on the target maintenance information.

[0135] As can be seen, the embodiment of implementing this device can first extract the current multimodal features from the initial partial image of the building to be retrieved based on the multimodal model, and then obtain the target building sub-model with the maximum feature similarity to the current multimodal features from the target building model database determined based on the current building positioning data. This simplifies the calculation process of retrieving the three-dimensional model from the image, not only improving the efficiency of image retrieval results, but also allowing simultaneous viewing of the design drawing data related to the three-dimensional model.

[0136] The aforementioned image-based building model data search device can be implemented as a computer program, which can, for example... Figure 7 It runs on the computer device shown.

[0137] Please see Figure 7 , Figure 7 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. This computer device integrates any of the image-based architectural model data search devices provided in the embodiments of the present invention.

[0138] See Figure 7 The computer device 400 includes a processor 402, a memory, and a network interface 405 connected via a system bus 401. The memory may include a storage medium 403 and internal memory 404.

[0139] The storage medium 403 may store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions that, when executed, cause the processor 402 to perform an image-based building model data search method.

[0140] The processor 402 provides computing and control capabilities to support the operation of the entire computer device.

[0141] The internal memory 404 provides an environment for the computer program 4032 in the storage medium 403 to run. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the above-described image-based building model data search method.

[0142] This network interface 405 is used for network communication with other devices. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0143] The processor 402 is used to run the computer program 4032 stored in the memory to implement the above-described image-based building model data search method.

[0144] It should be understood that, in this embodiment of the invention, the processor 402 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0145] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0146] Therefore, the present invention also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the above-described image-based building model data search method.

[0147] The storage medium can be any computer-readable storage medium that can store program code, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0149] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0150] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0151] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0152] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for searching architectural model data based on images, characterized in that, include: In response to an image search command sent by the requesting end, an initial partial image of the building to be searched corresponding to the image search command is obtained; wherein, the partial image of the building to be searched is a real-scene photograph, a hand-drawn sketch, a modified drawing, or a two-dimensional structural drawing of the building; The current multimodal features corresponding to the initial local image of the building to be retrieved are obtained by a pre-trained multimodal model; wherein the current multimodal features are obtained by fusing at least the building image features and the 3D model features; Obtain the current building location data corresponding to the initial partial image of the building to be retrieved, and obtain the target building model database corresponding to the current building location data from the building model dataset database; Obtain the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database, and obtain the building sub-model corresponding to the sub-model multimodal feature with the maximum feature similarity to the current multimodal feature, as the target building sub-model; Obtain the target BIM model and target 2D drawing file corresponding to the target building sub-model, and send the target BIM model and target 2D drawing file to the requesting end.

2. The method according to claim 1, characterized in that, The step of obtaining the current multimodal features corresponding to the initial local image of the building to be retrieved through a pre-trained multimodal model includes: The ViT encoder in the multimodal model is used to extract features from the initial local image of the building to be retrieved, thereby obtaining the features of the current building image; The current building geometric features of the initial building local image to be retrieved are obtained through the random sampling model in the multimodal model, and the current building topological features of the initial building local image to be retrieved are obtained through the Reeb graph model in the multimodal model; wherein, the current building geometric features and the current building topological features constitute the three-dimensional model features corresponding to the initial building local image to be retrieved; The current building image features, the current building geometric features, and the current building topological features are fused using the cross-attention mechanism in the multimodal model to obtain the current multimodal features.

3. The method according to claim 1, characterized in that, The step of obtaining the current building location data corresponding to the initial partial image of the building to be retrieved, and obtaining the target building model database corresponding to the current building location data from the building model dataset database, includes: If the current building location data corresponding to the initial partial image of the building to be retrieved includes the current building name and the current building number, then the current building name and the current building number are used as the current search criteria to retrieve the target building model database from the building model dataset database.

4. The method according to claim 1, characterized in that, The step of obtaining the current building location data corresponding to the initial partial image of the building to be retrieved, and obtaining the target building model database corresponding to the current building location data from the building model dataset database, includes: If it is determined that the current building location data corresponding to the initial partial image of the building to be retrieved does not include the current building name and the current building number, then the current network communication information corresponding to the requesting end is obtained; Based on the unique identifier data of the current building corresponding to the current network communication information, the target building model database corresponding to the current building location data is obtained from the building model dataset database.

5. The method according to claim 1, characterized in that, The step of obtaining the feature similarity between the current multimodal features and the sub-model multimodal features of each building sub-model in the target building model database includes: The current hash code of the current multimodal feature is obtained based on the deep cross-modal hashing model; Obtain the sub-model hash code corresponding to the sub-model multimodal features of each building sub-model in the target building model database; Obtain the Hamming distance between the current hash code and the sub-model hash codes of each building sub-model in the target building model database, and use it as the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database.

6. The method according to claim 1, characterized in that, The process of obtaining the target BIM model and target 2D drawing file corresponding to the target building sub-model includes: Obtain the current user type corresponding to the requesting end; the current user type includes at least production management personnel, technical personnel, quality management personnel, and construction workers. Obtain the unique number of the target BIM model corresponding to the target building sub-model; The original BIM model of the target is obtained from the target building model database based on the unique number of the target BIM model and the current user type. The target original BIM model is subjected to lightweight data conversion processing using a preset lightweight data conversion model to obtain the target BIM model; Based on the current user type, retrieve the target 2D drawing file corresponding to the target building sub-model from the target building model database.

7. The method according to claim 1, characterized in that, After the steps of obtaining the target BIM model and target 2D drawing file corresponding to the target building sub-model, and sending the target BIM model and target 2D drawing file to the requesting end, the method further includes: If a building maintenance request corresponding to the target BIM model is detected by the requesting end, then target maintenance information corresponding to the target BIM model is obtained and the target maintenance information is sent to the requesting end; wherein, the target maintenance information includes at least one of maintenance text information and maintenance video information.

8. A device for searching architectural model data based on images, characterized in that, include: The building image acquisition unit is used to acquire an initial partial image of the building to be retrieved in response to an image search command sent by the requesting end; wherein the partial image of the building to be retrieved is a real-scene photograph, a hand-drawn sketch, a modified drawing, or a two-dimensional structural drawing of the building; A multimodal feature acquisition unit is used to acquire current multimodal features corresponding to the initial local image of the building to be retrieved through a pre-trained multimodal model; wherein, the current multimodal features are obtained by fusing at least building image features and 3D model features; The target building model database acquisition unit is used to acquire the current building location data corresponding to the initial partial image of the building to be retrieved, and to acquire the target building model database corresponding to the current building location data from the building model dataset database; The target building sub-model acquisition unit is used to acquire the feature similarity between the current multimodal feature and the sub-model multimodal features of each building sub-model in the target building model database, and to acquire the building sub-model corresponding to the sub-model multimodal feature with the maximum feature similarity to the current multimodal feature, as the target building sub-model. The target BIM model sending unit is used to obtain the target BIM model and target two-dimensional drawing file corresponding to the target building sub-model, and send the target BIM model and the target two-dimensional drawing file to the requesting end.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the image-based building model data search method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, can implement the image-based building model data search method as described in any one of claims 1-7.

Images