A method for indexing and retrieving architectural engineering image data

Abstract

The present invention discloses a construction engineering image data indexing and retrieval method, the indexing method comprising: step 1: collecting construction engineering image data; step 2: labeling and analyzing the collected construction engineering image data to obtain a semantic information set including project goals, construction stage locations, construction processes, quality issues and safety hazards; step 3: establishing a multidimensional index of construction engineering image data according to the content of the semantic information set. The retrieval method is based on the indexing method and also comprises step 4: selecting a conditional retrieval method or a vector retrieval method to retrieve the construction engineering image data according to the user's retrieval requirements. The present invention can quickly and accurately retrieve in massive image data, realize accurate search and utilization of image data, and improve the management efficiency and utilization rate of construction engineering image data.

Description

Technical Field

[0001] The present invention relates to the field of construction engineering management, and in particular to a construction engineering image data indexing and retrieval method. Background Art

[0002] At construction sites, video surveillance and other image recordings are common management methods. These image data can provide real-time conditions at the construction site, making it easier for managers to monitor and make decisions. However, as construction projects proceed, the amount of image data continues to increase, and how to efficiently index and retrieve this data has become an urgent problem to be solved. In the prior art, the management of image data often faces the following difficulties:

[0003] 1. Massive data: Video surveillance at construction sites will accumulate massive amounts of image data, which is difficult to retrieve.

[0004] 2. Multi-type data: Different types of image sources increase the complexity of data processing. Different types of images include but are not limited to video surveillance images, drone aerial images, and panoramic inspection images.

[0005] 3. Data silos: There is a lack of connection between image data and other engineering data (such as BIM models, construction logs, etc.), making it difficult to integrate and utilize information. Summary of the invention

[0006] The purpose of the present invention is to provide a construction engineering image data indexing and retrieval method to solve the problems of difficulty in retrieving massive image data, complex retrieval of multiple types of image sources, and low retrieval efficiency, which affect the management and utilization efficiency of construction engineering image data.

[0007] In order to solve the above technical problems, the present invention provides a construction engineering image data indexing method, comprising:

[0008] Step 1: Construction engineering image data collection;

[0009] Step 2: Label and analyze the collected construction engineering image data to obtain a semantic information set including project goals, construction stage locations, construction processes, quality issues, and safety hazards;

[0010] Step 3: Establish a multidimensional index of construction engineering image data based on the content of the semantic information set.

[0011] Furthermore, in the construction engineering image data indexing method provided by the present invention, in step 1, the construction engineering image data collection includes one or more image sources including video surveillance images, drone aerial images, panoramic inspection images and other images.

[0012] Furthermore, the construction engineering image data indexing method provided by the present invention uses an adaptive image enhancement algorithm to pre-process the image data before labeling and analyzing the collected construction engineering image data to improve the image quality.

[0013] Furthermore, in the construction engineering image data indexing method provided by the present invention, in step 2, the method of labeling and analyzing the collected construction engineering image data to obtain a semantic information set including project objectives, construction stage locations, construction processes, quality issues, and safety hazards includes:

[0014] Step 21: Use deep learning target recognition algorithm to obtain various engineering target information at the construction site;

[0015] Step 22: Use a multi-scale feature fusion network algorithm to classify the construction stages and parts in the construction engineering image data, obtain the construction stage and part information, and further identify the construction process information of each construction stage and part;

[0016] Step 23: Use a multimodal large model to automatically identify quality issues and safety hazards in the labeled image data.

[0017] Furthermore, in the construction engineering image data indexing method provided by the present invention, in step 3, the method of establishing a multidimensional index of construction engineering image data based on a semantic information set includes: fusing the semantic dimensions of project objectives, construction stage locations, construction processes, quality issues and safety hazards, establishing a multidimensional indexing system, and constructing an index based on spatial semantic information.

[0018] Furthermore, in the construction engineering image data indexing method provided by the present invention, in step 3, the semantic dimensions of the engineering target, construction stage location, construction technology, quality problems and safety hazards are integrated to establish a multidimensional indexing system, and the method for constructing an index based on spatial semantic information includes:

[0019] Step 31: Project target indexing, accurately locating the project target in the GIS matching image, using the GIS coordinates as indexes;

[0020] Step 32: Indexing of construction stage parts, using hierarchical clustering AI algorithm to build a multi-level fine-grained index;

[0021] Step 33: Construction process indexing: Based on the text of the construction organization design, semantic search and similarity matching are performed to build a refined index of the construction process;

[0022] Step 34: Quality problem indexing: using the image-to-text technology, the quality problems in the image are converted into text descriptions, and then the large model is used for semantic generalization to establish a quality problem index.

[0023] Step 35: Safety hazard indexing: Using the image-to-text technology, the safety hazards in the image are converted into text descriptions, and then a large model is used to perform semantic generalization to establish a safety hazard index.

[0024] Step 36: Integrate the project target index, construction stage location index, construction process index, quality problem index and safety hazard index to establish a multi-dimensional index system based on spatial semantic information.

[0025] In order to solve the above technical problems, the present invention further provides a construction engineering image data retrieval method, which adopts the above construction engineering image data indexing method and comprises:

[0026] Step 4: Select a conditional retrieval method or a vector retrieval method to retrieve the construction engineering image data according to the user's retrieval requirements.

[0027] Furthermore, the construction engineering image data retrieval method provided by the present invention comprises:

[0028] Build a knowledge graph in the construction field to retrieve related words or synonyms;

[0029] When the user's search needs are composed of conditions determined by multiple search terms, related synonyms or other closely related words are retrieved from the knowledge graph and combined with the original search terms as search conditions to achieve efficient and accurate image data retrieval.

[0030] Furthermore, the construction engineering image data retrieval method provided by the present invention, the vector retrieval method comprises:

[0031] When the user's search conditions are relatively broad, the deep natural language analysis algorithm is used to segment and extract the engineering goals, construction stage locations, construction processes, quality issues and safety issues in the user's search sentences to form quantitative semantic vectors, calculate the similarity between vectors, and compare them with the feature vectors of the semantic information set of the engineering image. i ,l i ,p i ,q i ,s i > Perform similarity calculation and comparison to achieve fuzzy query and similar image search, where the feature vector b i is the i-th engineering goal, l i is the position of the i-th construction stage, p i is the i-th construction process, q i is the i-th quality problem, s i is the i-th safety hazard, where i is a positive integer.

[0032] ​Compared with the prior art, the present invention has the following beneficial effects:

[0033] The construction engineering image data indexing and retrieval method provided by the present invention establishes a multidimensional index of construction engineering image data according to the content of a semantic information set, which can quickly and accurately retrieve in massive image data, thereby realizing accurate search and utilization of image data; by establishing a multidimensional index of construction engineering image data containing a semantic information set in each image data from different sources, the indexing and retrieval difficulty of image data from multiple sources is reduced, thereby improving the management efficiency and utilization rate of construction engineering image data.

[0034] The construction engineering image data indexing and retrieval method provided by the present invention supports conditional retrieval and vector retrieval to meet the needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the indexing and retrieval method of construction engineering image data;

[0036] Figure 2 It is a construction engineering image diagram of water leakage in the post-cast zone;

[0037] Figure 3 This is an image of a construction project where the cantilever of the top support of the rack is too long. DETAILED DESCRIPTION

[0038] The present invention is described in detail below in conjunction with the accompanying drawings: The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and in non-precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0039] The embodiment of the present invention provides a construction engineering image data indexing and retrieval method, including a construction engineering image data indexing and a construction engineering image data retrieval method.

[0040] Please refer to Figure 1 The embodiment of the present invention provides a method for indexing construction engineering image data, which may include the following steps:

[0041] Step 1: Construction engineering image data collection. Construction engineering image data collection includes one or more image sources such as video surveillance images, drone aerial images, panoramic inspection images, and other images, that is, it can be single image data from one image source, or multi-source image data from different image sources. Other images can be image data taken by electronic devices with camera functions such as mobile phones. In order to improve the quality of image data, before the collected construction engineering image data is labeled and analyzed, an adaptive image enhancement algorithm is used to pre-process the image data to improve the image quality. Preprocessing refers to processing the quality of image data.

[0042] Step 2: Label and analyze the collected construction engineering image data to obtain a semantic information set including project goals, construction stage locations, construction technology, quality issues, and safety hazards. The semantic information set is {B, L, P, Q, S}, where B is the project goal, L is the construction stage location, P is the construction technology, Q is the quality issue, and S is the safety hazard. The semantic information set {B, L, P, Q, S} includes the project goal set B = {b i}、Construction stage location set L={l i}、Construction process set P={p i}、Quality problem set Q={q i}、Security risk set S={s i}. Among them b i is the i-th engineering goal, l i is the position of the i-th construction stage, p i is the i-th construction process, q i is the i-th quality problem, s i is the ith safety hazard, where i is a positive integer. The specific method of step 2 includes:

[0043] Step 21: Use deep learning target recognition algorithm to obtain information on various engineering targets at the construction site, that is, to accurately identify and locate each engineering target.

[0044] Step 22: Use the multi-scale feature fusion network algorithm to classify the construction stages and locations in the construction engineering image data, obtain the construction stage location information, and further identify the construction process information of each construction stage location.

[0045] Step 23: Use a multimodal large model to automatically identify quality issues and safety hazards in the labeled image data.

[0046] Step 3: Establish a multidimensional index of construction engineering image data based on the content of the semantic information set. In step 3, the method of establishing a multidimensional index of construction engineering image data based on the semantic information set includes: integrating the semantic dimensions of project goals, construction stage locations, construction processes, quality issues, and safety hazards, establishing a multidimensional index system, and constructing an index based on spatial semantic information. Specifically, it includes:

[0047] Step 31: Project target indexing, accurately locate the project target in the GIS matching image, and use the GIS coordinates as the index; that is, location indexing.

[0048] Step 32: Index the construction stage parts using a hierarchical clustering AI algorithm to build a multi-level fine-grained index.

[0049] Step 33: Construction process indexing: Based on the text of the construction organization design, semantic search and similarity matching are performed to build a refined index of construction processes.

[0050] Step 34: Quality problem indexing: Using image-to-text technology, the quality problems in the image are converted into text descriptions, and then a large model is used to perform semantic generalization to establish a quality problem index.

[0051] Step 35: Safety hazard indexing: Using the image-to-text technology, the safety hazards in the image are converted into text descriptions, and then a large model is used to perform semantic generalization to establish a safety hazard index.

[0052] Step 36: Integrate the project target index, construction stage location index, construction process index, quality problem index and safety hazard index to establish a multi-dimensional index system based on spatial semantic information.

[0053] Please refer to Figure 1 The embodiment of the present invention further provides a construction engineering image data retrieval method based on the construction engineering image data indexing method, comprising:

[0054] Step 4: Select a conditional retrieval method or a vector retrieval method to retrieve the construction engineering image data according to the user's retrieval requirements.

[0055] The conditional search methods include:

[0056] Build a knowledge graph in the construction field to search for related words or synonyms. When the user's search needs are composed of conditions determined by multiple search terms, retrieve related synonyms or other closely related words from the knowledge graph and combine them with the original search terms as search conditions to achieve efficient and accurate image data retrieval.

[0057] The vector retrieval methods include:

[0058] When the user's search conditions are relatively broad, the deep natural language analysis algorithm is used to segment and extract the engineering goals, construction stage locations, construction processes, quality issues and safety issues in the user's search sentences to form quantitative semantic vectors, calculate the similarity between vectors, and compare them with the feature vectors of the semantic information set of the engineering image. i ,l i ,p i ,q i ,s i > Perform similarity calculation and comparison to achieve fuzzy query and similar image search, where the feature vector b i is the i-th engineering goal, l i is the position of the i-th construction stage, p i is the i-th construction process, q i ​is the i-th quality problem, s i is the i-th safety hazard, where i is a positive integer.

[0059] The following is an illustrative description of the construction project image data retrieval method according to an embodiment of the present invention through a specific construction project.

[0060] Through on-site investigation and shooting, we can collect images of construction projects. Figure 2 and Figure 3 .

[0061] Figure 2 It is a construction engineering image of post-casting belt leakage. The semantic information set obtained by analysis includes the engineering target set B={concrete, beam, wall, post-casting belt}, the construction stage part set L={concrete wall, concrete pouring, A-axis wall construction}, the construction process set P={post-casting belt construction}, the quality problem set Q={concrete cracking, water leakage}, and the safety hazard set S={∅}. Among them, "∅" is an empty set, that is, there is no safety hazard information.

[0062] Figure 3 It is an image of a construction project in which the top support cantilever of the rack is too long. The semantic information set obtained by analysis includes the project target set B={formwork, bracket, formwork system, bracket system}, the construction stage part set L={formwork engineering, 2-story floor slab formwork construction}, the construction process set P={formwork bracket process}, the quality problem set Q={∅}, and the safety hazard set S={top support cantilever is too long, and the cantilever does not meet the requirements}.

[0063] According to the indexing method in steps 1-3 above, the semantic information set is indexed and stored as a database and knowledge graph in preparation for retrieval. For example, if the search condition is "A-axis, wall, post-cast strip, water leakage", which is a relatively certain condition, the search method based on the knowledge graph is used to find that the associated semantics of the post-cast strip is "concrete", and the combination of the search conditions can be retrieved. Figure 2 .

[0064] For example, the user's search condition is "safety issues of formwork support for floor slabs", which is a relatively broad description. Using the deep natural language analysis algorithm, the search condition is extracted into a quantitative semantic vector of <formwork support, # layers, formwork process, ∅, # safety issues#>, which is consistent with the semantic information feature vector of the image in the library.<B,L,P,Q,S> By performing similarity calculation and comparison, we can find Figure 3 .

[0065] The construction engineering image data indexing and retrieval method provided in the embodiment of the present invention establishes a multidimensional index of construction engineering image data according to the content of a semantic information set, which can quickly and accurately retrieve in massive image data, thereby realizing accurate search and utilization of image data; by establishing a multidimensional index of construction engineering image data containing a semantic information set in each image data from different sources, the indexing and retrieval difficulty of image data from multiple sources is reduced, thereby improving the management efficiency and utilization rate of construction engineering image data.

[0066] The construction engineering image data indexing and retrieval method provided in the embodiment of the present invention combines multiple AI models to improve the analysis and annotation accuracy of image data and the accuracy of data indexing; it supports conditional retrieval and vector retrieval to meet the needs of different users.

[0067] In the construction engineering image data indexing and retrieval method provided by the embodiment of the present invention, the term "above" includes the number itself.

[0068] The embodiments of the present invention are not limited to the above-mentioned specific implementation modes. Obviously, the above-mentioned embodiments are only some embodiments of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the described embodiments of the present invention belong to the scope of protection of the present invention. Those skilled in the art can make other levels of modifications and changes to the present invention. In this way, if these modifications and changes of the present invention fall within the scope of the claims of the present invention, the embodiments of the present invention are also intended to include these changes and changes.

Claims

1. A construction engineering image data indexing method, characterized in that: include: Step 1: Acquisition of construction engineering image data at the construction site, wherein the construction engineering image data acquisition includes one or more image sources selected from video surveillance images, drone aerial images, panoramic inspection images, and other images; Step 2: Label and analyze the collected construction engineering image data to obtain a semantic information set including project goals, construction stage locations, construction processes, quality issues, and safety hazards, including: Step 21: Use deep learning target recognition algorithm to obtain various engineering target information at the construction site; Step 22: Use a multi-scale feature fusion network algorithm to classify the construction stages and parts in the construction engineering image data, obtain the construction stage and part information, and further identify the construction process information of each construction stage and part; Step 23: Use a multimodal large model to automatically identify quality issues and safety hazards in the labeled image data; Step 3: Establish a multidimensional index of construction engineering image data based on the content of the semantic information set, including: integrating the semantic dimensions of project goals, construction stage locations, construction processes, quality issues, and safety hazards, establishing a multidimensional index system, and constructing an index based on spatial semantic information; including: Step 31: Project target indexing, accurately locating the project target in the GIS matching image, using the GIS coordinates as indexes; Step 32: Indexing of construction stage parts, using hierarchical clustering AI algorithm to build a multi-level fine-grained index; Step 33: Construction process indexing: Based on the text of the construction organization design, semantic search and similarity matching are performed to build a refined index of the construction process; Step 34: Quality problem indexing: using the image-to-text technology, the quality problems in the image are converted into text descriptions, and then the large model is used for semantic generalization to establish a quality problem index; Step 35: Safety hazard indexing: Using the image-to-text technology, the safety hazards in the image are converted into text descriptions, and then a large model is used to perform semantic generalization to establish a safety hazard index. Step 36: Integrate the project target index, construction stage location index, construction process index, quality problem index and safety hazard index to establish a multi-dimensional index system based on spatial semantic information.

2. The construction engineering image data indexing method according to claim 1, characterized in that: Before annotating and analyzing the collected construction engineering image data, an adaptive image enhancement algorithm is used to pre-process the image data to improve the image quality.

3. A construction engineering image data retrieval method, characterized in that: The construction engineering image data indexing method according to any one of claims 1 to 2 comprises: Step 4: Select a conditional retrieval method or a vector retrieval method to retrieve the construction engineering image data according to the user's retrieval requirements.

4. The construction engineering image data retrieval method according to claim 3, characterized in that: The conditional retrieval method comprises: Build a knowledge graph in the construction field to retrieve related words or synonyms; When the user's search needs are composed of conditions determined by multiple search terms, related synonyms or other closely related words are retrieved from the knowledge graph and combined with the original search terms as search conditions to achieve efficient and accurate image data retrieval.

5. The construction engineering image data retrieval method according to claim 3, characterized in that: The vector retrieval method comprises: When the user's search conditions are relatively broad, the deep natural language analysis algorithm is used to segment and extract the engineering goals, construction stage locations, construction processes, quality issues and safety issues in the user's search sentences to form quantitative semantic vectors, calculate the similarity between vectors, and compare them with the feature vectors of the semantic information set of the engineering image. i ,l i ,p i ,q i ,s i > Perform similarity calculation and comparison to achieve fuzzy query and similar image search, where the feature vector b i is the i-th engineering goal, l i is the position of the i-th construction stage, p i is the i-th construction process, q i is the i-th quality problem, s i is the i-th safety hazard, where i is a positive integer.​

Images