A construction site quality inspection navigation method, quality inspection method and system

Abstract

This invention belongs to the technical field of construction quality inspection, and discloses a construction site quality inspection navigation method, quality inspection method, and system. The method includes: generating an initial inspection navigation path based on quality inspection instructions and building space information; wherein the quality inspection instructions include at least one space to be inspected; performing overall path planning based on global path points on the initial inspection navigation path, and acquiring real-time site information at each step location; and performing local path planning between two adjacent global path points using a visual navigation method based on the real-time site information. This invention proposes to first combine building space information to generate an initial inspection navigation path, making the overall path planning more reasonable, and then performing local path planning between two global path points using a visual language navigation method based on real-time site information collected on-site. This allows for dynamic adjustment to adapt to complex quality inspection tasks at various stages of the construction site.

Description

Technical Field

[0001] This invention belongs to the technical field of construction quality inspection, and more specifically, relates to a construction site quality inspection navigation method, quality inspection method and system. Background Technology

[0002] Quality control is one of the three major objectives of construction project management. Serious construction quality defects or hidden dangers can cause enormous loss of life and property. Currently, there are relatively comprehensive laws, regulations, and technical guidance documents for construction quality control. However, at the on-site management level, quality management at each stage of construction largely relies on the on-site inspections and patrols of supervisors. While this can meet basic quality control needs, relevant information and technical briefings are mostly communicated and recorded verbally or manually, which cannot guarantee objectivity. Furthermore, quality problems on construction sites are dynamic and diverse. This traditional quality management method can only rectify quality problems and dangerous conditions discovered through spot checks and inspections, and cannot provide a continuous, real-time, and accurate grasp of the overall quality status of the project.

[0003] With the development of cutting-edge technologies such as intelligent sensors, computer vision, and mobile robots, using mobile intelligent agents to replace manual inspections has become a new way to improve the efficiency of quality inspections and continuously discover construction quality defects or problems. However, construction site environments are complex, with numerous facilities, equipment, and various workers. Traditional mobile robot navigation and path planning methods lack flexibility and are difficult to dynamically adjust to adapt to the complex quality inspection tasks at different stages of the construction site. Furthermore, existing mobile robot intelligent inspection methods and systems have a low degree of interaction and collaboration with on-site management personnel, insufficient understanding of abstract natural language navigation instructions, and the issuance and execution of quality inspection tasks require a certain level of knowledge and technical expertise, making it impossible to flexibly and reliably complete continuous quality inspections at construction sites. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a construction site quality inspection navigation method, quality inspection method and system, which is used to solve the problem that traditional mobile robot navigation and path planning methods are not flexible enough in construction sites and are difficult to dynamically adjust to adapt to the complex quality inspection tasks at various stages of construction sites.

[0005] To achieve the above objectives, according to one aspect of the present invention, a construction site quality inspection navigation method is provided, comprising:

[0006] An initial inspection navigation path is generated based on the quality inspection instructions and the building space information of the construction site; wherein the building space information includes the spatial composition of the building and the connectivity between the spaces, and the quality inspection instructions include at least one space to be inspected;

[0007] Overall path planning is performed based on global path points on the initial inspection navigation path, and real-time field information is obtained at each step between two adjacent global path points. Based on the real-time field information, a visual navigation method is used to perform local path planning between two adjacent global path points.

[0008] According to the construction site quality inspection navigation method provided by the present invention, generating the initial inspection navigation path specifically includes:

[0009] The quality inspection instructions are received from natural language using a generative pre-trained large language model, and the initial inspection navigation path is generated by combining the retrieval results of building space information with retrieval enhancement generation technology.

[0010] The construction site quality inspection navigation method provided by this invention acquires real-time site information at each step location between two adjacent global path points, and performs local path planning between two adjacent global path points using a visual navigation method based on the real-time site information. Specifically, it includes:

[0011] Acquire point cloud information around the current location, and build a local raster map of the current location based on the point cloud information;

[0012] Multiple candidate orientations are divided around the current location. The first preset neighborhood range of the candidate orientations is searched based on the distance between the grid on the local grid map and the current location, the obstacle information on the grid, and the orientation information of the grid. One of the grids is then determined as the next candidate path point corresponding to the candidate orientation.

[0013] According to the construction site quality inspection navigation method provided by the present invention, determining one of the grid cells as the candidate path point for the next step corresponding to the candidate orientation specifically includes:

[0014] Obstacle information of grid cells on a local grid map is obtained from point cloud information, and passable grid cells are obtained from obstacle information;

[0015] Select one of the passable grids within the first preset neighborhood of the candidate orientation as the next candidate path point corresponding to the candidate orientation; and make the probability of the passable grid being selected greater the greater the distance between the passable grid and the current position, the lower the probability of the passable grid colliding with the obstacle, and the smaller the deviation between the direction of the passable grid and the current position and the candidate orientation.

[0016] The construction site quality inspection navigation method provided by the present invention acquires real-time site information at each step location between two adjacent global path points, and performs local path planning between two adjacent global path points using a visual navigation method based on the real-time site information, and further includes:

[0017] The system acquires visual images of multiple candidate orientations at the current location and extracts image features from each. Simultaneously, it extracts the instruction semantic features of the navigation command corresponding to the current location in the initial inspection navigation path. The image features corresponding to the candidate orientations and the instruction semantic features are then fused to obtain visual-language fusion features.

[0018] Based on the visual-language fusion features of the candidate orientation, the orientation direction features, and the current step number, multi-dimensional feature vectors are established for the currently reached path points and candidate path points, respectively.

[0019] The node representation vectors of the reached path points and the candidate path points are fused together to obtain the node representation vectors of the reached path points and the candidate path points.

[0020] Based on the node representation vectors of candidate path points, a scoring model is used to select the candidate path point with the highest score as the target path point for the next step at the current position.

[0021] According to the construction site quality inspection navigation method provided by the present invention, the multi-dimensional feature vector of the currently reached path point N Specifically, it can be obtained through the following formula:

[0022] ;

[0023] In the formula, n Indicates the number of candidate orientations; Indicates the first i Visual-linguistic fusion features of candidate orientations; D i Indicates the first i Orientation direction features of each candidate orientation; S Indicates the current step number;

[0024] Multidimensional feature vectors of candidate path points N i Specifically, it can be obtained through the following formula:

[0025] ;

[0026] In the formula, m This indicates the number of reached path points corresponding to the candidate orientation of a candidate path point; Indicates the candidate orientation corresponding to the first j Visual-linguistic fusion features of each reached path point in the candidate orientation; D j This indicates the candidate orientation to which the candidate path point belongs, and the corresponding candidate orientation. j The orientation characteristics of the reached path points.

[0027] According to the construction site quality inspection navigation method provided by this invention, the scoring model is as follows:

[0028] The scoring model takes the node representation vectors of candidate path points as input and outputs the scores of the candidate path points; during the training of the scoring model, the target nodes of each step in the inspection path are defined by experts. y * Supervised learning is performed by minimizing the cross-entropy loss of the negative logarithmic prediction probability. L Specifically as follows:

[0029] ;

[0030] in, It is the predicted probability of the target node, which is obtained by normalizing the scores of all candidate path points.

[0031] According to a second aspect of the present invention, a construction site quality inspection method is provided, which includes planning a quality inspection path using the construction site quality inspection navigation method described in any one of the above claims, and further includes:

[0032] A quality inspection plan is generated based on the quality inspection instructions and a pre-established quality inspection knowledge database; the quality inspection instructions also include quality inspection items.

[0033] Upon reaching a global pathpoint within the space to be inspected, a quality inspection is conducted according to the quality inspection plan.

[0034] According to the construction site quality inspection method provided by this invention, the method for establishing a quality inspection knowledge database is as follows:

[0035] Collect relevant documents for construction quality inspection;

[0036] The relevant documents for construction quality inspection are pre-processed and divided into clause structures;

[0037] After the text structure is divided, a semantic embedding model is used to encode each text into a vector, extract deep semantic features, and generate a high-dimensional knowledge vector with a unified dimension.

[0038] Each clause in the construction quality inspection-related documents is ultimately stored as an independent knowledge node. The node contains the original text, a high-dimensional knowledge vector, and standardized metadata. The standardized metadata includes at least one of the following: file name, clause number, and document attribute information.

[0039] According to a third aspect of the present invention, a construction site quality inspection and navigation system is provided, the system including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the construction site quality inspection and navigation method described in any of the above claims.

[0040] Overall, compared with the prior art, the construction site quality inspection navigation method, quality inspection method, and system provided by this invention offer the following advantages:

[0041] 1. It is proposed to first use conventional path planning methods combined with building spatial information to generate an initial inspection navigation path, so that the overall path planning is more reasonable. Then, between two global path points, a visual navigation method is used to perform local path planning based on real-time on-site information collected on-site. This can support dynamic navigation decision-making and enable mobile robots to perform flexible and autonomous navigation when performing quality inspection tasks in complex construction sites. It can dynamically adjust to adapt to complex quality inspection tasks at various stages of construction sites.

[0042] 2. Based on a large language model, natural language is used to issue quality inspection tasks. The prior spatial knowledge provided by building spatial information is combined to improve the ability of the large language model to parse tasks, generate inspection methods and navigation instructions, and improve the efficiency and flexibility of issuing inspection tasks.

[0043] 3. Establishing candidate path points based on point cloud information of the construction site can effectively avoid obstacles on site, thereby improving adaptability to the complex environment of the construction site and ensuring the smooth progress of quality inspection; the efficient fusion and alignment of navigation instructions and site images based on a large language model enables the comprehensive consideration of visual images and navigation instructions of the construction site when selecting candidate path points, forming a visual language navigation method, which helps to ensure that navigation decisions are consistent with both the construction site and navigation instructions, thereby improving the effectiveness and adaptability of navigation;

[0044] 4. A spatial topology map can be constructed based on the reached path points and the candidate path points of the current position as a navigation memory mechanism, which improves the planning ability of mobile robots to continuously navigate in complex construction environments. Attached Figure Description

[0045] Figure 1 This is a flowchart of the construction site quality inspection navigation method provided by the present invention.

[0046] Figure 2 This is a flowchart illustrating the construction process of the construction quality inspection knowledge database provided by this invention.

[0047] Figure 3 This is a schematic diagram of knowledge nodes in the quality inspection knowledge database provided by this invention.

[0048] Figure 4 This invention provides a data flow diagram for generating spatial topology diagrams from building information models.

[0049] Figure 5This is a schematic diagram of the spatial information data of the second floor of an office building in a specific example provided by the present invention.

[0050] Figure 6 This is a flowchart of the initial inspection scheme generation provided by the present invention.

[0051] Figure 7 This is a flowchart of the visual-language fusion feature generation process provided by the present invention.

[0052] Figure 8 This is the preliminary graph embedding representation process of spatial topology graph nodes provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0054] Please see Figure 1 This embodiment provides a construction site quality inspection navigation method, which includes:

[0055] An initial inspection navigation path is generated based on the quality inspection instructions and the building space information of the construction site; wherein the building space information includes the spatial composition of the building and the connectivity between the spaces, and the quality inspection instructions include at least one space to be inspected;

[0056] Overall path planning is performed based on global path points on the initial inspection navigation path, and real-time field information is obtained at each step between two adjacent global path points. Based on the real-time field information, a visual navigation method is used to perform local path planning between two adjacent global path points.

[0057] In another embodiment, a construction site quality inspection method is also provided. This method employs a construction site quality inspection navigation method for quality inspection path planning. The method further includes:

[0058] A quality inspection plan is generated based on the quality inspection instructions and a pre-established quality inspection knowledge database; the quality inspection instructions also include quality inspection items.

[0059] Upon reaching a global pathpoint within the space to be inspected, a quality inspection is conducted according to the quality inspection plan.

[0060] Furthermore, generating the initial inspection navigation path specifically includes:

[0061] A generative pre-trained large language model is used to receive the quality inspection instructions in natural language, and combined with the retrieval results of building space information using retrieval enhancement generation technology, an initial inspection navigation path is generated. Alternatively, a generative pre-trained large language model can be used to receive the quality inspection instructions in natural language, and combined with the retrieval results of building space information and a quality inspection knowledge database using retrieval enhancement generation technology, an initial inspection plan can be generated; wherein, the initial inspection plan includes an initial inspection navigation path and a quality inspection plan.

[0062] In some specific embodiments, a smart construction quality inspection navigation method and a quality inspection method based on a large language model are provided. This method generates an executable initial inspection plan based on natural language commands and building spatial information, and establishes a spatial topology graph based on on-site collected image data and point cloud information to support dynamic navigation decisions. This enables mobile robots to navigate flexibly and autonomously when performing quality inspection tasks in complex construction sites. This method solves the technical problems of existing inspection task language visual navigation methods being unable to fully understand human-issued natural language construction quality inspection navigation commands and lacking the ability to dynamically adjust navigation paths in complex and changing construction environments.

[0063] A smart construction quality inspection navigation method and quality inspection method based on a large language model include the following parts:

[0064] First, construct a quality inspection knowledge database to store the semantic vector representations and structured meta-information of standard clauses and various work manuals;

[0065] Second, extract building space information from the building information model to provide prior building environment information for inspection task navigation, and store it in the map database;

[0066] Third, using generative pre-trained large language models and retrieval-enhanced generation technology, a structured initial inspection plan is generated based on quality inspection instructions and extracted prior building spatial information and quality inspection knowledge database, and exported as a document (e.g., a JSON format file).

[0067] Fourth, analyze the initial inspection plan document and extract the navigation instructions; extract spatial visual features from multi-view images of the construction site perceived by the mobile robot, and fuse the navigation instructions to generate unified visual-language fusion features; establish candidate path points corresponding to each view image at the current location based on the point cloud information of the construction site perceived by the mobile robot.

[0068] Fifth, a large language model is used to process the visual-language fusion features and navigation instructions in the initial inspection plan. During the navigation process, a local spatial topology map is constructed, and dynamic navigation decisions are made based on this local spatial topology map until the entire inspection task is completed.

[0069] As attached Figure 2 As shown, the first part constructs a quality inspection knowledge database containing the latest construction quality standards, specifications, and quality assessment rules. Specifically, the method for establishing this quality inspection knowledge database includes:

[0070] Collect relevant documents for construction quality inspection, such as construction quality standards and specifications and quality assessment rules.

[0071] The relevant documents for construction quality inspection are preprocessed and divided into clauses; that is, the content of the relevant documents is divided into clauses, which can be done through content filtering, clause division, and word segmentation.

[0072] Specifically, the first step is to preprocess construction quality inspection documents, such as standard texts, including content filtering, clause division, and word segmentation. For example, the standard text is divided into natural paragraphs by line, and the paragraph type is determined based on punctuation: paragraphs ending with a period, question mark, exclamation mark, colon, or semicolon are identified as heading paragraphs, and the rest are non-heading paragraphs. Non-heading paragraphs are further checked to see if they are numbered paragraphs. Numbered paragraphs are usually located at the beginning of the paragraph, flush left, with a single Chinese character space between the number and the main text. Identified numbered paragraphs are stored independently, while unnumbered paragraphs are merged upwards to the previous non-heading paragraph. The content of each heading paragraph is automatically appended to the beginning of the non-heading paragraph below it until a new heading paragraph is identified, thus forming semantically complete clause units. Dividing the document content into clause units is called clause structure.

[0073] After the text structure is divided, a semantic embedding model is used to vectorize each text, extract its deep semantic features, and generate a high-dimensional knowledge vector representation with a unified dimension. This high-dimensional knowledge vector will serve as the basis for subsequent semantic retrieval and comparison based on semantic similarity. For example, the semantic embedding model uses the Bge-M3 vector embedding model to vectorize the text units.

[0074] Each clause of the construction quality-related specifications and assessment rules, i.e., the documents related to construction quality inspection, is ultimately stored as an independent knowledge node, as shown in the appendix. Figure 3 As shown, the node contains original text, high-dimensional knowledge vectors, and standardized metadata. The standardized metadata includes at least one of information such as file name, clause number, and document attributes, thereby supporting the acquisition of construction inspection knowledge based on operations such as conditional retrieval, filtering, and semantic matching, and serving the construction quality inspection command robot to perform semantic understanding and reasoning on the natural language issued by humans.

[0075] The second part allows exporting the project's Building Information Model (BIM) into an IFC format file, and extracting the building's spatial topology and layout information using algorithms, as shown in the attached document. Figure 4 and Figure 5 Using prior architectural spatial information, we can improve the localization and path planning capabilities of large language models in complex scenarios, and enhance the accuracy of generated inspection plans and navigation instructions.

[0076] The third part receives quality inspection task instructions in natural language from the user. First, using Retrieval Augmented Generation (RAG) technology, the task instructions are encoded based on a semantic model and input into a quality inspection knowledge database and a map database. A retrieval engine extracts knowledge vectors related to the natural language instructions from the quality inspection knowledge database, and simultaneously extracts the spatial topology and connectivity information of the project from the map database. The above information and task instructions are then rearranged, summarized, and fused for analysis. A unified set of prompt words is then constructed and input into a generative pre-trained large language model (LLM) to generate corresponding inspection paths and task execution plans, as shown in the appendix. Figure 6 As shown.

[0077] The chosen large language model is Deepseek-chat. The third part of the process is as follows: user inputs instructions → the task instructions are encoded using a semantic model (BGE-M3, etc.) → the retrieval module searches for relevant text in the database → the generator reads the knowledge vectors of these texts → outputs a more reliable and knowledge-supported inspection plan.

[0078] Quality inspection instructions, one such Figure 5 An example of the architectural space information shown is: "Inspect the general plastering quality of rooms 209, 211, and 218 on the 2nd floor of the teaching building project. The starting point and ending point are both the entrance door of the 2nd floor corridor." Based on this, according to the provided architectural space information, an initial inspection plan is output, including the inspection sequence, the global path node sequence, the inspection content, and the inspection standards. Navigation instructions for this task are also generated, and finally, an initial inspection plan containing the above content is formed in JSON file format.

[0079] An example of the initial inspection scheme output by a multimodal pre-trained large language model is as follows:

[0080] "Based on the received instructions, the system first searches the teaching building map to locate the target floor and designated room, extracting their spatial location and connecting paths. Then, it searches the quality inspection knowledge database for relevant regulatory clauses and quality assessment rules related to 'general plastering work quality' to determine the construction content and testing standards to be inspected. Combining spatial topology information and inspection knowledge, the system generates an initial inspection path and task planning scheme containing the following:"

[0081] Starting point: Enter the space [Corridor-2F] through the door [Door-id: 12206];

[0082] Inspection order: The inspection paths are sorted by spatial connectivity and shortest path by query, and the order is: Office-209 → Office-211 → Office-218;

[0083] Inspect the global path node sequence: By querying the doors and spatial path chains contained in each space, the sequence is determined as follows: [Door-id:12206]→ [Corridor-2F]→ [Door-id:12253]→ [Office-209]→ [Door-id:12253]→ [Corridor-2F]→ [Door-id:12280]→ [Office-211]→ [Door-id:12280]→ [Corridor-2F]→ [Door-id:12334]→ [Office-218]→ [Door-id:12334]→ [Corridor-2F]→ [Door-id: 12206];

[0084] Destination: Door [Door-id:12206];

[0085] Testing standards: Based on Articles 4.1 and 4.2 of GB 50210-2018 "Code for Acceptance of Construction and Decoration Engineering Quality", each item is bound to the original text of the corresponding article, the allowable deviation of the code and the testing method; the specific testing methods can be carried out by taking pictures, videos, radar scanning or with appropriate testing instruments, etc., and the corresponding testing method can be executed when reaching the corresponding global path point inside the space to be tested;

[0086] Inspection items: General plastering project quality inspection items include "surface quality, verticality of facade, surface flatness, squareness of internal and external corners, and straightness of dividing strips (joints), etc."

[0087] The navigation instructions corresponding to the initial inspection navigation path could be, for example:

[0088] 1. From the entrance of corridor F2, walk straight ahead until you reach door 12253;

[0089] 2. Enter office 209 through door 12253 and reach the preset global path point inside the space to be inspected. For example, walk towards the window and stop 2m in front of the window to scan the information of the interior walls of the room.

[0090] 3. Turn around and walk back to door 12253 to leave office 209, re-enter corridor F2, and stop;

[0091] 4. Continue straight ahead from corridor F2 until you reach door 12280;

[0092] 5. Enter office 211 from door 12280 and reach the preset global path point inside the space to be inspected. For example, walk towards the window and stop 2m in front of the window to scan the information of the interior walls of the room.

[0093] 6. Turn around and walk back to door 12280 to leave office 211, re-enter corridor F2, and stop;

[0094] 7. Continue straight ahead from corridor F2 until you reach door number 12334;

[0095] 8. Enter office 218 from number 12334 and reach the preset global path point inside the space to be inspected. For example, walk towards the window and stop 2m in front of the window to scan the wall information inside the room.

[0096] 9. Turn around and walk back to door 12334 to leave office 218, re-enter corridor F2, and stop;

[0097] 10. Proceed straight along corridor F2 back to the starting point.

[0098] The user should review and confirm the plan's content. Once confirmed and corrected, the system can output a JSON file of the initial inspection plan. The initial inspection navigation path consists of global path points. Based on building space information, global path points can be generated in relevant spaces, and the traversal order of these global path points can be planned. For example, in spaces not to be inspected, global path points can be generated at the entrance and exit. In spaces to be inspected, global path points can be generated at the entrance, exit, and inside. Internal global path points can be set in the middle of the space to be inspected, or in corners, etc. The specific location and number are not limited, with the aim of facilitating the completion of the corresponding inspection items.

[0099] In the fourth part, after parsing the initial inspection plan JSON file, the system extracts the navigation instructions, which are then executed by the on-site robot. During task execution, starting from the first global waypoint, the robot uses its onboard cameras to detect and provide feedback on multi-view images of the robot's current position at each step. Spatial visual features are extracted through a visual model and fused with the inspection task navigation instructions to generate unified visual-language fusion features.

[0100] Specifically, the initial inspection plan JSON file is first parsed to extract the navigation instruction text. Then, a language model (such as BERT) is used to segment and semantically encode the navigation instructions to obtain semantic features of the instructions with contextual structure. W The encoded object is the navigation instruction part (1-10) in the inspection plan generated above. That is, the navigation instruction corresponding to the current position on the initial inspection navigation path, that is, the navigation instruction between the two global path points where the current position is located.

[0101] Then, for each observed image, an encoding operation is performed using a pre-trained visual transformation model (e.g., ViT) to convert the image into a fixed-length visual feature representation vector, i.e., image features. V i This constitutes a multi-view image feature set. V (Representation vector of each image) V i Constructing a multi-view image feature set, for example V = V 1 , V 2 ,…, V n , n =8), obtain image semantic embedding, and realize the representation of the features of the real spatial image on site.

[0102] Multi-view images specifically refer to images captured at angles θ = 0°, 45°, 90°, 135°, 180°, 215°, 270°, and 315° between the current position and the current orientation, collectively forming the environmental perception information for the current position. In other words, the area surrounding the current position is divided into multiple candidate orientations, and a visual image is acquired for each candidate orientation. In this embodiment, navigation commands are encoded as command semantic features, and multi-view images are encoded as image features, achieving semantic unity of "language + vision," enabling navigation to have task target perception capabilities.

[0103] Finally, the semantic features of the received instructions are processed through a query-based attention model (Q-former). W ) and image features ( V Cross-modal alignment is performed to obtain visual-language fusion features representing the semantic fusion of "image + instruction" in the current state. H , H = H 1 , H 2 ,…, H n , n =8), as input to the navigation strategy model, such as Figure 7 As shown, a correspondence can be established between each viewpoint image and the semantic meaning of the command, enabling environmental understanding under semantic perception.

[0104] The fourth part also includes: acquiring real-time point cloud information at each step location between two adjacent global path points using LiDAR, and calculating candidate path points corresponding to the aforementioned multi-view images taken at that location, specifically including:

[0105] Acquire point cloud information around the current location, and build a local raster map of the current location based on the point cloud information;

[0106] Divide the candidate orientations around the current position that are consistent with the images from each viewpoint. Search the first preset neighborhood range of the candidate orientations based on the distance between the grid on the local grid map and the current position, the obstacle information on the grid, and the orientation information of the grid, and determine one of the grids as the next candidate path point corresponding to the candidate orientation.

[0107] For example, in a navigation strategy model, a local grid map of the current position is built based on the lidar sensor on the robot. The optimal target grid coordinates corresponding to the candidate orientation are calculated based on the grid map and used as the next path point on the candidate orientation.

[0108] Furthermore, one of the grid cells is selected as the candidate path point for the next step of the candidate orientation, specifically including:

[0109] Obstacle information of grid cells on a local grid map is obtained from point cloud information, and passable grid cells are obtained from obstacle information;

[0110] Select one of the passable grids within the first preset neighborhood of the candidate orientation as the next candidate path point corresponding to the candidate orientation; and make the probability of the passable grid being selected greater the greater the distance between the passable grid and the current position, the lower the probability of the passable grid colliding with the obstacle, and the smaller the deviation between the direction of the passable grid and the current position and the candidate orientation.

[0111] It can calculate the Euclidean distance between the passable grid and the current position, as well as the directional deviation index between the passable grid and the current position and the candidate orientation. It can also calculate the obstacle index of the passable grid based on the obstacle information on the local grid map. The Euclidean distance, directional deviation index and obstacle index are weighted and summed. The next candidate path point corresponding to the candidate orientation is selected based on the weighted sum of the three indices of the passable grid.

[0112] Specifically, each grid cell can be assigned a cost value representing the probability of a collision between the grid cell and an obstacle based on point cloud information; the average cost value of all grid cells within a second preset neighborhood of a passable grid cell can be obtained, and the obstacle index of the passable grid cell can be obtained based on the average value. A navigation module (e.g., a move-base navigation package) can be used to assign a cost value to each grid cell based on point cloud information.

[0113] For example, the cost value of each grid in a local grid map ranges from [0, 255]. The specific value is assigned based on the probability of a robot colliding with the grid, according to the following rules: ① Fatal obstacle: The grid cost value is 254. In this case, the obstacle overlaps with the center of the robot, and a collision is inevitable. ② Inscribed obstacle: The grid cost value is 253. In this case, when the robot is on the grid, the obstacle is inside the robot's inscribed circle, and a collision is inevitable. ③ Outscribed obstacle: The grid cost value is [128, 252]. In this case, the obstacle is inside the robot's outscribed circle and is at the collision threshold, but a collision is not guaranteed. ④ Non-free space: The grid value is (0, 127). In this case, the robot is near the obstacle and belongs to the danger warning zone. Entering this area may result in a collision. ⑤ Free area: The grid value is 0. The robot can pass freely here. ⑥ Unknown area: The grid value is 255. It has not yet been determined whether there are obstacles.

[0114] In one specific embodiment, taking the current robot pose (position (x0, y0) and orientation θ degrees) as the origin, a fan-shaped search area is extended in the direction of θ ± 22.5° to define the range of candidate positions, i.e., the first preset neighborhood range of candidate orientations θ. The radius of the fan is the maximum single-step distance of the robot (e.g., set to 2 meters). All passable grids within the fan-shaped area (provided by a local grid map with a resolution of 0.2) are traversed, and the orientation angles and distances of these passable grids relative to the current position are sent to the system.

[0115] For each passable grid cell, its comprehensive score is calculated based on three core metrics, including distance. (Euclidean distance between the passable grid and the current position; greater distance is better); obstacle index (For example, construct a fixed-size window (10×10) centered on a passable grid, calculate the average cost value within its neighborhood to represent the surrounding obstacle density; a smaller value is better. Obtain obstacle indices based on the average cost value); Orientation deviation index (The angular deviation between the passable grid direction and the selected candidate orientation direction θ is acceptable. The smaller the deviation, the better. The orientation deviation index can be obtained based on the angular deviation.)

[0116] The three indicators mentioned above are normalized separately, and then weighted and merged according to the set weight coefficients to calculate the total score of each passable grid as follows:

[0117] ;

[0118] Specifically, since smaller obstacle density and angle are better, the average value of the grid cost within the second preset area of ​​the passable grid needs to be normalized and then reverse-processed (i.e., represented by the difference between 1 and the normalized value) to obtain the obstacle index; the angle deviation between the passable grid direction and the corresponding candidate orientation needs to be normalized and then reverse-processed to obtain the direction deviation index. The grid coordinates with the highest score are used as the next candidate path point on the corresponding candidate orientation, that is, if the next step selects this candidate orientation, the path will move along this candidate orientation to the corresponding candidate path point.

[0119] Part 5 involves acquiring real-time situational information at each step between two adjacent global pathpoints, selecting the next target pathpoint using a visual-language navigation method based on this information, and performing local path planning between adjacent global pathpoints. This also includes:

[0120] The system acquires visual images of multiple candidate orientations at the current location and extracts image features from each. Simultaneously, it extracts the instruction semantic features of the navigation command corresponding to the current location in the initial inspection navigation path. The image features corresponding to the candidate orientations and the instruction semantic features are then fused to obtain visual-language fusion features.

[0121] Based on the visual-language fusion features of the candidate orientation, the orientation direction features, and the current step number, multi-dimensional feature vectors are established for the currently reached path points and candidate path points, respectively.

[0122] The node representation vectors of the reached path points and the candidate path points are fused together to obtain the node representation vectors of the reached path points and the candidate path points.

[0123] Based on the node representation vectors of candidate path points, a scoring model is used to select the candidate path point with the highest score as the target path point for the next step at the current position.

[0124] In some specific embodiments, the instruction semantic features ( W ) and visual-language fusion features ( H The input is fed into a large language model (LLM) to construct a spatial topology graph, and navigation reasoning and decision-making are performed based on the graph, specifically including:

[0125] (1) First, use a Large Language Model (LLM) to extract the semantic features of the instructions ( W ) and visual-language fusion features ( H As input, the output is a new instruction semantic feature used for navigation reasoning. W' ) and visual-language fusion features ( H' The selected Large Language Model (LLM) is T5, which is used to update the semantic features of instructions and the visual-language fusion feature dimensions, so as to facilitate the subsequent construction of multi-dimensional feature vectors.

[0126] (2) Spatial Topology Graph Establishment and Update Module: During navigation, a dynamic spatial topology graph consisting of nodes and edges is gradually established and maintained as a navigation memory mechanism. This graph records the spatial states experienced by the robot and serves as the basis for navigation reasoning, helping to select the next target node. In the spatial topology graph, nodes (N) represent positions, and edges (E) represent paths between different positions. The topology graph constructed at time t is denoted as G. t =(N t E t ).

[0127] The nodes of the spatial topology graph are divided into reached path points and candidate path points for the current location, with each path point representing a location point. Path points have multiple visual observation perspectives, using updated visual-linguistic fusion features (...). H' In addition, each viewpoint is associated with orientation embedding (D, which is the encoding of the orientation corresponding to each current image viewpoint) and step embedding (S, which reflects the current task progress, i.e., the encoding of which step the current node is in the entire path sequence).

[0128] Based on the visual-language fusion features, orientation features, and current step count of multiple candidate orientations at the current location, a multi-dimensional feature vector of the currently reached path point is obtained. Then, based on the visual-language fusion features and orientation features of the reached path points corresponding to the candidate orientations, a multi-dimensional feature vector of the candidate path point corresponding to the candidate orientation is established. The current candidate orientation of the current location (i.e., the currently reached path point) may belong to other reached path points simultaneously; in this case, the reached path point to which the current candidate orientation belongs is the reached path point corresponding to that candidate orientation.

[0129] By establishing multi-dimensional feature vectors for reached and candidate path points, directional and step embeddings can be added to the visual-linguistic fusion features of each path point to jointly enhance the node's structural awareness. The construction of the spatial topology graph establishes a graph structure of path points and edges, providing a memory framework for navigation history and feasible path structures. This forms the basis for subsequent graph reasoning and facilitates source tracing and error correction.

[0130] Specifically, the quantity that has reached the path point. N (The average pooling feature across all observation views) is obtained using the following formula:

[0131] ;

[0132] In the formula, n Indicates the number of candidate orientations; Indicates the first iVisual-linguistic fusion features of candidate orientations; D i Indicates the first i Orientation direction features of each candidate orientation; S Indicates the current step number.

[0133] Where the orientation is embedded:

[0134] ;

[0135] h Each image's angle relative to the current orientation is converted to radians, with 0° corresponding to... 45° corresponds to 90° corresponds to wait.

[0136] MLP stands for Multilayer Perceptron, which maps the input vector to the same dimension as the visual-semantic embedding; LayerNorm represents the normalization process of the embedding result to ensure the stability of the feature distribution.

[0137] Step embedding:

[0138] ;

[0139] in t The current position is represented by its number (step number) in the inspection path. It is an integer and is encoded as a representation in the same dimension as the visual-semantic embedding.

[0140] The features of candidate path points (unreached path points) are composed of the viewpoint images of the corresponding reached path points facing them, and the average pooling features of the corresponding orientation embedding and step embedding. Assume there are... m If a candidate path point can be observed by all already reached path points, then the multidimensional feature vector of the candidate path point... N i Specifically, it can be obtained through the following formula:

[0141] ;

[0142] In the formula, m This indicates the number of reached path points corresponding to the candidate orientation of a candidate path point; Indicates the candidate orientation corresponding to the first j Visual-linguistic fusion features of each reached path point in the candidate orientation; D j This indicates the candidate orientation to which the candidate path point belongs, and the corresponding candidate orientation. j The orientation features of the reached path points. The embedding of steps for unreached nodes is 0.

[0143] Then, a self-attention module (e.g., the Transformer model) is used to process the features of all nodes in the spatial topology graph to obtain a global node representation matrix with semantic context, i.e., node representation vectors. At this point, each node's features no longer consist solely of its own node features, but are fused with information from all other nodes through a self-attention mechanism. By performing self-attention propagation on reached and candidate path points, context-enhanced node representation vectors are output, capturing spatial dependencies and semantic relationships between nodes, thus enabling each node representation to possess global contextual information.

[0144] Then, based on the node representation vector, a scoring model is used to select the candidate path point with the highest score as the next target path point for the current position. This also includes:

[0145] The node representation vectors of arrived path points and candidate path points are cross-modal aligned with the instruction semantic features corresponding to the current position. Then, the features between arrived path points and candidate path points are further fused using the Graph Self-Attention (GASA) mechanism to obtain the final graph node features of arrived path points and candidate path points, which are then input into the scoring model. These final graph node features are task-relevant final features used for navigation decisions.

[0146] The final features of the graph nodes are obtained as follows:

[0147] Cross-modal alignment of graphs and navigation instructions is achieved through representations of global nodes with semantic context. As a query vector, with updated instruction semantic features For key-value pairs, a cross-modal attention mechanism is used to compute the semantic association between nodes and instructions, outputting a node representation that is sensitive to task semantics. This process can be expressed as:

[0148] ;

[0149] , , It is a learnable parameter matrix; d The vector dimension is represented. By aligning global node representation features with instruction semantic features across modalities, task-related node representations are output, which enhances the matching degree between the current node and the task target, making the scoring task-sensitive.

[0150] Then, the spatial structure information and linguistic semantic relationships between nodes are further fused using the graph self-attention mechanism (GASA) to obtain the final graph node features with task relevance for navigation decision-making. The process is expressed as follows:

[0151] ;

[0152] , , It is a learnable parameter matrix. d Represents the vector dimension. Spatial bias term, i.e., the spatial distance matrix between nodes.

[0153] Then, the final state representation vector of each candidate path point is... Input the scoring model, calculate the score of each node as the next navigation target, compare it with the inspection path defined by the expert, select the node with the highest score as the next navigation target, and advance the inspection path sequence in sequence.

[0154] The scoring model is as follows:

[0155] The scoring model is a multilayer perceptron network (MLP) used to receive... Calculate the score , Indicates the first i The score of each candidate node is selected as the next navigation target;

[0156] ;

[0157] After normalization, the scores output the probability that each candidate node is selected as the next navigation target. The specific normalization formula is as follows:

[0158] ;

[0159] The scoring model takes the node representation vectors of candidate path points as input and outputs the scores of the candidate path points; during the training of the scoring model, the target nodes of each step in the inspection path are defined by experts. y * Supervised learning is performed by minimizing the cross-entropy loss (loss function) of the negative log-predicted probability. L Specifically as follows:

[0160] ;

[0161] in, This is the predicted probability of the target node, obtained by normalizing the scores of all candidate path points. The scores of all candidate path points are normalized and converted into a set of probabilities. (

[0162] The next node selects the highest score: , Indicate steps tThe next selected target node is then identified. A scoring model outputs scores for candidate path points, providing a differentiable and supervised-training scoring mechanism for selecting the optimal path forward direction. It can also output the next optimal navigation node, enabling strategy decision-making for each navigation step. By comparing predicted probabilities with expert path labels, model parameters are optimized, allowing the scoring model to gradually learn to optimally score the target direction / position during training, ensuring the rationality and interpretability of path generation.

[0163] The system also supports path backtracking and error correction mechanisms. When path planning fails or the site becomes impassable, it automatically reverts to the previous node and re-evaluates the path to improve robustness and generalization capabilities in real-world inspection scenarios. Specifically, the path backtracking and error correction mechanisms work as follows: when the next navigation node cannot be reached, local replanning is first performed to attempt to bypass the obstacle; if this fails, a revert to the previous navigation node is attempted for re-evaluation; if all error correction actions fail, remote assistance is requested for manual intervention, and the current progress status is saved to support task recovery.

[0164] Repeat the above steps until the inspection task is completed.

[0165] This embodiment also provides a construction site quality inspection and navigation system. The system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the construction site quality inspection and navigation method described above.

[0166] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A construction site quality inspection navigation method, characterized in that, The method comprises the following steps: generating an initial inspection navigation path according to a quality inspection instruction and building space information where a construction site is located; wherein the building space information comprises the spatial composition inside the building and the connection relationship between spaces, and the quality inspection instruction comprises at least one space to be inspected; based on global path points on the initial inspection navigation path, overall path planning is performed, and real-time site information is obtained at each step between adjacent two global path points, and local path planning between the adjacent two global path points is performed according to the real-time site information by using a visual navigation method; at each step between adjacent two global path points, real-time site information is obtained, and local path planning between the adjacent two global path points is performed according to the real-time site information by using a visual navigation method, which specifically comprises: obtaining point cloud information around the current position, and establishing a local grid map of the current position according to the point cloud information; dividing a plurality of candidate orientations around the current position, searching a first preset neighborhood range of the candidate orientations according to the distance between the grid on the local grid map and the current position, obstacle information on the grid, and direction information of the grid, and determining one grid as a next candidate path point corresponding to the candidate orientation; at each step between adjacent two global path points, real-time site information is obtained, and local path planning between the adjacent two global path points is performed according to the real-time site information by using a visual navigation method, which further comprises: obtaining visual images on a plurality of candidate orientations of the current position and extracting image features respectively, and extracting instruction semantic features of a corresponding navigation instruction of the current position in the initial inspection navigation path, and fusing the image features corresponding to the candidate orientations and the instruction semantic features to obtain visual-linguistic fusion features; establishing a multi-dimensional feature vector of the current reached path point and the candidate path point according to the visual-linguistic fusion features of the candidate orientation, the orientation direction features, and the current step number; fusing the multi-dimensional feature vectors of the reached path point and the candidate path point to obtain node representation vectors of the reached path point and the candidate path point; based on the node representation vectors of the candidate path points, a scoring model is used to select a candidate path point with the highest score as the target path point of the next step of the current position.

2. The construction site quality inspection navigation method of claim 1, wherein, generating the initial inspection navigation path specifically comprises: receiving the quality inspection instruction in natural language by using a generative pre-training large language model, and combining a retrieval enhanced generation technology with a retrieval result of the building space information to generate the initial inspection navigation path.

3. The construction site quality inspection navigation method of claim 1, wherein, determining one grid as a next candidate path point corresponding to the candidate orientation, specifically comprising: obtaining obstacle information of the grid on the local grid map according to the point cloud information, and obtaining passable grids according to the obstacle information; selecting one passable grid in the first preset neighborhood range of the candidate orientation as the next candidate path point corresponding to the candidate orientation; and the farther the distance between the passable grid and the current position, the lower the probability of collision between the passable grid and the obstacle, and the smaller the deviation between the direction between the passable grid and the current position and the candidate orientation, the greater the probability of the passable grid being selected.

4. The construction site quality inspection navigation method of claim 1, wherein, A multi-dimensional feature vector of the current reached path point N Specifically, it is obtained by the following formula: ； In the formula, n denotes the number of candidate orientations; denotes the visual-linguistic fusion feature of the i th candidate orientation; D i representing a candidate orientation of the object; and i an orientation direction feature representing a candidate orientation of the object; i>S represents the current step number; Multi-dimensional feature vector of candidate waypoint N i Specifically, the following formula is used: ； In the formula, m represents the number of reached path points corresponding to the candidate orientation to which the candidate path point belongs; represents the visual-linguistic fusion feature of the candidate orientation on the reached path point corresponding to the candidate orientation; j the candidate orientation. D j representing a candidate orientation to which the candidate path point belongs, and a direction feature of the candidate path point with respect to a candidate orientation corresponding to the candidate orientation. j a direction feature of the candidate path point with respect to a candidate orientation corresponding to the candidate orientation.

5. The construction site quality inspection navigation method of claim 1, wherein, The scoring model is as follows: The scoring model takes the node representation vector of the candidate path point as input and outputs the score of the candidate path point; during training of the scoring model, according to the target node of each step in the defined inspection path by an expert y * Supervised learning is performed by minimizing the cross-entropy loss of the negative logarithmic prediction probability L Specifically as follows: ； wherein, is the predicted probability of the target node, obtained by normalizing all candidate path point scores.

6. A construction site quality inspection method, characterized by, The construction site quality inspection navigation method according to any one of claims 1-5 is used for quality inspection path planning, and further includes: generating a quality inspection scheme according to the quality inspection instruction and a pre-established quality inspection knowledge database; wherein the quality inspection instruction further includes a quality inspection item; when reaching a global path point in the space to be inspected, performing quality inspection according to the quality inspection scheme.

7. The construction site quality inspection method of claim 6, wherein, The method for establishing the quality inspection knowledge database is as follows: collecting construction quality inspection related files; preprocessing the construction quality inspection related files to divide them into article structures; after the article structure division is completed, using a semantic embedding model to perform vector coding on each article, extract deep semantic features, and generate high-dimensional knowledge vectors of a unified dimension; each article of the construction quality inspection related files is finally stored as an independent knowledge node, and the node contains original text, high-dimensional knowledge vectors and standardized metadata, and the standardized metadata includes at least one of a file name, an article number and document attribute information.

8. A construction site quality inspection navigation system, characterized by, The system includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to execute the construction site quality inspection navigation method according to any one of claims 1-5.

Images