Embedded part detection method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of building construction detection, in particular to an embedded part detection method and device, electronic equipment and a storage medium. According to the embedded part detection method, image acquisition needs to be carried out on a construction environment to obtain a target image sequence; wherein the target image sequence comprises a plurality of environment local images with different environment view-finding angles; inputting the target image sequence into an embedded part detection model for azimuth detection to obtain embedded part positioning information corresponding to each target embedded part; performing three-dimensional reconstruction according to the plurality of environment local images in the target image sequence to obtain environment three-dimensional point cloud data; determining embedded part point cloud data of each target embedded part according to the embedded part positioning information and the environment three-dimensional point cloud data; and performing parameter detection based on the embedded part point cloud data of each target embedded part to obtain an embedded part detection parameter of each target embedded part. More detailed and accurate embedded part parameter information is provided, and the requirement for construction quality detection is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of building construction detection, in particular to a pre-embedded part detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] The construction field has strict requirements for the accurate detection of pre-embedded parts. Traditional detection methods mainly include total station measurement and manual line cone measurement. As a precision instrument, the total station measures the distance by emitting a laser beam and receiving the reflected signal, and determines the three-dimensional coordinates of the pre-embedded part in combination with angle measurement. Manual line cone measurement relies on the experience and technology of construction personnel, and measures the deviation by setting control lines, hanging line cones and visually or using simple tools, to determine whether the position and perpendicularity of the pre-embedded part meet the design requirements.

[0003] In related technologies, these two methods play a certain role in different construction scenarios, but as the construction industry continues to improve the efficiency requirements, the traditional methods are difficult to balance the accuracy, cost and data management of pre-embedded part detection, and therefore it is difficult to meet the efficiency requirements of pre-embedded part detection. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes a pre-embedded part detection method and device, electronic equipment and storage medium, which can better balance the accuracy, cost and data management of pre-embedded part detection, and better meet the efficiency requirements of pre-embedded part detection.

[0005] The pre-embedded part detection method according to the first aspect of the present application comprises:

[0006] Image acquisition is performed on a construction environment containing a plurality of target pre-embedded parts to obtain a target image sequence; wherein the target image sequence contains a plurality of environmental local images of different environmental viewing angles;

[0007] The target image sequence is input into a pre-trained pre-embedded part detection model for orientation detection to obtain pre-embedded part positioning information corresponding to each target pre-embedded part;

[0008] Three-dimensional reconstruction is performed according to a plurality of environmental local images in the target image sequence to obtain environmental three-dimensional point cloud data;

[0009] According to the pre-embedded part positioning information and the environmental three-dimensional point cloud data, pre-embedded part point cloud data matching each target pre-embedded part is determined;

[0010] Parameter detection is performed based on the pre-embedded part point cloud data of each target pre-embedded part to obtain pre-embedded part detection parameters matching each target pre-embedded part.

[0011] According to some embodiments of the present application, the image collection for the construction environment containing a plurality of target embedded parts obtains a target image sequence, including:

[0012] A plurality of original view angles are determined for the construction environment; wherein each original view angle is used to cover the complete view of the construction environment;

[0013] Based on each original view angle, a corresponding view simulation picture is determined;

[0014] The overlap rate calculation is performed on each two adjacent view simulation pictures to obtain a corresponding simulation picture overlap rate;

[0015] In response to the existence of the simulation picture overlap rate not meeting the preset adjacent overlap constraint condition, the corresponding original view angle is adjusted to re-determine the two adjacent view simulation pictures;

[0016] In response to each simulation picture overlap rate corresponding to the construction environment reaching the adjacent overlap condition, the adjusted original view angle is determined as the environment view angle;

[0017] Based on each environment view angle, image collection is performed for the construction environment to obtain the target image sequence.

[0018] According to some embodiments of the present application, the target image sequence is input into a pre-trained embedded part detection model for orientation detection to obtain the embedded part positioning information corresponding to each target embedded part, including:

[0019] The target image sequence is input into the embedded part detection model;

[0020] Each environment local image is segmented in the embedded part detection model to obtain a corresponding embedded part image area;

[0021] Edge extraction is performed on each embedded part image area in the embedded part detection model to obtain the embedded part positioning information corresponding to each target embedded part.

[0022] According to some embodiments of the present application, the edge extraction is performed on each embedded part image area to obtain the embedded part positioning information corresponding to each target embedded part, including:

[0023] Minimum rectangular edge extraction is performed on each embedded part image area to obtain corresponding minimum rectangular edge information;

[0024] Extract an angular point orientation parameter from the minimum rectangular edge information of each of the embedded part image regions as the embedded part positioning information corresponding to each of the target embedded parts.

[0025] According to some embodiments of the present application, the target image sequence is configured with multi-view image constraint information for each of the environment local images, and the three-dimensional reconstruction according to the plurality of environment local images in the target image sequence obtains environment three-dimensional point cloud data, including:

[0026] Feature extraction is performed on each of the environment local images in the target image sequence to obtain key feature points of each of the environment local images.

[0027] Point cloud construction is performed based on the key feature points of each of the environment local images to obtain the environment three-dimensional point cloud data.

[0028] According to some embodiments of the present application, the feature extraction performed on each of the environment local images in the target image sequence to obtain key feature points of each of the environment local images includes:

[0029] Feature extraction is performed on each of the environment local images in the target image sequence to obtain original feature points corresponding to each of the environment local images.

[0030] Correlation matching is performed on the original feature points corresponding to each of the environment local images to obtain correlation feature points corresponding to each of the environment local images.

[0031] Geometric verification is performed based on the correlation feature points corresponding to an environment local image.

[0032] The correlation feature points that pass the geometric verification are determined as the key feature points corresponding to the environment local image.

[0033] According to some embodiments of the present application, the point cloud construction performed based on the key feature points of each of the environment local images to obtain the environment three-dimensional point cloud data includes:

[0034] Two of the environment local images are selected from the target image sequence, and initial construction is performed on the key feature points corresponding to the two environment local images to obtain initial environment point cloud data.

[0035] From the next environment local image in the target image sequence, incremental reconstruction is performed on the key feature points corresponding to the environment local image and the environment point cloud data to update the environment point cloud data.

[0036] in response to the environment local images in the target image sequence not participating in the incremental reconstruction, returning a next environment local image from the target image sequence, and performing incremental reconstruction on the environment point cloud data according to the key feature points corresponding to the environment local image;

[0037] in response to the environment local images in the target image sequence participating in the incremental reconstruction, determining the environment point cloud data as the environment three-dimensional point cloud data.

[0038] According to some embodiments of the present application, the calibration board is placed in the construction environment, and the point cloud construction based on the key feature points of each environment local image obtains the environment three-dimensional point cloud data, which comprises:

[0039] obtaining reference calibration information matched with the calibration board;

[0040] performing point cloud construction based on the key feature points of each environment local image to obtain intermediate point cloud data; wherein the intermediate point cloud data comprises calibration board point cloud data;

[0041] performing scale conversion on the intermediate point cloud data based on the calibration board point cloud data and the reference calibration information to obtain the environment three-dimensional point cloud data.

[0042] According to some embodiments of the present application, the embedded part positioning information comprises an embedded part detection frame in the environment local image, and the determination of the embedded part point cloud data matched with each target embedded part based on the embedded part positioning information and the environment three-dimensional point cloud data comprises:

[0043] projecting the environment three-dimensional point cloud data into each environment local image to obtain a corresponding point cloud projection image; wherein the environment three-dimensional point cloud data comprises a plurality of key feature points;

[0044] extracting the key feature points projected in the embedded part detection frame of each point cloud projection image to obtain the embedded part point cloud data matched with each target embedded part.

[0045] According to some embodiments of the present application, the parameter detection based on the embedded part point cloud data of each target embedded part obtains embedded part detection parameters matched with each target embedded part, which comprises:

[0046] performing plane fitting operation on the embedded part point cloud data of each target embedded part to obtain a fitted construction plane;

[0047] from a plurality of environment view angles, determining a parameter detection angle for each target embedded part that satisfies a preset direct facing condition;

[0048] According to each parameter detection angle, parameter calculation is performed on the corresponding target embedded part in the fitting construction plane to obtain the embedded part detection parameter matched to each target embedded part.

[0049] According to some embodiments of the present application, a calibration board is placed in the construction environment, and according to each parameter detection angle, parameter calculation is performed on the corresponding target embedded part in the fitting construction plane to obtain the embedded part detection parameter matched to each target embedded part, including:

[0050] Reference calibration information matched to the calibration board is obtained.

[0051] According to each parameter detection angle, parameter calculation is performed on the corresponding target embedded part in the fitting construction plane to obtain the preliminary estimation parameter of each target embedded part.

[0052] The preliminary estimation parameter is scaled based on the reference calibration information to obtain the embedded part detection parameter of each target embedded part.

[0053] According to the second aspect of the present application, the embedded part detection device includes:

[0054] An image acquisition module is configured to acquire images for a construction environment containing a plurality of target embedded parts to obtain a target image sequence; wherein the target image sequence contains a plurality of environment local images of different environment viewing angles.

[0055] An embedded part orientation detection module is configured to input the target image sequence into a pre-trained embedded part detection model to perform orientation detection and obtain embedded part positioning information corresponding to each target embedded part.

[0056] A three-dimensional reconstruction module is configured to perform three-dimensional reconstruction according to a plurality of environment local images in the target image sequence to obtain environment three-dimensional point cloud data.

[0057] A point cloud extraction module is configured to determine embedded part point cloud data matched to each target embedded part according to the embedded part positioning information and the environment three-dimensional point cloud data.

[0058] A parameter detection module is configured to perform parameter detection based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matched to each target embedded part.

[0059] In a third aspect, the present application provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the embedded part detection method according to any one of the first aspect of the present application.

[0060] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the storage medium storing a program, and the program being executed by a processor to implement the pre-embedded part detection method according to any one of the embodiments of the first aspect of the present application.

[0061] The pre-embedded part detection method and device, the electronic device, and the storage medium according to the embodiments of the present application have at least the following beneficial effects:

[0062] According to the pre-embedded part detection method of the embodiments of the present application, image collection is first performed on a construction environment containing a plurality of target pre-embedded parts to obtain a target image sequence; the target image sequence contains a plurality of environment local images of different environment view angles; the target image sequence is input into a pre-trained pre-embedded part detection model for orientation detection to obtain pre-embedded part positioning information corresponding to each target pre-embedded part; three-dimensional reconstruction is performed on the plurality of environment local images in the target image sequence to obtain environment three-dimensional point cloud data; pre-embedded part point cloud data matched to each target pre-embedded part is determined according to the pre-embedded part positioning information and the environment three-dimensional point cloud data; and parameter detection is performed based on the pre-embedded part point cloud data of each target pre-embedded part to obtain pre-embedded part detection parameters matched to each target pre-embedded part. In this way, more detailed and accurate pre-embedded part parameter information can be provided to meet the requirements of construction quality detection.

[0063] Additional aspects and advantages of the application will be set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the following and / or by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0064] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the description of the embodiments, taken in conjunction with the following drawings in which:

[0065] Figure 1 A flowchart of a pre-embedded part detection method according to an embodiment of the present application is shown in FIG. 1;

[0066] Figure 2 Another flowchart of a pre-embedded part detection method according to an embodiment of the present application is shown in FIG. 2;

[0067] Figure 3 Another flowchart of a pre-embedded part detection method according to an embodiment of the present application is shown in FIG. 3;

[0068] Figure 4 Another flowchart of a pre-embedded part detection method according to an embodiment of the present application is shown in FIG. 4;

[0069] Figure 5 Another flowchart of a pre-embedded part detection method according to an embodiment of the present application is shown in FIG. 5;

[0070] Figure 6 Another flowchart of a pre-embedded part detection method provided by an embodiment of the present application is shown in FIG. 6;

[0071] Figure 7 Another flowchart of a pre-embedded part detection method provided by an embodiment of the present application is shown in FIG. 6;

[0072] Figure 8 Another flowchart of a pre-embedded part detection method provided by an embodiment of the present application is shown in FIG. 6;

[0073] Figure 9 Another flowchart of a pre-embedded part detection method provided by an embodiment of the present application is shown in FIG. 6;

[0074] Figure 10 Another flowchart of a pre-embedded part detection method provided by an embodiment of the present application is shown in FIG. 6;

[0075] Figure 11 Another flowchart of a pre-embedded part detection method provided by an embodiment of the present application is shown in FIG. 6;

[0076] Figure 12 Another flowchart of a pre-embedded part detection method provided by an embodiment of the present application is shown in FIG. 6;

[0077] Figure 13 Another flowchart of a pre-embedded part detection method provided by an embodiment of the present application is shown in FIG. 6; DETAILED DESCRIPTION

[0078] The embodiments of the present application are described in detail below with reference to the accompanying drawings. The embodiments described below are examples for explaining the present application and should not be understood as limiting the present application.

[0079] In the description of the present application, one or more of several means two or more, greater than, less than, more than, etc. are understood as not including the number, above, below, etc. are understood as including the number. If it is described as first, second, it is only used to distinguish the technical features for the purpose, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the sequence of indicated technical features.

[0080] In the description of the present application, it is understood that the orientation description, such as up, down, left, right, front, back, etc. indicates the orientation or position relationship based on the orientation or position relationship shown in the drawings, which is only for the purpose of describing the present application and simplifying the description, and is not intended to indicate or imply that the device or element indicated must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application.

[0081] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the exemplary description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0082] In the description of the present application, it should be noted that, unless otherwise explicitly limited, the words such as setting, installing, connecting and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution. In addition, the identification of specific steps in the following does not represent the limitation of the order and execution logic of the steps, and the execution order and execution logic between the steps should be understood and inferred with reference to the content expressed in the embodiments.

[0083] The field of building construction has strict requirements for the accurate detection of embedded parts. Traditional detection methods mainly include total station measurement and manual placement of line cone measurement.

[0084] As a kind of precision instrument, total station measures distance by emitting laser beam and receiving reflected signal, and determines the three-dimensional coordinates of embedded parts combined with angle measurement, and is widely used in building construction and topographic surveying.

[0085] Manual line cone measurement relies on the experience and technology of construction personnel, and measures the deviation by setting control line, placing line cone and visually or using simple tools, to judge whether the position and perpendicularity of embedded parts meet the design requirements.

[0086] These two methods have played a certain role in different construction scenes, but as the requirements of the construction industry for construction efficiency, cost control and data management continue to improve, the traditional methods gradually expose a series of problems, which are difficult to meet the needs of modern building construction.

[0087] The total station instrument measurement is inefficient. Multiple professionals are required to cooperate and measure point by point, and the average time consumption of each wall is about 1 hour. In large engineering projects, this inefficient measurement method will cause project delay and is difficult to meet the requirements of rapid construction. Secondly, the total station instrument equipment is expensive, and the purchase and maintenance cost is high. At the same time, multiple professionals are required to participate in measurement and post-data analysis, which further increases the labor cost. In addition, the data collected by the total station instrument needs to be processed in a complex post-processing to generate a detailed detection report. This process not only consumes time, but also requires professional technical personnel to operate, which increases the complexity and cost of the project. Finally, the operation of the total station instrument is relatively complex, and it requires specially trained professionals to use it skillfully, which is a big challenge for on-site construction personnel, especially in the case of tight construction period, it is difficult to ensure that all operations can be accurately completed.

[0088] The manual line cone placement measurement method relies on simple tools such as line cones and tapes, which have limited accuracy and cannot meet the high-precision requirements of modern building engineering. At the same time, due to the reliance on manual operation, the measurement results are easily affected by factors such as operator skill level and vision, resulting in large errors in the measurement results. In addition, manual line cone placement measurement is inefficient, requiring multiple workers to climb and cooperate, with an average time consumption of about 2 hours per wall. Each measurement requires manual adjustment of the position and direction of the line cone, and multiple readings and records, which is complex and time-consuming. Moreover, manual line cone placement measurement lacks an effective data management system, and measurement results can be recorded manually on paper forms or electronic documents, which is prone to writing errors or data loss, making it difficult to achieve efficient storage, query and analysis of data, affecting subsequent quality traceability and improvement. Finally, this method is highly dependent on the experience and technical level of construction personnel, and the operation results of different personnel may have large differences, making it difficult to ensure standardization and consistency.

[0089] It can be seen that the traditional embedded part detection method has many shortcomings in precision, efficiency, cost and data management, and cannot meet the needs of modern building construction.

[0090] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes a pre-embedded part detection method and device, electronic equipment and storage medium, which can better balance the precision, cost and data management of pre-embedded part detection, and better meet the efficiency requirements of pre-embedded part detection.

[0091] Further explanation is made below with reference to the accompanying drawings.

[0092] Reference Figure 1 According to the pre-embedded part detection method of the present application, the method can include:

[0093] Step S101, image collection is performed on a construction environment containing several target embedded parts to obtain a target image sequence; wherein the target image sequence contains multiple environmental local images of different environmental perspective angles;

[0094] Step S102, the target image sequence is input into a pre-trained embedded part detection model for orientation detection to obtain embedded part positioning information corresponding to each target embedded part;

[0095] Step S103, three-dimensional reconstruction is performed according to the multiple environmental local images in the target image sequence to obtain environmental three-dimensional point cloud data;

[0096] Step S104, according to the embedded part positioning information and the environmental three-dimensional point cloud data, embedded part point cloud data matched to each target embedded part is determined;

[0097] Step S105, parameter detection is performed based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matched to each target embedded part.

[0098] Step S101 of some embodiments, image collection is performed on a construction environment containing several target embedded parts to obtain a target image sequence; wherein the target image sequence contains multiple environmental local images of different environmental perspective angles;

[0099] It should be noted that step S101 is the starting point of the entire process, and its core task is to perform image collection on a construction environment containing several target embedded parts to obtain a target image sequence. The key of this step is to ensure that the collected images can fully cover the positions of all embedded parts in the construction environment, and multiple environmental local images are obtained from different environmental perspective angles. This multi-angle image collection method can provide rich and comprehensive data basis for subsequent embedded part detection and three-dimensional reconstruction, ensuring that no important details are missed.

[0100] Specifically, the image collection process needs to consider the complexity of the construction environment and the distribution of the embedded parts. The construction environment is often full of various obstacles and interference factors, such as steel bars, formworks, construction tools, etc., which may block the embedded parts or affect the quality of the images. Therefore, when collecting images, it is necessary to take pictures from multiple different angles and positions to ensure that each embedded part can be clearly visible in at least one image. At the same time, different perspective angles can also provide more information about the embedded parts in three-dimensional space, which is helpful for subsequent three-dimensional reconstruction and accurate positioning.

[0101] The target image sequence includes a plurality of environment partial images, each of which focuses on a specific area in the construction environment. These environment partial images not only help to reduce the complexity of data processing, but also improve the accuracy of detection. Because the environment partial image can more clearly display the details of the embedded part and its surrounding environment, the features of the embedded part are more prominent, which facilitates the recognition and analysis of the detection model.

[0102] Referring to Figure 2 According to some embodiments of the present application, step S101 performs image acquisition on a construction environment containing several target embedded parts to obtain a target image sequence, which can include:

[0103] Step S201 determines a plurality of original view angles for the construction environment; wherein each original view angle is used to cover the complete view of the construction environment.

[0104] Step S202 determines a corresponding view simulation picture based on each original view angle.

[0105] Step S203 calculates the overlap rate of each two adjacent view simulation pictures to obtain the corresponding simulation picture overlap rate.

[0106] Step S204, in response to the existence of a simulation picture overlap rate that does not meet the preset adjacent overlap constraint condition, adjusts the corresponding original view angle to re-determine the two adjacent view simulation pictures.

[0107] Step S205, in response to each simulation picture overlap rate corresponding to the construction environment meeting the adjacent overlap condition, determines the adjusted original view angle as the environment view angle.

[0108] Step S206 performs image acquisition on the construction environment based on each environment view angle to obtain a target image sequence.

[0109] Some embodiments of step S201 determine a plurality of original view angles for the construction environment; wherein each original view angle is used to cover the complete view of the construction environment.

[0110] It should be noted that a plurality of original view angles are determined for the construction environment, and these angles are set to cover the complete view of the construction environment. This process needs to consider the size, shape and distribution of the embedded parts of the construction environment to ensure that all key areas can be captured by shooting from these angles. The determination of each original view angle is the basis, which will determine the range and effect of subsequent image acquisition.

[0111] Some embodiments of step S202 determine a corresponding view simulation picture based on each original view angle.

[0112] It should be noted that the corresponding view simulation picture is determined based on each original view angle. This step is to preview and evaluate the picture that can be captured for each angle before actual shooting. Through the simulation picture, problems that may be encountered in actual shooting, such as occlusion and insufficient light, can be predicted, and adjustments can be made in advance.

[0113] In step S203 of some embodiments, the overlap rate calculation is performed on each two adjacent view simulation pictures to obtain the corresponding simulation picture overlap rate.

[0114] It should be noted that the overlap rate calculation is performed on each two adjacent view simulation pictures to obtain the corresponding simulation picture overlap rate. The overlap rate is a key factor affecting the three-dimensional reconstruction effect. Sufficient overlap rate can ensure the continuity between images and the matching of feature points, thereby improving the accuracy and integrity of three-dimensional reconstruction.

[0115] In step S204 of some embodiments, in response to the existence of a simulation picture overlap rate that does not meet the preset adjacent overlap constraint condition, the corresponding original view angle is adjusted to re-determine the two adjacent view simulation pictures.

[0116] It should be noted that if there is a simulation picture overlap rate that does not meet the preset adjacent overlap constraint condition, the corresponding original view angle needs to be adjusted to re-determine the two adjacent view simulation pictures. The purpose of this step is to ensure that there is enough overlap area between all adjacent images, so that accurate splicing and fusion can be performed in the subsequent three-dimensional reconstruction process.

[0117] In step S205 of some embodiments, in response to the fact that the simulation picture overlap rates corresponding to the construction environment all meet the adjacent overlap condition, the adjusted original view angle is determined as the environment view angle.

[0118] It should be noted that when the simulation picture overlap rates corresponding to the construction environment all meet the adjacent overlap condition, the adjusted original view angle is determined as the environment view angle. This step marks the completion of the image acquisition preparation work, and all view angles have been optimized, and the actual image acquisition work can begin.

[0119] In step S206 of some embodiments, image acquisition is performed based on each environment view angle for the construction environment to obtain a target image sequence.

[0120] It should be noted that image acquisition is performed based on each environment view angle for the construction environment to obtain a target image sequence. This step is the process of actually performing image acquisition, and through the adjusted view angle, it is ensured that the image sequence acquired can meet the requirements of subsequent embedded part detection and three-dimensional reconstruction.

[0121] Some more specific embodiments, in the image acquisition process, in order to ensure that the collected images can meet the requirements of subsequent three-dimensional reconstruction and pre-embedded part detection, the method of mobile phone translation shooting is adopted. This method can effectively cover the complete wall surface in the construction environment by maintaining the stable translation movement of the mobile phone, while ensuring that there is enough overlap between adjacent photos, for example, requiring more than 70%. The setting of such high overlap rate is to smoothly perform feature point matching and three-dimensional reconstruction in the subsequent image processing process, ensuring the continuity and accuracy of the reconstructed model.

[0122] During shooting, choosing the right shooting distance is crucial, which directly affects the clarity and coverage of the image. Too close distance may result in too small image coverage, requiring more photos to cover the complete wall surface; while too far distance may result in insufficient image details, affecting the subsequent pre-embedded part recognition and positioning accuracy. Therefore, when shooting, the shooting distance needs to be adjusted dynamically according to the size of the wall and the distribution of the pre-embedded parts, in order to achieve the best image quality and coverage effect.

[0123] In addition, in order to ensure the quality and availability of the images, those images with poor imaging effect such as overexposure, overdarkness or blur will be deleted during the acquisition process. Overexposed or overdark images will result in unclear features of the pre-embedded parts, increasing the difficulty of recognition; while blurred images will reduce the accuracy of feature point matching, affecting the quality of three-dimensional reconstruction. Therefore, strict selection of the collected images to remove those with substandard quality is a key step to ensure the efficiency and accuracy of the entire pre-embedded part detection process.

[0124] It should be understood that the combination of mobile phone translation shooting, appropriate shooting distance and image selection provides high-quality image data for subsequent pre-embedded part detection and three-dimensional reconstruction. These data not only cover the complete wall surface in the construction environment, but also lay a foundation for feature point matching and three-dimensional reconstruction through sufficient overlap rate and clear image quality. Through such acquisition process, the efficiency and accuracy of pre-embedded part detection can be effectively improved, meeting the requirements of accurate detection of pre-embedded parts in building construction.

[0125] Some embodiments of step S102 input the target image sequence into the pre-trained pre-embedded part detection model for orientation detection to obtain the pre-embedded part positioning information corresponding to each target pre-embedded part;

[0126] It should be noted that step S102 inputs the collected target image sequence into the pre-trained pre-embedded part detection model for orientation detection, thereby obtaining the pre-embedded part positioning information of each target pre-embedded part. This step is a key link in the entire pre-embedded part detection process, which converts image data into specific positioning information, providing a basis for subsequent three-dimensional reconstruction and parameter detection.

[0127] Firstly, the pre-trained embedded part detection model can be constructed based on deep learning techniques. This model is trained on a large-scale labeled dataset, learning various features of embedded parts, including shape, texture, color, etc. In this way, the embedded part detection model can automatically identify the embedded parts in the input image sequence and determine their positions and orientations when new images are input.

[0128] When the target image sequence is input into the detection model, the embedded part detection model can analyze each image, identify the embedded parts therein, and generate corresponding positioning information. This positioning information can include the two-dimensional coordinates of the embedded parts in the image, as well as possible orientation angles and other parameters. These information not only indicates the position of the embedded parts in the image, but also provides the orientation of the embedded parts, which is crucial for subsequent three-dimensional reconstruction and parameter detection.

[0129] In addition, the orientation detection function of the embedded part detection model can handle images with different environmental viewing angles, thanks to the diverse data the model has been exposed to during training. The embedded part detection model has learned to recognize embedded parts from various angles and lighting conditions, making it adaptable to complex construction environments in practical applications. This capability makes the method highly robust and reliable in real-world construction environments.

[0130] Reference Figure 3 According to some embodiments of the present application, step S102 inputs the target image sequence into the pre-trained embedded part detection model for orientation detection to obtain the embedded part positioning information corresponding to each target embedded part, which can include:

[0131] Step S301 inputs the target image sequence into the embedded part detection model.

[0132] Step S302 segments each environmental local image in the embedded part detection model to obtain the corresponding embedded part image region.

[0133] Step S303 performs edge extraction on each embedded part image region in the embedded part detection model to obtain the embedded part positioning information corresponding to each target embedded part.

[0134] Step S301 of some embodiments inputs the target image sequence into the embedded part detection model.

[0135] It should be noted that the target image sequence is input into the pre-trained embedded part detection model. This step is the starting point of the entire detection process, and the model will process each environmental local image in the image sequence one by one.

[0136] Step S302 of some embodiments segments each environmental local image in the embedded part detection model to obtain the corresponding embedded part image region.

[0137] It's important to note that in the embedded parts detection model, each local environment image is segmented to obtain the corresponding embedded parts image region. Image segmentation separates the embedded parts from the background, forming independent image regions. This step is crucial for subsequent edge extraction and location information acquisition, as it reduces background interference and focuses on the embedded parts themselves.

[0138] In step S303 of some embodiments, edge extraction is performed on each embedded part image region in the embedded part detection model to obtain embedded part positioning information corresponding to each target embedded part.

[0139] It should be noted that in the embedded parts detection model, edge extraction is performed for each embedded parts image area. Edge extraction technology can identify and depict the outline of the embedded parts, thereby obtaining the shape and boundary information of the embedded parts. These edge information are critical for determining the position and direction of the embedded parts. Through the contour information obtained by edge extraction, the model can determine the embedded parts positioning information corresponding to each target embedded part. This positioning information includes the position coordinates and direction angles of the embedded parts in the image, providing accurate two-dimensional positioning data for subsequent three-dimensional reconstruction and parameter detection. In this way, not only the accuracy of embedded parts detection is improved, but also the foundation is laid for subsequent three-dimensional reconstruction and parameter calculation. Through such refinement steps, the entire embedded parts detection method can more accurately identify and locate embedded parts in the construction environment, meeting the requirements for accurate detection of embedded parts in construction.

[0140] Reference Figure 4 According to some embodiments of the present application, step S303 performs edge extraction on each embedded part image region to obtain embedded part positioning information corresponding to each target embedded part, which may include:

[0141] Step S401, extracting the minimum rectangle edge for each embedded part image area to obtain corresponding minimum rectangle edge information;

[0142] Step S402 : extracting corner point orientation parameters from the minimum rectangular edge information of each embedded part image region as embedded part positioning information corresponding to each target embedded part.

[0143] In step S401 of some embodiments, minimum rectangular edge extraction is performed on each embedded component image region to obtain corresponding minimum rectangular edge information;

[0144] It should be noted that the minimum rectangular edge extraction is performed for each embedded part image area to obtain the corresponding minimum rectangular edge information. The purpose of this step is to find the smallest rectangle that can completely contain the embedded part in the image area of the embedded part. In this way, the position and direction of the embedded part in the image can be effectively determined. The minimum rectangular edge extraction technique can identify the boundary of the embedded part and represent it as a rectangle whose edges are aligned with the edges of the embedded part. This not only simplifies the subsequent processing steps, but also provides a basis for corner point extraction.

[0145] Step S402 of some embodiments extracts the corner point orientation parameters from the minimum rectangular edge information of each embedded part image area as the corresponding embedded part positioning information of each target embedded part.

[0146] It should be noted that the corner point orientation parameters are extracted from the minimum rectangular edge information of each embedded part image area. The key of this step lies in identifying the four corner points of the rectangle and determining their precise positions in the image. These corner point orientation parameters directly correspond to the position and direction of the embedded part in the image and are the basis for subsequent three-dimensional reconstruction and positioning. By accurately extracting these corner points, accurate positioning of the embedded part can be achieved. In this way, through minimum rectangular edge extraction and corner point orientation parameter extraction, accurate positioning information is obtained from the embedded part image area. This process not only improves the accuracy of embedded part detection, but also provides key data support for subsequent three-dimensional reconstruction and parameter calculation. Through such detailed steps, the entire embedded part detection method can more accurately identify and position the embedded parts in the construction environment, meeting the requirements of accurate detection of embedded parts in construction.

[0147] For some embodiments of step S102, the application can effectively detect rectangular flat embedded parts in the construction environment and select the most suitable deep learning model to achieve this goal.

[0148] Firstly, considering that most construction embedded parts are rectangular flat embedded parts, and the background is mostly a steel frame, such embedded parts can have a high degree of recognition in the image because of their regular shape and distinct contrast with the background. This feature makes them easy to be learned and recognized by deep learning models in the image. Therefore, the application mainly targets such embedded parts and uses AI detection technology based on deep learning for embedded part detection.

[0149] For example, to train an efficient embedded part detection model, the present application can collect and label embedded part samples on site in the construction environment, creating a dataset containing 10,000 samples. These samples are divided into training samples and test samples in a ratio of 3:1 for model training and verification. Through such data preparation, the embedded part detection model can learn various features of the embedded part during training and verify its performance during testing.

[0150] In terms of model selection for the embedded part detection model, the present application compares YOLOv5, YOLOv8 and YOLOv11. Experimental results show that YOLOv11 has obvious advantages in various indicators, including higher detection accuracy and faster processing speed. Therefore, YOLOv11 is selected as the embedded part detection model in some embodiments. The advanced and efficient nature of YOLOv11 makes it an ideal choice for processing embedded part detection in construction environments.

[0151] To extract the embedded part edge as accurately as possible, the present application uses the YOLOv11-seg model. YOLOv11-seg is an instance segmentation model based on YOLACT, which can achieve pixel-level segmentation of individual objects in an image. Through this segmentation method, the edge of the embedded part can be accurately extracted, and the minimum rectangular bounding box of each segmented region can be further obtained. Recording the four corner points can complete the positioning of the embedded part image.

[0152] As can be seen, the optional embodiments of the present application achieve high-precision detection and positioning of embedded parts by utilizing the high-recognition characteristics of rectangular flat embedded parts, combining the training and optimization of deep learning models, and the pixel-level segmentation capability of the YOLOv11-seg model. This method not only improves the efficiency of embedded part detection, but also ensures the accuracy and reliability of the detection results, providing an effective technical means for construction quality detection.

[0153] In step S103 of some embodiments, three-dimensional reconstruction is performed according to the plurality of environment local images in the target image sequence to obtain environment three-dimensional point cloud data;

[0154] It should be noted that step S103 focuses on using the plurality of environment local images in the target image sequence to carry out three-dimensional reconstruction work, and then obtains environment three-dimensional point cloud data. This process first requires analyzing the collected multiple environment local images, and finding feature point matching between images with the help of computer vision technology. These feature points, like key connection points, can help establish geometric relationships between different images.

[0155] The three-dimensional reconstruction process relies on the principle of multi-view geometry. Specifically, by analyzing images taken from different angles, embodiments of the present application can calculate the positions of feature points in three-dimensional space. This calculation process can involve techniques such as triangulation, which determines the coordinates of feature points in the real world by determining their positions in multiple images.

[0156] After obtaining the three-dimensional coordinates of the feature points, embodiments of the present application will further use these data to generate a three-dimensional point cloud of the environment. The three-dimensional point cloud is composed of a large number of points, each of which represents the spatial position of a point in the environment. These points together constitute a three-dimensional model of the construction environment, providing rich spatial information for subsequent embedded part detection.

[0157] The three-dimensional point cloud data of the environment not only contains the position information of the embedded parts, but also includes other elements in the construction environment, such as walls, floors, and steel bars. These data provide the basis for accurate positioning and parameter measurement of embedded parts. By analyzing the three-dimensional point cloud, the orientation and attitude of the embedded parts in three-dimensional space can be more accurately determined, which is of great significance for subsequent construction quality assessment and decision-making.

[0158] As can be seen, step S103 converts the two-dimensional image sequence into three-dimensional point cloud data through three-dimensional reconstruction technology, providing necessary spatial information for detailed detection and analysis of embedded parts.

[0159] Reference Figure 5 According to some embodiments of the present application, the target image sequence is configured with multi-view image constraint information for each environment local image, and step S103 performs three-dimensional reconstruction based on the plurality of environment local images in the target image sequence to obtain environment three-dimensional point cloud data, which can include:

[0160] Step S501, feature extraction is performed on each environment local image in the target image sequence to obtain key feature points of each environment local image;

[0161] Step S502, point cloud construction based on the key feature points of each environment local image to obtain environment three-dimensional point cloud data.

[0162] Step S501 of some embodiments, feature extraction is performed on each environment local image in the target image sequence to obtain key feature points of each environment local image;

[0163] It should be noted that feature extraction is performed on each environment local image in the target image sequence to obtain key feature points of each environment local image. Feature extraction is the cornerstone of three-dimensional reconstruction, which identifies significant points in the image to provide a basis for subsequent point matching and spatial positioning. These key feature points can be regions with high contrast, unique texture or obvious boundaries in the image, which can help establish a connection between images of different angles.

[0164] Reference Figure 6 According to some embodiments of the present application, step S501 performs feature extraction on each local environment image in the target image sequence to obtain key feature points of each local environment image, which may include:

[0165] Step S601, performing feature extraction on each local environment image in the target image sequence to obtain original feature points corresponding to each local environment image;

[0166] Step S602: performing correlation matching on the original feature points corresponding to each local image of the environment to obtain the correlation feature points corresponding to each local image of the environment;

[0167] Step S603, performing geometric verification based on associated feature points corresponding to a local image of an environment;

[0168] Step S604: determining the associated feature points that have passed the geometric verification as key feature points of the corresponding environment local image.

[0169] In step S601 of some embodiments, feature extraction is performed on each local environment image in the target image sequence to obtain original feature points corresponding to each local environment image;

[0170] It should be noted that feature extraction is performed on each local image of the target environment in the target image sequence to obtain the corresponding raw feature points. The purpose of this step is to identify significant and discriminative feature points in the image. These feature points may include corner points, edge points, or areas with unique textures. The extraction of raw feature points is the foundation of 3D reconstruction because they provide the necessary data for subsequent feature point matching and 3D spatial positioning.

[0171] In step S602 of some embodiments, correlation matching is performed on the original feature points corresponding to each local image of the environment to obtain the correlation feature points corresponding to each local image of the environment;

[0172] It should be noted that correlation matching is performed on the original feature points corresponding to each local image of the environment to obtain the associated feature points corresponding to each local image of the environment. The goal of correlation matching is to find common feature points between different images, that is, to identify the projection of the same physical point in multiple images. This step utilizes the principles of multi-view geometry to ensure that feature points in images captured from different angles are correctly matched, providing the necessary geometric constraints for subsequent 3D reconstruction.

[0173] In step S603 of some embodiments, geometric verification is performed based on associated feature points corresponding to a local image of an environment;

[0174] It should be noted that the associated feature points that pass the geometric verification are determined as the key feature points corresponding to the local image of the environment. The purpose of this step is to ensure that the matched feature points are geometrically consistent, i.e., they satisfy the projection relationship in multi-view geometry. Geometric verification can be achieved by checking whether the feature points satisfy certain geometric conditions (such as coplanar condition, collinear condition, etc.) to exclude false matching points.

[0175] In step S604 of some embodiments, the associated feature points that pass the geometric verification are determined as the key feature points corresponding to the local image of the environment.

[0176] It should be noted that the associated feature points that pass the geometric verification are determined as the key feature points corresponding to the local image of the environment. These key feature points are the core data in the process of three-dimensional reconstruction, and they will be used for subsequent point cloud construction and three-dimensional model generation. It should be understood that these key feature points not only contain important information in the image, but also meet the consistency requirements of multi-view geometry, providing reliable data support for subsequent three-dimensional reconstruction. This process is the basis of three-dimensional reconstruction, ensuring the accuracy and reliability of the generated three-dimensional point cloud data.

[0177] In step S502 of some embodiments, point cloud construction is performed based on the key feature points of each local image of the environment, and three-dimensional point cloud data of the environment is obtained.

[0178] It should be noted that point cloud construction is performed based on the key feature points of each local image of the environment, and three-dimensional point cloud data of the environment is obtained. Point cloud construction is the process of converting two-dimensional image information into three-dimensional spatial data. By analyzing the matching relationship of feature points in multiple images and using geometric methods such as triangulation, the coordinates of these feature points in three-dimensional space can be calculated. This set of coordinate points constitutes the three-dimensional point cloud data of the environment, providing necessary spatial information for subsequent pre-embedded part positioning and parameter detection.

[0179] It should be understood that in the process of three-dimensional reconstruction, through the two steps of feature extraction and point cloud construction, the two-dimensional image sequence can be converted into three-dimensional point cloud data. This process not only depends on the quality of the image itself, but also depends on the accurate extraction and matching of feature points. Multi-view image constraint information plays an important role in this process, which ensures that image data obtained from different angles can be correctly integrated into a unified three-dimensional model. This step is the key link in the whole pre-embedded part detection method to realize the conversion from image data to three-dimensional spatial information, laying a foundation for subsequent pre-embedded part positioning and parameter detection.

[0180] Referring to Figure 7 According to some embodiments of the present application, step S502 of performing point cloud construction based on the key feature points of each local image of the environment to obtain three-dimensional point cloud data of the environment can include:

[0181] Step S701, selecting two environment local images from the target image sequence, and performing initial construction on the key feature points corresponding to the two environment local images to obtain initial environment point cloud data;

[0182] Step S702, from the next environment local image in the target image sequence, incrementally reconstructing according to the key feature points corresponding to the environment local image and the environment point cloud data to update the environment point cloud data;

[0183] Step S703, in response to the existence of an environment local image in the target image sequence that does not participate in incremental reconstruction, returning to the next environment local image in the target image sequence and incrementally reconstructing according to the key feature points corresponding to the environment local image and the environment point cloud data;

[0184] Step S704, in response to the fact that all environment local images in the target image sequence participate in incremental reconstruction, determining the environment point cloud data as the environment three-dimensional point cloud data.

[0185] Step S701 of some embodiments, selecting two environment local images from the target image sequence, and performing initial construction on the key feature points corresponding to the two environment local images to obtain initial environment point cloud data;

[0186] It should be noted that two environment local images are selected from the target image sequence, and the key feature points corresponding to the two images are initially constructed to obtain initial environment point cloud data. This step is the starting point of three-dimensional reconstruction. By selecting two images and matching their key feature points, the initial positions of these feature points in three-dimensional space are calculated using triangulation technology, thereby forming the initial point cloud.

[0187] Step S702 of some embodiments, from the next environment local image in the target image sequence, incrementally reconstructing according to the key feature points corresponding to the environment local image and the environment point cloud data to update the environment point cloud data;

[0188] It should be noted that the next environment local image is selected from the target image sequence, and the key feature points corresponding to the image are incrementally reconstructed with the existing environment point cloud data to update the environment point cloud data. The process of incremental reconstruction involves matching the key feature points of the new image with the existing three-dimensional point cloud to determine the position and pose of the new image in three-dimensional space, and calculating more three-dimensional points through triangulation to expand and enrich the point cloud data.

[0189] Step S703 of some embodiments, in response to the existence of an environment local image in the target image sequence that does not participate in incremental reconstruction, returning to the next environment local image in the target image sequence and incrementally reconstructing according to the key feature points corresponding to the environment local image and the environment point cloud data;

[0190] It should be noted that if there are still environment local images in the target image sequence that do not participate in the incremental reconstruction, the embodiments of the present application will return and continue to select the next environment local image, and repeat the above incremental reconstruction process. This loop operation ensures that all images are included in the three-dimensional reconstruction process, so that the final three-dimensional point cloud data can fully cover all parts of the construction environment.

[0191] In step S704 of some embodiments, in response to all environment local images in the target image sequence participating in the incremental reconstruction, the environment point cloud data is determined as the environment three-dimensional point cloud data.

[0192] It should be noted that once all environment local images in the target image sequence participate in the incremental reconstruction, the current environment point cloud data is determined as the final environment three-dimensional point cloud data by the embodiments of the present application. This step marks the completion of the three-dimensional reconstruction process, and the generated three-dimensional point cloud data contains detailed spatial information of the construction environment, providing a basis for subsequent embedded part detection and parameter analysis. It should be understood that this process makes full use of the information of multi-view images, ensuring the accuracy and integrity of three-dimensional reconstruction, and providing key three-dimensional spatial information for embedded part detection in building construction.

[0193] Referring to Figure 8 According to some embodiments of the present application, a calibration board is placed in the construction environment, and step S502 performs point cloud construction based on the key feature points of each environment local image to obtain environment three-dimensional point cloud data, which can include:

[0194] Step S801, obtaining reference calibration information matched with the calibration board;

[0195] Step S802, performing point cloud construction based on the key feature points of each environment local image to obtain intermediate point cloud data; wherein the intermediate point cloud data includes calibration board point cloud data;

[0196] Step S803, performing scale conversion on the intermediate point cloud data based on the calibration board point cloud data and the reference calibration information to obtain the environment three-dimensional point cloud data.

[0197] In some embodiments, a calibration board is placed in the construction environment, which provides an important reference for the three-dimensional reconstruction process. It should be understood that placing a calibration board of known specifications in the construction environment plays an important role in achieving the automated absolute orientation of the embedded parts. The selection and placement of the calibration board need to meet certain conditions to ensure its effectiveness in three-dimensional reconstruction and subsequent embedded part detection. The calibration board is a planar object with known geometric features, such as a checkerboard pattern. The feature points of such a pattern are easy to be recognized and located by computer vision algorithms. In the construction environment, the calibration board needs to be placed in a suitable position, which can be a blank area on the wall. This position should be flat enough to allow the calibration board to be placed horizontally, and the board surface needs to be close to the wall while maintaining parallelism with the wall. Placing the calibration board horizontally in the blank area of the wall is to ensure that the calibration board is in the same plane or parallel plane as the wall, which simplifies subsequent geometric calculations. The placement of the board close to and parallel to the wall helps to keep the calibration board and wall embedded parts in the same angle range during image acquisition, making it easier to acquire image information of the calibration board and embedded parts at the same time. In the three-dimensional reconstruction process, the known geometric features of the calibration board can be used to determine the internal and external parameters of the image, including the focal length of the camera, the position of the optical axis, and the orientation and angle during shooting. These parameters are crucial for converting the collected image data into three-dimensional space data. Through the recognition and positioning of the calibration board, the relationship between the image coordinate system and the world coordinate system can be established, thereby achieving the absolute orientation of the embedded parts. In addition, the calibration board can also be used to verify and correct the accuracy of the three-dimensional reconstruction. By comparing the position of the calibration board in the reconstructed model with the known actual position, the accuracy of the reconstructed model can be evaluated and adjusted. This calibration process helps to improve the reliability and accuracy of the entire embedded part detection system.

[0198] In step S801 of some embodiments, reference calibration information matching the calibration board is obtained;

[0199] It should be noted that the reference calibration information matching the calibration board is obtained. This step involves placing a calibration board in the construction environment and ensuring that the calibration board is included in at least one local image of the environment during image acquisition. The reference calibration information includes the known size and geometric features of the calibration board, which will serve as key reference data for subsequent scale conversion.

[0200] In step S802 of some embodiments, point cloud construction is performed based on the key feature points of each local image of the environment to obtain intermediate point cloud data; wherein the intermediate point cloud data includes calibration board point cloud data;

[0201] It should be noted that the intermediate point cloud data is generated by constructing a point cloud based on the key feature points of each local image of the environment. This step is similar to the previous 3D reconstruction process. Through matching and triangulation of key feature points, the intermediate point cloud data containing the construction environment and the calibration plate is generated. The calibration plate point cloud data in the intermediate point cloud data will be used for subsequent scale conversion.

[0202] In step S803 of some embodiments, scale conversion is performed on the intermediate point cloud data based on the calibration plate point cloud data and the reference calibration information to obtain three-dimensional point cloud data of the environment.

[0203] It's important to note that, based on the calibration plate point cloud data and the benchmark calibration information, the intermediate point cloud data is scaled to produce the final 3D point cloud data for the environment. This step is crucial to the entire process. By comparing the calibration plate point cloud in the intermediate point cloud data with the known benchmark calibration information, the scale deviation of the intermediate point cloud data can be determined. This deviation is used to rescale the entire intermediate point cloud data, resulting in a 3D point cloud data for the environment that is consistent with the real-world dimensions.

[0204] In some embodiments, step S103 utilizes incremental 3D image reconstruction, reconstructing the spatial structure of the wall within the construction environment where the formwork embedded components are located using a sequence of captured target images. The core of this method is multi-view image constraints, which gradually calibrate new images to expand the reconstruction range and enhance the stability of the reconstructed structure.

[0205] Specifically, the basic process of incremental 3D reconstruction includes several key steps:

[0206] The first step is feature point extraction. This step identifies representative key points from the image. These points have good discrimination and stability, and can be repeatedly recognized in different images. Next is feature point matching. By comparing feature points in different images, the corresponding relationship between them is established, thus providing geometric constraints for subsequent 3D reconstruction.

[0207] Initial reconstruction is the starting point of the entire process. It can begin with two images with sufficient overlap. Using techniques such as triangulation, the initial 3D point cloud and camera pose are calculated. This step lays the foundation for subsequent incremental reconstruction.

[0208] As incremental reconstruction progresses, new images are continuously added to the existing 3D model. Each time a new image is added, the 3D model is updated through feature point matching and triangulation, expanding the reconstruction range and optimizing the model's accuracy. This process gradually accumulates a large amount of 3D point cloud data, gradually completing the reconstructed wall structure.

[0209] The multi-view dense reconstruction phase utilizes the information from multiple images to generate a denser and more detailed three-dimensional point cloud. This step fills in the gaps in the sparse reconstruction by combining data from multiple perspectives, improving the model's detail richness and completeness.

[0210] As can be seen, the incremental image three-dimensional reconstruction technology effectively reconstructs the spatial structure of the wall surface by using image sequences captured by a mobile phone through feature point extraction, matching, initialization reconstruction, incremental reconstruction, and multi-view dense reconstruction. This process not only improves the accuracy and stability of the reconstruction but also expands the reconstruction range, providing detailed three-dimensional geometric information for the detection and analysis of embedded parts.

[0211] In some more specific embodiments, using the COLMAP algorithm to complete incremental three-dimensional reconstruction is an efficient and accurate choice. COLMAP is a three-dimensional reconstruction algorithm that can reconstruct an accurate three-dimensional model from multiple-view images.

[0212] First, the COLMAP algorithm starts from the input target image sequences, which can be collected from a mobile phone or other imaging devices. The COLMAP algorithm performs feature extraction on each image, identifying key points in the image. These key points can be areas with high contrast or unique textures in the image, which can help the subsequent feature matching process.

[0213] Next, the COLMAP algorithm performs feature point matching by comparing feature points in different images to establish their correspondence. This step uses multi-view geometry principles to ensure that the matched feature points correspond to the same point in three-dimensional space.

[0214] Then, the COLMAP algorithm performs initialization reconstruction, which can start from two images with sufficient overlapping areas. Through triangulation technology, the initial three-dimensional point cloud and camera pose are calculated. This step lays the foundation for subsequent incremental reconstruction.

[0215] As the incremental reconstruction progresses, the COLMAP algorithm gradually adds new images to the existing three-dimensional model. With each new image added, the algorithm updates the three-dimensional model through feature point matching and triangulation, expanding the reconstruction range and optimizing the accuracy of the model. This process continuously accumulates three-dimensional point cloud data, making the reconstructed model gradually complete.

[0216] The COLMAP algorithm also includes an outlier filtering step to remove noise points and erroneous estimates during reconstruction, improving the accuracy and robustness of the reconstruction. Through this filtering process, the algorithm ensures that only reliable feature points are used for subsequent reconstruction steps.

[0217] In the triangulation step, the COLMAP algorithm computes more 3D points based on the feature point matches and camera poses, enriching the details of the reconstructed model. Subsequently, this global optimization process is refined through bundle adjustment, minimizing the reprojection errors of all feature points, optimizing camera parameters and 3D point positions, thereby improving the accuracy and consistency of the entire reconstructed model.

[0218] Finally, the COLMAP algorithm performs multi-view dense reconstruction, generating a more dense and detailed 3D point cloud using information from multiple images. This step combines data from multiple perspectives, filling in gaps in the sparse reconstruction, and improving the model's detail richness and completeness.

[0219] It should be understood that the COLMAP algorithm effectively completes the incremental 3D reconstruction task through steps such as feature extraction, matching, initial reconstruction, incremental reconstruction, outlier filtering, triangulation, bundle adjustment, and multi-view dense reconstruction. This process not only improves the accuracy and stability of the reconstruction, but also expands the scope of the reconstruction, providing detailed 3D geometric information for the detection and analysis of embedded parts.

[0220] Some embodiments of step S104, according to the embedded part positioning information and the environment three-dimensional point cloud data, determine the embedded part point cloud data matching each target embedded part;

[0221] It should be noted that step S104 is to determine the embedded part point cloud data matching each target embedded part according to the embedded part positioning information and the environment three-dimensional point cloud data. This step is a key link for combining two-dimensional positioning information of embedded parts with three-dimensional environment data, providing accurate three-dimensional data support for subsequent embedded part parameter detection.

[0222] Specifically, the embedded part positioning information is obtained in step S102, which includes the position and direction of each embedded part in the image. These information provides preliminary clues for filtering out the point cloud related to the embedded part in the three-dimensional point cloud data. The environment three-dimensional point cloud data is obtained through the three-dimensional reconstruction of step S103, which contains detailed three-dimensional information of the construction environment.

[0223] In the process of determining the embedded part point cloud data, the two-dimensional positioning information of the embedded part needs to be first mapped to the three-dimensional space. This can be achieved by converting image coordinates to three-dimensional coordinates, using camera intrinsic and extrinsic parameters for projection transformation. Once the approximate three-dimensional position of the embedded part is determined, the point cloud related to this position can be filtered out in the three-dimensional point cloud data.

[0224] Next, a series of geometric and spatial analysis techniques are used to further refine the screening process. For example, a spatial region bounding method can be used to retain only those point cloud data that are located near the expected location of the embedded part. In addition, the point cloud data can be screened according to the shape and size characteristics of the embedded part, and those point clouds that are obviously not part of the embedded part are removed.

[0225] In addition, in order to improve the accuracy of the match, a similarity measurement method can also be used. By comparing the expected shape of the embedded part with the actual shape in the point cloud data, those point clouds with low similarity are removed, thereby ensuring the accuracy and reliability of the final embedded part point cloud data.

[0226] As can be seen, step S104 determines the embedded part point cloud data that matches each target embedded part by combining the two-dimensional positioning information of the embedded part with the three-dimensional environmental data, using techniques such as geometric analysis and similarity measurement. These data not only contain the three-dimensional position of the embedded part, but also possibly include its shape, size, etc. information, providing a basis for subsequent parameter detection. This step plays a role in bridging the gap between image acquisition and parameter calculation in the embedded part detection method, ensuring a smooth transition.

[0227] Reference Figure 9 According to some embodiments of the present application, the embedded part positioning information includes an embedded part detection box in the environmental local image, and step S104 determines the embedded part point cloud data that matches each target embedded part according to the embedded part positioning information and the environmental three-dimensional point cloud data, which can include:

[0228] Step S901 projects the environmental three-dimensional point cloud data into each environmental local image to obtain a corresponding point cloud projection image; wherein the environmental three-dimensional point cloud data includes a plurality of key feature points;

[0229] Step S902 extracts the key feature points that are projected in each point cloud projection image within the embedded part detection box to obtain the embedded part point cloud data that matches each target embedded part.

[0230] In some more specific embodiments, the YOLO embedded part detection technology can be used to obtain the detection box of each embedded part in the image as the embedded part positioning information. When using the YOLO detection model as the embedded part detection model, the embedded part in the image can be quickly and accurately identified, and its position in the image, i.e. the embedded part detection box, is given. This detection box provides a clear area for subsequent three-dimensional point cloud projection and matching.

[0231] Step S901 of some embodiments projects the environmental three-dimensional point cloud data into each environmental local image to obtain a corresponding point cloud projection image; wherein the environmental three-dimensional point cloud data includes a plurality of key feature points;

[0232] It should be noted that the environmental three-dimensional point cloud data is projected into each environmental local image to obtain the corresponding point cloud projection image. The purpose of this step is to convert the point cloud data in three-dimensional space into a projection on a two-dimensional image plane, so as to match with the embedded part detection frame. The environmental three-dimensional point cloud data contains a plurality of key feature points in the construction environment, which are mapped to the corresponding two-dimensional image positions in the projection process.

[0233] In step S902 of some embodiments, key feature points projected in each point cloud projection image within the embedded part detection frame are extracted to obtain embedded part point cloud data matched with each target embedded part.

[0234] It should be noted that the key feature points projected in each point cloud projection image within the embedded part detection frame are extracted. The embedded part detection frame is obtained by the embedded part detection model in step S102, which indicates the position of the embedded part in the image. By projecting the three-dimensional point cloud data into the two-dimensional image, it can be determined which key feature points are located within the embedded part detection frame. These key feature points within the detection frame are the point cloud data matched with the target embedded part. In this way, the three-dimensional point cloud data can be effectively associated with the embedded part detection frame in the two-dimensional image, so as to determine the point cloud data belonging to each target embedded part. This process not only utilizes the two-dimensional positioning information provided by the embedded part detection model, but also combines the point cloud data obtained by three-dimensional reconstruction, realizing accurate matching from two-dimensional to three-dimensional. The finally obtained embedded part point cloud data will contain detailed information of each embedded part in three-dimensional space, providing a basis for subsequent parameter detection and analysis.

[0235] In step S105 of some embodiments, parameter detection is performed based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matched with each target embedded part.

[0236] It should be noted that step S105 performs parameter detection based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matched with each target embedded part. This step is a key link in the entire detection process, which converts the data obtained in the previous steps into specific measurement results for evaluating the installation quality and position accuracy of the embedded part.

[0237] Specifically, the embedded part point cloud data contains detailed information of the embedded part in three-dimensional space, such as position, shape and size, etc. By analyzing these point cloud data, key parameters of the embedded part can be extracted. For example, the center coordinates, size deviation, flatness, perpendicularity, etc. of the embedded part can be calculated. These parameters can accurately reflect the installation state of the embedded part and whether it meets the design requirements.

[0238] During parameter detection, a series of geometric calculations and analysis methods can be adopted. For example, by fitting a plane, it can be determined whether the installation plane of the embedded part meets the design flatness requirement; by calculating the center point of the point cloud data, it can be determined the position deviation of the embedded part; by analyzing the distribution of the point cloud data, it can be evaluated whether the shape and size of the embedded part are within the allowable tolerance range.

[0239] In addition, the embedded part detection parameters can also be used for comparison and verification with other construction data. For example, the detected embedded part position can be compared with the theoretical position on the design drawing to calculate the actual deviation; the detected embedded part size can be compared with the design size to evaluate the size accuracy. These comparison results can help the construction team to discover and correct problems in the installation process in a timely manner, ensuring the construction quality.

[0240] As can be seen, step S105 obtains a series of key embedded part detection parameters through detailed analysis and geometric calculation of the embedded part point cloud data. These parameters not only provide quantitative evaluation of the installation state of the embedded part, but also provide scientific basis for construction quality control and decision-making. This step realizes the transformation from three-dimensional point cloud data to specific detection results, and is an indispensable link in the entire embedded part detection method, ensuring the accuracy and reliability of embedded part detection.

[0241] Reference Figure 10 According to some embodiments of the present application, step S105 performs parameter detection based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matched to each target embedded part, which can include:

[0242] Step S1001 performs plane fitting operation on the embedded part point cloud data of each target embedded part to obtain a fitted construction plane;

[0243] Step S1002 determines a parameter detection angle that satisfies a preset direct facing condition for each target embedded part from a plurality of environment viewing angles;

[0244] Step S1003 performs parameter calculation on the corresponding target embedded part in the fitted construction plane according to each parameter detection angle to obtain embedded part detection parameters matched to each target embedded part.

[0245] Step S1001 of some embodiments performs plane fitting operation on the embedded part point cloud data of each target embedded part to obtain a fitted construction plane;

[0246] It should be noted that the pre-embedded part point cloud data of each target pre-embedded part is subjected to plane fitting operation to obtain a fitting construction plane. The purpose of this step is to determine the plane where the pre-embedded part is located through mathematical method, to provide a reference plane for subsequent parameter calculation. The plane fitting technology can analyze the point cloud data to find the best fitting plane, thereby determining the installation plane of the pre-embedded part, which is crucial for evaluating the flatness and inclination of the pre-embedded part and other parameters.

[0247] Step S1002 of some embodiments determines, from a plurality of environmental view angles, a parameter detection angle that satisfies a preset normal condition for each target pre-embedded part;

[0248] It should be noted that the parameter detection angle that satisfies the preset normal condition is determined for each target pre-embedded part from a plurality of environmental view angles. This step involves selecting an angle that best reflects the characteristics of the pre-embedded part from multiple shooting angles. This angle can refer to the angle with the smallest angle with the normal of the pre-embedded part plane, which can ensure the accuracy of parameter calculation. Selecting such an angle helps to improve the accuracy and reliability of measurement.

[0249] Step S1003 of some embodiments performs parameter calculation on the corresponding target pre-embedded part in the fitting construction plane according to each parameter detection angle, to obtain pre-embedded part detection parameters matched to each target pre-embedded part.

[0250] It should be noted that parameter calculation is performed on the corresponding target pre-embedded part in the fitting construction plane according to each parameter detection angle, to obtain pre-embedded part detection parameters matched to each target pre-embedded part. This step uses the selected detection angle and the fitting construction plane to accurately calculate the key parameters of the pre-embedded part such as position, size, perpendicularity, etc. These parameter detection results will provide detailed pre-embedded part installation state information to ensure that the construction quality meets the design requirements. This process not only improves the accuracy of parameter detection, but also provides a scientific basis for construction quality control and decision-making. Through such detailed steps, the entire pre-embedded part detection method can more accurately identify and locate the pre-embedded part in the construction environment, meeting the requirements of precise detection of pre-embedded parts in building construction.

[0251] In some more specific embodiments, the process of determining the most normal image of each target pre-embedded part is a key step of accurate parameter detection. The pre-embedded part detection frame provides a clear area for three-dimensional point cloud projection and matching.

[0252] To perform parameter detection, the corresponding imaging model can be calibrated in the three-dimensional reconstruction process, and the reconstructed three-dimensional point cloud is projected onto each image. This step maps the point cloud data in three-dimensional space to a two-dimensional image plane, enabling the point cloud data of each embedded part in three-dimensional space to correspond to the detection box on the image. Specifically, only those spatial points that project within the embedded part detection box are selected, and these points constitute the spatial point cloud of the embedded part.

[0253] Then, the spatial point cloud of each embedded part is plane-fitted to estimate the plane in which the embedded part is located. Plane fitting is a mathematical method for finding a best plane to describe the distribution of the spatial point cloud. By plane-fitting the spatial point cloud of the embedded part, the planar position and direction of the embedded part in three-dimensional space can be determined.

[0254] After determining the plane in which the embedded part is located, the most direct image needs to be selected from multiple environmental perspective angles. The purpose of this step is to find the image that best reflects the true position and direction of the embedded part. Specifically, the most direct image is the image with the smallest angle between the straight line from the image center to the optical center and the normal line of the embedded part plane. Such an image can provide the most direct and accurate view of the embedded part, reducing measurement errors caused by perspective deviation.

[0255] Finally, four rays are constructed from the optical center of the selected most direct image through the four corner points of the embedded part detection box on the image. The intersection of the four rays and the plane in which the embedded part is located is the four corner points of the embedded part. By calculating the average of the four corner points, the center point position of the embedded part is obtained. This step not only utilizes the two-dimensional information provided by the YOLO detection box, but also combines three-dimensional reconstruction and geometric calculation to achieve accurate positioning of the embedded part in three-dimensional space.

[0256] In summary, through YOLO embedded part detection, three-dimensional point cloud projection, plane fitting, and geometric calculation, the most direct image of each embedded part can be determined from multiple environmental perspective angles, and the center point position of the embedded part is finally obtained. This process fully utilizes the advantages of computer vision and deep learning technology, providing an efficient and accurate solution for construction quality detection.

[0257] Reference Figure 11 According to some embodiments of the present application, a calibration board is placed in the construction environment, and step S1003 performs parameter calculation on the corresponding target embedded part in the fitted construction plane according to each parameter detection angle, obtaining embedded part detection parameters matched to each target embedded part, which can include:

[0258] Step S1101, obtaining reference calibration information matched to the calibration board;

[0259] Step S1102, according to each parameter detection angle in the fitting construction plane to the corresponding target embedded part parameter measurement, get each target embedded part of the preliminary estimate parameter;

[0260] Step S1103, based on the reference calibration information to the preliminary estimate parameter scale conversion, get each target embedded part of the embedded part detection parameter.

[0261] It should be noted that, through the reference calibration information of the calibration board, the internal and external parameters of the image can be determined, including the focal length of the camera, the optical axis position and the direction and angle at the time of shooting. These parameters are crucial for converting the collected image data into three-dimensional space data. In the process of three-dimensional reconstruction, the known geometric features of the calibration board can be used to establish the relationship between the image coordinate system and the world coordinate system, so as to realize the absolute orientation of the embedded part. The calibration board can be used to verify and correct the accuracy of three-dimensional reconstruction. By comparing the position of the calibration board in the reconstructed model with the known actual position, the accuracy of the reconstructed model can be evaluated and adjusted. This calibration process helps to improve the reliability and accuracy of the entire embedded part detection system.

[0262] Step S1101 of some embodiments, obtaining the reference calibration information matching the calibration board;

[0263] It should be noted that, obtaining the reference calibration information matching the calibration board, this step is the basis, through the known size and geometric features of the calibration board, a known reference scale can be provided for subsequent parameter measurement. This step ensures the absolute accuracy of parameter measurement, because the known size of the calibration board can be used to calibrate and verify the measurement results.

[0264] Step S1102 of some embodiments, according to each parameter detection angle in the fitting construction plane to the corresponding target embedded part parameter measurement, get each target embedded part of the preliminary estimate parameter;

[0265] It should be noted that, according to each parameter detection angle in the fitting construction plane to the corresponding target embedded part parameter measurement, get each target embedded part of the preliminary estimate parameter. This step uses the reference information provided by the calibration board to combine the embedded part point cloud data for parameter measurement. The preliminary estimate parameter may include the position, size, perpendicularity of the embedded part, etc.

[0266] Step S1103 of some embodiments, based on the reference calibration information to the preliminary estimate parameter scale conversion, get each target embedded part of the embedded part detection parameter.

[0267] It should be noted that the scale conversion of the preliminary estimated parameters based on the benchmark calibration information obtains the embedded part detection parameters of each target embedded part. This step is crucial. By comparing and converting the preliminary estimated parameters with the benchmark information of the calibration plate, the measurement result can be converted from a relative scale to an absolute scale, ensuring the accuracy and reliability of the detection parameters. It should be emphasized that the calibration plate provides a known reference scale for parameter measurement, so that the detection result can accurately reflect the true state of the embedded part.

[0268] In some more specific embodiments, the complete process from three-dimensional reconstruction to embedded part parameter detection. The entire process involves four coordinate systems, which can be represented as: camera coordinate system calibration plate coordinate system wall surface coordinate system and design value coordinate system Specifically, it can include:

[0269] First, through three-dimensional reconstruction technology, the three-dimensional point cloud of the construction environment is reconstructed by using the photographed image sequence. This process can be based on incremental three-dimensional reconstruction technology, which can accurately restore the spatial structure of the construction environment. However, the coordinate system of the reconstructed three-dimensional point cloud is based on the camera coordinate system, which means that its spatial scale and structure are consistent with the actual scene, but the origin and direction of the coordinate system are relative to the camera.

[0270] In order to measure the error between the calculated embedded part center (calculated center) and the design center, it is necessary to convert the calculated center to the design coordinate system. This coordinate conversion process is completed with the help of the calibration plate as an intermediate coordinate system. The calibration plate provides a known geometric reference, and its coordinate system (calibration plate coordinate system) can be determined by the corner points of the checkerboard. The calibration plate coordinate system is a coordinate system with the calibration plate as the X b -Z b plane, the X b axis parallel to the ground, the Z b axis perpendicular to the ground, and the origin at a corner point of the checkerboard on the calibration plate.

[0271] After determining the calibration plate coordinate system, the wall surface coordinate system also needs to be established. The wall surface coordinate system is only one translation away from the calibration plate coordinate system (or its rotation by 180 degrees around the Z b axis). The coordinate system is determined with the horizontal direction of the wall surface as the X q axis and the vertical ground direction as the Z q axis.

[0272] The final design coordinate system is a coordinate system determined by design values. In order to realize the conversion from the camera coordinate system to the design coordinate system, the correspondence between the center point calculation value and the design value needs to be found by using a topology-based similarity measure. This step involves comparing and matching the three-dimensional reconstructed point cloud with the design model, thereby determining the position of the calculated center of each embedded part in the design coordinate system.

[0273] The embodiments of the present application illustrate in detail how to realize the accurate calculation of the embedded part detection parameters through three-dimensional reconstruction and coordinate system conversion. This process not only involves the acquisition and processing of multi-view images, but also relies on the use of a calibration board and the conversion of multiple coordinate systems, and finally realizes the accurate correspondence from three-dimensional reconstruction to the design coordinate system.

[0274] Reference Figure 12 According to the embedded part detection device, the embedded part detection device can include:

[0275] The image acquisition module 1201 is configured to acquire images of a construction environment containing a plurality of target embedded parts to obtain a target image sequence, wherein the target image sequence contains a plurality of local environment images of different environment perspectives.

[0276] The embedded part orientation detection module 1202 is configured to input the target image sequence into a pre-trained embedded part detection model to perform orientation detection and obtain embedded part positioning information corresponding to each target embedded part.

[0277] The three-dimensional reconstruction module 1203 is configured to perform three-dimensional reconstruction based on the plurality of local environment images in the target image sequence to obtain environment three-dimensional point cloud data.

[0278] The point cloud extraction module 1204 is configured to determine embedded part point cloud data matched to each target embedded part based on the embedded part positioning information and the environment three-dimensional point cloud data.

[0279] The parameter detection module 1205 is configured to perform parameter detection based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matched to each target embedded part.

[0280] It can be seen that the contents in the above-mentioned embedded part detection method embodiments are applicable to the embodiments of the present embedded part detection device. The functions specifically realized by the embodiments of the present embedded part detection device are the same as those of the above-mentioned embedded part detection method embodiments, and the beneficial effects achieved are also the same as those of the above-mentioned embedded part detection method embodiments.

[0281] Reference Figure 13 , Figure 13 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device can include:

[0282] The processor 1301 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0283] The memory 1302 can be implemented by a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), and the like. The memory 1302 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1302 and are called and executed by the processor 1301 to implement the pre-embedded part detection method of the embodiments of the present application.

[0284] The input / output interface 1303 is configured to realize information input and output.

[0285] The communication interface 1304 is configured to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, or the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, or the like).

[0286] The bus 1305 is configured to transmit information between various components (for example, the processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304) of the device.

[0287] The processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304 are connected to each other through the bus 1305 to realize the communication connection between the device.

[0288] The embodiments of the present application further provide a computer program product, which includes a computer program. The processor of the computer device reads the computer program and executes it, so that the computer device executes the pre-embedded part detection method.

[0289] The terms "first", "second", "third", "fourth", and the like in the description of the disclosure and the above drawings, if any, are used to distinguish similar objects, and do not necessarily have to be used to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the disclosure described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "contain" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0290] It should be understood that in the present disclosure, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0291] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple) is two or more, greater than, less than, more than, etc. is not included in the number, and above, below, etc. is included in the number.

[0292] In several embodiments provided by the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0293] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0294] In addition, each functional unit in various embodiments of the present disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0295] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present disclosure essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present disclosure. The aforementioned storage medium can include: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0296] It should also be understood that the various embodiments provided by the present application can be combined in any way to achieve different technical effects.

[0297] The above is a specific description of the embodiments of the present disclosure, but the present disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present disclosure, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present disclosure.

Claims

1. A method for detecting embedded parts, characterized in that: include: Capturing images of a construction environment containing a plurality of target embedded parts to obtain a target image sequence; wherein the target image sequence comprises a plurality of partial images of the environment at different environment viewing angles; Inputting the target image sequence into a pre-trained embedded part detection model for orientation detection to obtain embedded part positioning information corresponding to each target embedded part; Performing three-dimensional reconstruction based on the plurality of local environment images in the target image sequence to obtain three-dimensional point cloud data of the environment; Determining embedded part point cloud data matching each of the target embedded parts according to the embedded part positioning information and the environmental three-dimensional point cloud data; Parameter detection is performed based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matching each target embedded part.

2. The method according to claim 1, characterized in that The image acquisition is performed on a construction environment including a plurality of target embedded parts to obtain a target image sequence, including: Determining a plurality of original framing angles for the construction environment; wherein each of the original framing angles is used to cover a complete framing of the construction environment; Determining a corresponding framing simulation picture based on each of the original framing angles; Calculating the overlap rate of every two adjacent framing simulation pictures to obtain the corresponding simulation picture overlap rate; In response to the existence of the simulated picture overlap ratio not satisfying a preset adjacent overlap constraint condition, adjusting the corresponding original framing angle to re-determine two adjacent framing simulation pictures; In response to the overlap rates of the simulation images corresponding to the construction environment all reaching the adjacent overlap condition, determining the adjusted original framing angle as the environment framing angle; Image collection is performed on the construction environment based on each of the environmental viewing angles to obtain the target image sequence.

3. The method according to claim 1 or 2, characterized in that Inputting the target image sequence into a pre-trained embedded part detection model for orientation detection to obtain embedded part positioning information corresponding to each target embedded part includes: Inputting the target image sequence into the embedded parts detection model; Segmenting each of the local environment images in the embedded parts detection model to obtain corresponding embedded parts image regions; In the embedded part detection model, edge extraction is performed on each embedded part image region to obtain the embedded part positioning information corresponding to each target embedded part.

4. The method according to claim 3, characterized in that The performing edge extraction on each embedded part image region to obtain the embedded part positioning information corresponding to each target embedded part includes: Performing minimum rectangular edge extraction on each of the embedded component image regions to obtain corresponding minimum rectangular edge information; Corner point orientation parameters are extracted from the minimum rectangular edge information of each embedded part image area as the embedded part positioning information corresponding to each target embedded part.

5. The method according to claim 1, wherein The target image sequence is configured with multi-view image constraint information for each of the local environment images, and the three-dimensional reconstruction is performed based on the multiple local environment images in the target image sequence to obtain three-dimensional point cloud data of the environment, including: Performing feature extraction on each of the local environment images in the target image sequence to obtain key feature points of each of the local environment images; A point cloud is constructed based on the key feature points of each local image of the environment to obtain three-dimensional point cloud data of the environment.

6. The method according to claim 5, characterized in that The step of extracting features from each of the local environment images in the target image sequence to obtain key feature points of each local environment image includes: Performing feature extraction on each of the local environment images in the target image sequence to obtain original feature points corresponding to each of the local environment images; Performing correlation matching on the original feature points corresponding to each of the local environment images to obtain correlation feature points corresponding to each of the local environment images; Performing geometric verification based on associated feature points corresponding to a local image of the environment; The associated feature points that pass the geometric verification are determined as the key feature points corresponding to the local image of the environment.

7. The method according to claim 5, characterized in that The step of constructing a point cloud based on the key feature points of each local image of the environment to obtain the three-dimensional point cloud data of the environment includes: Selecting two local environment images from the target image sequence, and initially constructing the key feature points corresponding to the two local environment images to obtain initial environment point cloud data; performing incremental reconstruction on the next local environment image in the target image sequence according to the key feature points corresponding to the local environment image and the environment point cloud data to update the environment point cloud data; In response to the presence of the local environment image that does not participate in the incremental reconstruction in the target image sequence, returning the next local environment image from the target image sequence, and performing incremental reconstruction based on the key feature points corresponding to the local environment image and the environment point cloud data; In response to the fact that all the local environment images in the target image sequence participate in incremental reconstruction, the environment point cloud data is determined as the environment three-dimensional point cloud data.

8. The method according to claim 5, characterized in that A calibration plate is placed in the construction environment, and point cloud construction is performed based on key feature points of each local image of the environment to obtain three-dimensional point cloud data of the environment, including: Acquiring reference calibration information matching the calibration plate; constructing a point cloud based on key feature points of each of the local images of the environment to obtain intermediate point cloud data; wherein the intermediate point cloud data includes calibration plate point cloud data; Based on the calibration plate point cloud data and the reference calibration information, the intermediate point cloud data is scale-converted to obtain the environment three-dimensional point cloud data.

9. The method according to claim 1, characterized in that The embedded part positioning information includes an embedded part detection frame in the local image of the environment, and determining embedded part point cloud data matching each target embedded part based on the embedded part positioning information and the three-dimensional point cloud data of the environment includes: Projecting the three-dimensional point cloud data of the environment onto each of the local images of the environment to obtain a corresponding point cloud projection image; wherein the three-dimensional point cloud data of the environment includes a plurality of key feature points; The key feature points projected within the embedded part detection frame in each of the point cloud projection images are extracted to obtain the embedded part point cloud data matching each of the target embedded parts.

10. The method according to claim 9, characterized in that The performing parameter detection based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matching each target embedded part includes: Performing a plane fitting operation on the embedded part point cloud data of each target embedded part to obtain a fitting construction plane; Determining a parameter detection angle that satisfies a preset facing condition for each target embedded part from the plurality of environmental viewing angles; Parameters of the corresponding target embedded parts are measured in the fitting construction plane according to each parameter detection angle to obtain the embedded part detection parameters matching each target embedded part.

11. The method according to claim 10, characterized in that A calibration plate is placed in the construction environment, and parameter measurement is performed on the corresponding target embedded part in the fitting construction plane according to each parameter detection angle to obtain the embedded part detection parameter matching each target embedded part, including: Acquiring reference calibration information matching the calibration plate; Calculating parameters of the corresponding target embedded parts within the fitting construction plane according to each parameter detection angle to obtain preliminary estimated parameters of each target embedded part; The preliminary estimated parameters are scaled based on the reference calibration information to obtain the embedded part detection parameters of each target embedded part.

12. A pre-embedded parts detection device, characterized in that: include: An image acquisition module is used to acquire images of a construction environment containing a plurality of target embedded parts to obtain a target image sequence; wherein the target image sequence includes a plurality of local images of the environment at different environment viewing angles; An embedded part position detection module is used to input the target image sequence into a pre-trained embedded part detection model to perform position detection, and obtain embedded part positioning information corresponding to each target embedded part; A three-dimensional reconstruction module, configured to perform three-dimensional reconstruction based on the plurality of local environment images in the target image sequence to obtain three-dimensional point cloud data of the environment; a point cloud extraction module, configured to determine embedded part point cloud data matching each of the target embedded parts based on the embedded part positioning information and the environmental three-dimensional point cloud data; The parameter detection module is used to perform parameter detection based on the embedded part point cloud data of each target embedded part to obtain embedded part detection parameters matching each target embedded part.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the embedded parts detection method according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the embedded parts detection method according to any one of claims 1 to 11.

Images