Parallel laser line scanning device and measuring method for structural defects

Abstract

The invention provides a parallel laser line scanning device and a measuring method for structural defects. The scanning device includes: a laser source configured to emit two laser beams parallel to each other to project a scanned surface; the digital camera is configured to collect an image of the scanned surface, and the camera comprises a lens and an image sensor; the positioning rod comprises a laser positioning arm and a telescopic main positioning arm which are rotatably connected with each other, the camera is arranged at the tail end, away from the laser positioning arm, of the telescopic main positioning arm, and the laser source is installed on the laser positioning arm; and a controller configured to acquire a measurement parameter related to the scanning surface by processing the acquired image of the scanning surface. According to the scheme, the design of the scanning device is optimized by introducing the inclined telescopic positioning rod, the scanning device is more portable, the measurement range is expanded, and the accuracy of out-of-plane building defect detection and measurement is improved in combination with the optimized image processing process.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Hong Kong short-term patent application number 32023081077.4 filed on October 19, 2023, with invention name “Parallel laser line scanning device and measurement method for structural defects”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The invention belongs to the field of measurement technology, and in particular relates to a parallel laser line scanning device and an out-of-plane building defect measurement method. Background Art

[0004] Out-of-plane building defects, such as bulging and spalling, can have an adverse effect on the deterioration of buildings. Frequent repairs and increased maintenance costs caused by these defects can often exceed 4% of the total building cost per year. It is therefore imperative to consider these different types of defects to assess the condition of buildings. Existing technologies have proposed a defect-based model specifically for existing concrete buildings to address deterioration issues at an early stage. However, it is difficult to quantify out-of-plane defects (e.g., depth measurements) with existing instruments, especially when they are small and difficult to access. Current structural maintenance methods rely heavily on manual visual inspections, which are subjective, time-consuming, and require specialized expertise. Accurate and reliable out-of-plane building defect measurements play a vital role in effective defect management. They enable early identification of defect symptoms, thereby minimizing repair and maintenance costs. In addition, accurate measurements can also help develop robust condition assessment models, which can improve maintenance strategies and extend the service life of existing buildings.

[0005] There are various techniques to measure out-of-plane building defects. For example, structural health monitoring (SHM) for damage inspection can be roughly divided into two categories: vibration-based SHM and vision-based SHM. Vibration-based SHM relies on advanced sensor technology and signal processing algorithms to detect and assess the severity of damage. Although this method is widely used to measure various structures such as high-rise buildings, bridges, dams, tunnels, and offshore structures, it still has limitations such as complex sensor installation, high sensitivity to environmental conditions, high sensor cost, and labor-intensive data processing. In recent years, vision-based SHM has made significant progress due to the emergence of deep learning technology. Object detection algorithms such as YOLO (Redmon J, Divvala S, Girshick R and Farhadi A (2016). You only look once: unified, real-time object detection. IEEE) and Faster R-CNN (Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence, 39 (6), pp. 1137-1149) have been successfully applied to identify building defects such as cracks, peeling, rust, water seepage, etc. by drawing rectangular bounding boxes around target objects in images.In addition, semantic segmentation algorithms such as FCN (Wang, KY, et al., Multi-class object detection in tunnels from 3D point clouds: An auto-optimized lazy learning approach. Advanced Engineering Informatics, 2022. 52.) and U-Net (Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.) have been used to segment individual pixels of building defects. These methods have achieved high-precision automatic defect recognition. However, it is worth noting that these methods are mainly based on two-dimensional (2D) computer vision and do not provide detailed depth information for quantifying damage.

[0006] Three-dimensional (3D) computer vision has attracted great interest in recent years due to its automated, non-contact, and high-precision measurement capabilities. Various innovative 3D measurement technologies, such as stereo vision (Hirschmuller, H., Stereoprocessing by Semiglobal Matching and Mutual Information. Ieee Transactions on Pattern Analysis and Machine Intelligence, 2008. 30(2): p. 328-341.), time of flight (ToF) (Foix, S., G. Alenya, and C. Torras, Lock-in Time-of-Flight (ToF) Cameras: A Survey. Ieee Sensors Journal, 2011. 11(9): p. 1917-1926.) and laser triangulation-based systems (Yang, SM, et al., A Dual-Platform Laser Scanner for 3D Reconstruction of Dental Pieces. Engineering, 2018. 4(6): p. 796-805.) have been widely studied and applied to building inspections for concrete crack assessment, structural damage quantification and surface damage characterization. However, stereo vision technology is often limited by material texture and may not be able to effectively measure building defects (e.g., bulges without texture). High-end 3D terrestrial laser scanners based on ToF technology can generate dense and high-precision point clouds, but they are usually expensive and not easily accessible to inspectors. Laser triangulation-based systems (which utilize laser diodes and image capture devices) provide a cost-effective solution for accurate object profile measurement based on triangulation and are ideal for out-of-plane building defect measurement. Currently, laser triangulation-based systems are widely used in various fields to measure the three-dimensional profile of complex surfaces.

[0007] However, surface reconstruction using a laser triangulation system usually requires additional motion infrastructure, such as stepper motors, robotic arms, or rotation and translation platforms, which makes the data collection process complicated and time-consuming. In addition, these systems are usually used for short-distance measurements because the measurement error increases significantly with the increase of distance. In order to address these limitations, the inventors of the present application have designed a simple and flexible handheld parallel laser line scanning device based on laser triangulation, which can generate two sets of 3D depth profiles of an object at a time. The measurement distance range is 2m to 4m, and the error is within 0.5mm, which can meet the requirements of ceiling defect measurement (see Hong Kong patent application HK30079800B for details, which is incorporated herein by reference in its entirety). However, the scanning device is not suitable for structures with a measurement distance of less than 2m (e.g., wall scanning), and is affected by factors such as the assembly of the positioning rod and the potential relative movement between the laser diode and the camera during operation, and the measurement accuracy needs to be further improved. Summary of the invention

[0008] Therefore, the present application provides a new parallel laser line scanning device and measurement method, which not only has a larger measurement range and improves the accuracy of out-of-plane building defect detection and measurement, but is also more portable.

[0009] According to a first aspect of an embodiment of the present application, a parallel laser line scanning device is provided, comprising: a laser source, configured to emit two laser beams parallel to each other to project onto a scanned surface; a digital camera, configured to capture an image of the scanned surface, the camera comprising a lens and an image sensor; a positioning rod, comprising a laser positioning arm and a retractable main positioning arm rotatably connected to each other, the camera being arranged at an end of the retractable main positioning arm away from the laser positioning arm, and the laser source being mounted on the laser positioning arm; and a controller, configured to obtain measurement parameters related to the scanning surface by processing the captured image of the scanning surface.

[0010] In some embodiments, the measurement parameter may be the defect depth of the scanned surface, and the controller may obtain the measurement parameter by the following formula:

[0011]

[0012] Among them, n op is the number of pixels between the two parallel laser lines on the image, L0 and l are the sizes of the main positioning arm and the laser positioning arm respectively, θ is the laser projection angle, f is the focal length of the lens, s is the horizontal size of the image sensor, r is the total number of pixels included in the image sensor, H is the measurement distance, and d is the defect depth.

[0013] In some embodiments, the laser source may be a dual-wire laser diode or may include two lasers.

[0014] In some embodiments, the controller may be configured to perform the following steps:

[0015] Step S1: extracting the laser centerline of the image;

[0016] Step S2: recovering two parallel laser baselines from the image;

[0017] Step S3: constructing a three-dimensional measurement model, wherein the three-dimensional measurement model is used to represent the relative position relationship between object points on the scanned surface and image pixels in the image sensor; and

[0018] Step S4: generating two groups of laser point clouds corresponding to the two laser center lines in the image based on the three-dimensional measurement model, and obtaining the three-dimensional coordinates of the points to be measured based on the laser point clouds.

[0019] In some embodiments, step S1 may include the following steps:

[0020] S1a, detecting a region containing laser stripes from the image using a first neural network;

[0021] S1b, segmenting a laser stripe image from the region containing the laser stripes using a second neural network;

[0022] S1c, extracting the laser center line of the laser stripe image.

[0023] In some embodiments, step S1 may include the following steps:

[0024] S1a, detecting a structural defect region including laser stripes from the image using a first neural network;

[0025] S1b, segmenting a laser stripe image from the structural defect region using a second neural network;

[0026] S1c, extracting the laser center line of the laser stripe image.

[0027] In some embodiments, the first neural network may adopt a YOLOv7 target detection network; the second neural network model may adopt a U-Net image segmentation network model. In some other embodiments, the backbone network of the U-Net image segmentation network model adopts a ResNet-50 network architecture.

[0028] In some embodiments, in step S2, two parallel laser baselines can be recovered from the image by a pre-trained 3-layer back-propagation neural network, wherein the loss function adopted by the neural network in training determines whether the laser point belongs to its nearest laser baseline based on whether the distorted pixel of the laser point is less than half of the distance between the two parallel laser lines.

[0029] In some embodiments, the following relative position relationship between the object point and its image pixel in the image sensor is established in step S3:

[0030]

[0031] Where (X c ,Y c ,Z c ,1) is the homogeneous coordinate of the object point in the world coordinate system, (u,v,1) is the homogeneous coordinate in the pixel coordinate system, s is the random scale factor, and A is the 3×3 intrinsic parameter matrix of the camera, including the effective focal length, image sensor size, and principal point.

[0032] In some embodiments, the three-dimensional coordinates of the point to be measured are obtained based on the equation of the laser plane and the equation of the relative position relationship, wherein the equation of the laser plane is:

[0033] A i X c +B i Y c +C i Z c +D i =0

[0034] Among them A i , B i , C i and D i are the physical coefficients describing the single laser plane equation.

[0035] In some embodiments, the controller may be further configured to calibrate the camera and the laser plane.

[0036] According to a second aspect of an embodiment of the present application, a measurement method using the parallel laser scanning device according to the first aspect is provided, comprising: using a laser source to emit two laser beams parallel to each other to project onto a scanned surface; using a camera to capture an image of the scanned surface, the camera comprising a lens and an image sensor; and processing the captured image of the scanned surface to obtain measurement parameters related to the scanned surface.

[0037] In some embodiments, processing the acquired image of the scanned surface to obtain measurement parameters related to structural defects in the scanned surface may include:

[0038] Detecting a structural defect region containing laser stripes from the acquired image of the scanned surface using a first neural network;

[0039] Segmenting a laser stripe image from the structural defect region using a second neural network;

[0040] Extracting the laser center line from the laser stripe image;

[0041] For the laser stripe image, restoring two parallel laser baselines;

[0042] constructing a three-dimensional measurement model for representing the relative positional relationship between object points on the scanned surface and image pixels in the image sensor; and

[0043] Two groups of laser point clouds corresponding to two laser center lines in the laser stripe image are generated based on the three-dimensional measurement model, and the three-dimensional coordinates of the points to be measured are obtained based on the laser point clouds.

[0044] According to the third aspect of the embodiment of the present application, a measurement system based on parallel laser line scanning is provided, which includes a parallel laser line scanning device and a computing device communicatively coupled thereto. The parallel laser line scanning device includes: a laser source, which is configured to emit two beams of laser light parallel to each other to project the scanned surface; a digital camera, which is configured to capture an image of the scanned surface, the camera including a lens and an image sensor; a positioning rod, which includes a laser positioning arm and a retractable main positioning arm rotatably connected to each other, the camera is arranged at the end of the retractable main positioning arm away from the laser positioning arm, and the laser source is mounted on the laser positioning arm; and a controller, which is configured to send the captured image of the scanned surface and the internal parameters of the scanning device to the computing device. The computing device is configured to obtain measurement parameters related to the scanned surface by processing the captured image of the scanned surface.

[0045] Compared with the prior art, this application optimizes the design of the scanning device by introducing a tilted telescopic positioning rod, making the measurement more portable and extending the measurement range from 1m to 4m; and by integrating machine learning into the image processing process, the laser stripe area can be accurately located, the center of the laser stripe can be extracted, and the error caused by image distortion can be minimized. By projecting laser lines onto structural defects, structural defects with unclear surface features (such as protrusions) are more easily detected by machine learning algorithms. As a result, the accuracy, efficiency and automation of out-of-plane building defect detection and measurement are improved, which is conducive to achieving more objective and rapid building condition assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0047] Figure 1 A schematic diagram of a measurement scenario of a parallel laser line scanning device according to an embodiment of the present application;

[0048] Figure 2 A schematic diagram of a laser scanning image processing process according to an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of a process for extracting the center of a laser stripe according to an embodiment of the present invention;

[0050] Figure 4 A schematic diagram of the results of extracting the center of the laser stripe and restoring the baseline according to an embodiment of the present invention;

[0051] Figure 5 A schematic diagram showing a three-dimensional measurement model according to an embodiment of the present invention;

[0052] Figure 6 A schematic diagram of a scanning device for performing experimental testing according to an embodiment of the present invention;

[0053] FIG7( a ) shows a schematic diagram of the result of detecting the laser stripe area;

[0054] FIG7( b ) shows a schematic diagram of the result of segmenting the laser stripes;

[0055] FIG7( c ) shows the result of extracting the stripe center by combining deep learning and Steger algorithm;

[0056] FIG7( d ) shows the result of extracting the stripe center by combining the threshold method and the Steger algorithm;

[0057] FIG8( a ) shows a schematic diagram of the results of detecting the laser stripe area at the convex part;

[0058] FIG8( b ) shows the results of detecting the laser stripe area at the delamination;

[0059] FIG8( c ) shows the results of detecting the laser stripe area at the peeling location;

[0060] FIG9( a ) shows a schematic diagram of target detection network performance according to an embodiment of the present invention;

[0061] FIG9( b ) shows a schematic diagram of image segmentation network performance according to an embodiment of the present invention;

[0062] FIG10( a ) shows a schematic diagram of the laser baseline restoration effect according to an embodiment of the present invention;

[0063] FIG10( b ) shows a schematic diagram of the laser baseline restoration effect according to yet another embodiment of the present invention;

[0064] FIG. 11( a ) and FIG. 11( b ) respectively show the surface of an object used for testing in a laboratory and the schematic diagram of the test results;

[0065] FIG. 12( a ) and FIG. 12( b ) respectively show the surface of an object used for field testing and a schematic diagram of a test result;

[0066] Fig.13 The error diagram for the example scanning distance is 1m to 2m;

[0067] Fig.14 The error diagram for the example scanning distance is 2m to 4m;

[0068] Fig.15 Schematic diagram of measurement error analysis for an example. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail by specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0070] Figure 1 A schematic diagram of a measurement scenario of a novel parallel laser line scanning device for measuring out-of-plane building defects according to an embodiment of the present application is given. The scanning device mainly includes a laser source, a digital camera, a positioning rod and a controller (not shown). The laser source is configured to emit two laser beams parallel to each other to project the scanned surface. The digital camera (also referred to as a camera) is configured to capture images of the scanned surface. The positioning rod is used to install or couple the laser source and the digital camera. The controller is configured to obtain measurement parameters related to the scanned surface by processing the captured image of the scanned surface, such as the width of the wall crack, the depth of the surface bulge, etc. The laser source that can emit two parallel laser lines can be a dual-line laser diode, or a laser source composed of two lasers. The digital camera includes at least a lens and an image sensor. The positioning rod is a retractable metal rod or metal arm, and its length can be adjusted according to the actual measurement scenario.

[0071] like Figure 1As shown, the digital camera is attached to one end of the positioning rod through a coupling mechanism, and the laser source is attached to the rotating hinge 1 at the other end of the positioning rod. The rotating hinge 1 is fixed on the positioning rod and basically maintains the same direction as the positioning rod. During measurement, the digital camera is aimed at the scanned surface, and its horizontal direction is parallel to the measurement reference surface, while the positioning rod is tilted at a certain angle relative to the horizontal direction of the digital camera, and the angle can be adjusted by the coupling mechanism between the digital camera and the positioning rod. The laser source projects parallel laser lines onto the scanned surface at a certain angle.

[0072] In some embodiments, the digital camera and the laser source can be self-powered, each can be independently turned on and off, and each can be detachably coupled to the positioning pole, thereby facilitating portability and outdoor use. Figure 1 The controller is not shown in the figure, but it should be understood that the controller can be implemented by software, hardware, or a combination of software and hardware to achieve the function of obtaining the measurement parameters related to the scanned surface by processing the acquired image of the scanned surface as will be described in detail below. The specific implementation form of the controller is not limited in any way herein. For example, the controller can be a software module running on a digital camera, which is executed by a processor of the digital camera, or the controller can be a hardware module integrated in the digital camera in the form of an embedded chip, FPGA circuit, etc. Alternatively, the controller can be a computing device independent of the digital camera or software running on a computing device independent of the digital camera (such as a smart phone), and the computing device can communicate with the digital camera in a wired or wireless manner to obtain the image and other relevant parameters of the scanned surface acquired by the digital camera, so as to determine the measurement parameters related to the scanned surface.

[0073] The following is based on Figure 1 The example of a scene in which a scanning device according to an embodiment of the present application is used to measure a bulge on a ceiling surface is used to illustrate the measurement principle. In this measurement scene, the ceiling surface is the measurement reference plane, and the bulge is the scanned surface. The laser source projects the laser onto the bulge surface, and then the digital camera lens forms an image of the scanned bulge surface. The camera lens can focus the image of the laser line on the image sensor. Figure 1 As shown in the figure, taking point O on the bulge surface as an example, point O on the surface is focused on point A' on the image sensor through the lens of the digital camera. In the absence of bulge, the laser beam will be projected to point B on the measurement reference plane, and the laser point captured on the image sensor will be offset from A' to B' (the number of offset pixels is recorded as n op), and point A' on the image sensor will correspond to point A on the measurement reference plane. Point C represents the optical center of the lens, and point D indicates the intersection of the reverse extension of the laser line projected by the laser source and the imaging plane of the image sensor. Where θ is the laser projection angle, L0 is the length of the positioning rod, l is the size of the rotary hinge, α is the inclination angle of the positioning rod, s is the horizontal size of the image sensor, and f is the distance between the optical center of the lens and the image sensor (also represents the focal length of the camera lens). Assuming that the imaging plane of the image sensor is parallel to the measurement reference plane and there is no distortion effect in the camera imaging, the bulge depth at point O (denoted as d) can be calculated based on the triangulation angle between the laser diode and the camera. The derivation process of the bulge depth d is as follows:

[0074] refer to Figure 1 , considering the similarity of ΔOAB and ΔOCD, it can be verified that:

[0075]

[0076] Where H is the measured distance between the camera and the ceiling surface, and L is the equivalent horizontal distance between the camera and the laser diode. The equivalent horizontal distance L can be obtained by formula (2):

[0077]

[0078] Where L0 is the length of the positioning rod, α is the tilt angle of the positioning rod, and θ is the laser projection angle.

[0079] Similarly, considering the similarity of ΔCAB and ΔCA′B′, the offset distance A′B′ at the image sensor can be calculated by formula (3) as follows:

[0080]

[0081] Where f is the focal length of the camera. Therefore, the number of pixels offset (n op ) is calculated according to formula (4) as follows:

[0082]

[0083] Where r is the horizontal resolution of the image sensor, and s is the horizontal size of the image sensor.

[0084] Thus, by combining equations (1) to (4), we can obtain the number of offset pixels n: op The relationship between the bulge depth d is as follows:

[0085]

[0086] or

[0087]

[0088] From the above formulas (5) and (6), it can be seen that the bulge depth d of the laser scanning can be obtained by offsetting the number of pixels n op To represent. And the number of pixels offset n op (also referred to as the number of distorted pixels) can be determined using image processing techniques. For example, the number of offset pixels n can be determined by performing image processing on the captured image of the scanned surface by the controller. op , and then the bulge depth d can be calculated based on the above relationship expression. In the embodiment of the present application, the controller obtains the measurement parameters related to the scanned surface by analyzing the two parallel laser lines on the acquired image of the scanned surface, which will be explained in detail in the subsequent content.

[0089] And, you can op / d (i.e., the number of pixels per millimeter) is used to represent the measurement sensitivity, which is used to reflect the ability to detect small height changes. From the above formula (6), it can be seen that the measurement sensitivity is closely related to the relevant settings of the scanning device (for example, the length of the positioning rod, the inclination angle of the positioning rod, the laser projection angle, etc.). In actual measurement, the required measurement requirements, such as detecting small bulges, can be met by setting the relevant parameters in equation (6).

[0090] The positioning rod used in the parallel laser line scanning device provided in the embodiment of the present application is a retractable rod, the length of which can be flexibly adjusted, and the angle between the positioning rod and the digital camera is controllable. Thus, by adjusting the length L0 of the positioning rod and the angle between the positioning rod and the digital camera, the parameters such as the inclination angle of the positioning rod and the laser projection angle can be flexibly set, thereby adjusting the measurement sensitivity. Unlike the prior art, the scanning device does not rely on complex components, has a simple structure, can be handheld for scanning, and the length and inclination of the positioning rod can be adjusted, and the defects of various building surfaces (such as ceilings, beams, walls, etc.) can be flexibly measured. In addition, the scanning device is durable and affordable, and the components used (such as consumer-grade digital cameras and line laser generators, etc.) are easy to buy on the market and are relatively low in price, thereby making it a more cost-effective alternative to expensive three-dimensional terrestrial laser scanners. In addition, the scanning device flexibly adjusts the distance between the laser source and the measurement target and the equivalent horizontal distance between the laser source and the digital camera through the retractable positioning rod, thereby flexibly adjusting the measurement range and measurement sensitivity. And the controller of the scanning device also integrates an optimized image processing method to improve measurement accuracy (described in detail below).

[0091] As mentioned above, the measurement parameters related to the scanned surface can be obtained by performing image processing on the acquired image of the scanned surface. opAn inaccurate measurement will inevitably lead to an inaccurate calculation of the bulge depth d. Any changes made during scanning to the length of the positioning rod, the tilt angle of the positioning rod, the laser projection angle, etc. will also cause n op Change to n′ op , as shown below:

[0092]

[0093] Where L'0, α' and θ' represent the changed parameters. Accordingly, the bulge depth d calculated using the pre-calibrated parameters in equation (6) will also be modified to d'. This results in a measurement error. The measurement error of the offset pixel number is It can be expressed as:

[0094]

[0095] Depth measurement error (E d ) can be expressed as:

[0096]

[0097] From equation (6), we know that n op and d change in series, that is Integrating equations (6) to (9), we can obtain:

[0098]

[0099] In this application, equation (10) is used to evaluate the measurement error caused by the calibration of the scanning device, including the installation error during scanning and the error caused by the operation such as the movement of the device. In addition, in actual scenes, the imaging plane of the camera is often not parallel to the measurement reference plane, and image distortion effects may also cause inaccurate measurements.

[0100] In order to further improve the accuracy of measurement, an optimized image processing method is provided in an embodiment of the present application to process the image of the scanned surface captured by the digital camera (also referred to as the laser scanning image). Figure 2 The figure shows a schematic diagram of the process of image processing for laser scanning images. The process mainly includes: determining the center pixel coordinates (u, v) of each laser stripe in the laser scanning image to be processed, thereby extracting the laser center line; then using the baseline recovery algorithm to identify two groups of undistorted laser points; then, using the image model to establish the center pixel coordinates (u, v) and the corresponding three-dimensional spatial coordinates (X c ,Y c ,Z c) to obtain a three-dimensional measurement model. In this way, based on the transformed three-dimensional coordinates of the undistorted laser point, the relative position relationship between the camera and the scanned surface can be effectively defined, so that the depth profile of the scanned surface can be generated. Using the above image processing method, regardless of the shooting angle, the three-dimensional profile of the scanned surface can be automatically generated according to the laser scanning image, and then the parameters related to the corresponding defect measurement can be obtained based on such a profile, which is crucial for accurate defect measurement. In the entire image processing process, the extraction process of the center line of the laser stripe, the restoration of the laser baseline, and the calibration of the image model will affect the accuracy of the measurement. The above sub-processes are introduced below.

[0101] 1. Laser centerline extraction

[0102] In laser triangulation, the laser beam usually forms laser stripes on the image sensor instead of precise lines. Such stripes are relatively wide and not suitable for direct positioning. Accurate extraction of the center coordinates of the laser stripes is crucial for capturing the outline and size of building defects. In order to ensure efficient and robust extraction while reducing interference, a new laser stripe center extraction method is provided in one embodiment of the present application. The process of the method is as follows Figure 3 As shown, it mainly includes the following steps:

[0103] (a) Automatic detection of regions containing laser stripes from laser scanning images using a neural network.

[0104] The inventors found in their research that the computational cost of directly extracting the center line of the laser stripes from the entire scanned image is very high and is easily interfered by redundant information. This is because the laser stripes themselves account for a very small proportion of the laser scanned image captured by the digital camera. If the region of interest is set manually, it cannot adapt to building defects of different sizes and positions, and it is easy to introduce artificial errors. Therefore, in order to improve the accuracy of stripe center extraction, in this embodiment, a pre-trained neural network (for the convenience of distinction, it can also be called a first neural network or a target detection network) is used to automatically identify the area containing laser stripes (referred to as the laser stripe area) from the laser scanned image. In such a laser stripe area, the proportion of laser stripes is significantly increased, which is more conducive to the extraction of the center line of the laser stripes.

[0105] like Figure 3 As shown, in this embodiment, the 7th version of the YOLO network model (denoted as YOLOv7) is used to automatically detect the laser stripe area on the scanned surface in the laser scanning image. YOLOv7 is the most advanced target detection network, which can accurately locate the laser stripe area by classifying the probability and regressing the boundary position coordinates.

[0106] (b) Using a neural network, a laser stripe image is segmented from the laser stripe region obtained in step (a).

[0107] In order to speed up the stripe center extraction process, the inventors further processed the laser stripe area obtained in step (a) to segment the laser stripe image therefrom. A commonly used stripe segmentation technique can be a threshold-based segmentation method. For example, laser stripes are segmented based on intensity thresholding, thereby isolating the laser stripes from the surrounding area in the image. Determining a suitable threshold is crucial because it directly affects the extraction results. Setting a high threshold may result in the exclusion of certain laser pixels, resulting in the loss of extracted centerline details. Conversely, a low threshold may include unwanted background pixels, thereby compromising extraction speed and accuracy.

[0108] To overcome these limitations, in yet another embodiment, a pre-trained neural network (for ease of distinction, it may be referred to as a second neural network or an image segmentation network) is used to automatically segment the laser stripe image from the laser stripe region. Figure 3 As shown, in this embodiment, a U-Net network model is used to segment the laser stripes. The U-Net network model uses convolutional layers and downsampling pooling layers (encoders) to learn the distribution characteristics of laser stripes at different scales; and then convolutional layers and upsampling layers (decoders) are also used to output segmented laser stripe images. U-Net is used as an image segmentation model, and by fusing downsampling and upsampling features, low-level details and high-level global features are effectively fused, thereby obtaining accurate contextual information and improving the segmentation performance of laser stripes. In another embodiment, the backbone of the above-mentioned U-Net network model can also be replaced by ResNet-50. ResNet-50 is a convolutional neural network architecture known for its deeper structure and jump connections, which can produce better feature extraction and information flow. Such model modifications enhance the capabilities of the segmentation process and help to obtain more accurate and robust results.

[0109] (c) Extracting the laser center line from the laser stripe image obtained in step (b).

[0110] Common laser stripe center extraction algorithms include extreme value method, centroid method, template matching method, Steger method and other improved methods based on them. Among these algorithms, Steger method (Steger, C., Ulrich, M., & Wiedemann, C. (1998). Machine vision algorithms and applications. Wiley-VCH.) provides higher accuracy and robustness, especially in the presence of noise and interference. Therefore, in this embodiment, the Steger method is used to extract the laser center line, but other extraction algorithms are not excluded. This method uses the Hessian matrix to estimate the normal direction of the laser stripe for sub-pixel centroid extraction. However, the Steger method requires higher computing resources because it relies on matrix calculations, especially using the Steger method to directly extract the laser stripe center line from the entire scanned image, and the amount of calculation is very large. However, in the embodiment of the present application, the laser stripe area has been identified from the laser scanning image by using the YOLOv7 model, which significantly reduces the amount of data to be processed; and then the laser stripe image is segmented by the U-Net network, thereby suppressing a large amount of background and interference noise, thereby significantly reducing the processing time required for the subsequent extraction of the center line using the Steger method. Therefore, in this embodiment, the center pixel coordinates (u, v) of the laser stripe can be accurately extracted from the segmented laser stripe image by using the Steger method. This combination of deep learning-based technology and the Steger method not only improves accuracy and anti-interference ability, but also improves processing speed, providing an effective and efficient solution for laser stripe center extraction.

[0111] In another embodiment, a deep learning technique can also be used to first identify a structural defect region containing laser stripes from a laser scanning image, and subsequent processing is only performed on the identified structural defect region, thereby improving the detection and measurement effect of out-of-plane structural defects. More specifically, in this embodiment, in step (a), a pre-trained first neural network or target detection network is used to automatically detect a structural defect region containing laser stripes from a laser scanning image. By projecting a laser line onto the structural defect, a structural defect (such as a bulge) with unclear surface features is more easily detected by a machine learning algorithm, so that the region where the structural defect exists out of the plane can be more accurately located, which is more conducive to the extraction of the center line of the laser stripes and the improvement of the accuracy and efficiency of subsequent measurements. In step (b), a pre-trained second neural network or image segmentation network is used to segment a laser stripe image from the structural defect region obtained in step (a). Such a segmentation process helps to obtain more accurate and robust measurement results. In step (c), the Steger method is used to extract the laser center line of the laser stripe image obtained from the structural defect region in step (b).

[0112] In the scheme of this embodiment, the structural defect area containing laser stripes is first detected from the laser scanning image through the target detection network, which not only significantly reduces the amount of data that needs to be processed by the subsequent Steger method, but also more accurately locates the defect area, thereby improving the efficiency and accuracy of subsequent defect measurement; and then the laser stripe image is segmented from the detected structural defect area through the image segmentation network, thereby suppressing a large amount of background and interference noise, which is beneficial to improving the accuracy and anti-interference ability of defect measurement, thereby achieving more objective and rapid building condition assessment.

[0113] 2. Laser baseline recovery

[0114] From the laser scanning images, it can be seen that the captured laser line will show deformation or distortion in the damaged or defective area, while remaining intact on the flat surface. Therefore, two parallel laser baselines must be restored to establish a reference plane for the subsequent depth profile calculation of the damaged or defective area.

[0115] In one embodiment, the RANSAC method can be used to restore two parallel laser baselines. The RANSAC method (Fischler, MA, & Bolles, RC (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24 (6), 381-395.) is well known for its powerful regression function. It divides the data set into inliers and outliers and fits the model only to the inliers. In the above laser centerline extraction algorithm, a batch of feature point coordinates (x i ,y i )∈(x, y), which includes the undistorted laser points, the distorted laser points and the noisy background points (represented as pixel areas with uneven light intensity distribution), as shown in Figure 4 As shown in . By identifying undistorted laser points as inliers and other feature points as outliers, RANSAC effectively reduces the impact of erroneous feature points on the ideal model line. The RANSAC algorithm is executed twice to estimate two baselines from feature points. First, RANSAC is called with feature points to restore the first laser baseline (y=k1x+b1). A residual threshold is set to identify model inliers. Feature points belonging to the first laser line are then removed by defining a distance limit from the first model. Subsequently, RANSAC is called again with the cropped feature set to restore the second laser baseline (y=k2x+b2). ​​The two restored laser baselines are actually embodied as two sets of undistorted laser points.

[0116] However, the above solution does not satisfy all the constraints of the problem. For example, the laser line equation obtained by RANSAC should not be passed through the residual error (y prediction -y) to fit, but should be fitted by the minimum orthogonal distance between the model and the data points. In addition, RANSAC does not consider parallel constraints, which ignores the impact of image distortion (which will be explained in the subsequent process).

[0117] In order to solve the above limitations, in another embodiment, a hypothesis optimization (HO) method based on a three-layer back-propagation neural network is provided, which can satisfy all assumptions in this context, thereby achieving high-precision measurement. Back-propagation is a commonly used form of automatic differentiation in gradient descent optimization, and is still one of the most important methods in the field of machine learning, especially in the field of neural networks and deep learning.

[0118] In the context of this problem, assuming that the model has two parallel baselines, y = kx + b1 and y = kx + b2, and considering that the line distortion pixels caused by defects are less than half the distance between the laser lines, the distortion of the feature point can be quantified as the distance orthogonal to the nearest laser baseline:

[0119]

[0120] Assuming that most feature points belong to the baseline rather than the distorted region, it is reasonable to minimize the distortion of all feature points to determine the equation for the baseline. Therefore, the optimization problem can be formulated as a loss function that ensures that the parallel baseline model is only affected by the inliers, as determined by the residual threshold discussed in the RANSAC algorithm. The loss function is defined as follows:

[0121]

[0122] The distance limit is used as a threshold to determine whether a feature point belongs to its nearest laser baseline.

[0123] The loss function introduced is used to quantify the difference between the observed feature points and two parallel baselines. The function is designed to be differentiable with respect to parameters k, b1 and b2, realizing the expression kx i and -y i +b i Represented as a matrix form (x, y)×A T +b, where A = [k, -1]. In this formula, the parameters of the optimization problem can be likened to the weights of a linear layer in machine learning. The Adam optimizer is a popular gradient-based optimization algorithm used to efficiently compute the parameters and minimize the loss function. By leveraging backpropagation, the loss function over all feature points can be efficiently minimized, which helps to obtain the optimized parameters of (k, b1, b2) for the parallel baseline model. These optimized parameters obey all given constraints and provide an accurate representation of the parallel baseline model.

[0124] 3. Establish a 3D measurement model

[0125] The 3D measurement model can also be called a 3D laser camera measurement model, which is used to characterize the point P (X c , Y c , Z c ) and its corresponding image pixel coordinates P′(u, v), as Figure 5 This relationship is defined by using geometric and trigonometric associations, as described above in conjunction with the measurement principle. Where O-XYZ represents the world coordinate system, and O is the origin of the world coordinate system. c -X c Y c Z crepresents the camera coordinate system, O c is the origin of the camera coordinate system. o-xy represents the image coordinate system, and o is the origin of the image coordinate system. uv-uv represents the pixel coordinate system, O uv is the origin of the pixel coordinate system. Point P is the laser plane π p A point on the curve that intersects the surface of the object being measured. Point P′ is the ideal projection point of point P on the image plane. Applying the measurement principle, the pinhole camera model can be used to establish the positional association between the measurement point (P) and its image pixel (P′):

[0126]

[0127] where s represents the random scaling factor, and A is a 3×3 matrix of camera intrinsic parameters, including the effective focal length, camera image sensor size, and principal point. Equation (13) shows that along line O c Any spatial point of P can be projected onto the same pixel position (P′) in the image. In addition, the three-dimensional coordinates of the measurement point P satisfy the laser plane equation π P , which is expressed as the following formula (14):

[0128] A i X c +B i Y c +C i Z c +D i =0 (14)

[0129] Among them A i , B i , C i and D i is the physical coefficient describing the single laser plane equation. Combining (13) and (14) can realize the calculation of the spatial three-dimensional coordinates of the point P to be measured.

[0130] Considering that the camera model is an unrealistic linear imaging model, practical applications need to take into account nonlinear distortion or distortion caused by manufacturing and installation errors. Radial distortion comes from the appearance of the camera lens, while tangential distortion comes from the camera assembly process. Ideal laser point P′ u (x u ,y u ) and the captured laser point P′ d (x d ,y d ) can be expressed in the image coordinate system as follows:

[0131]

[0132] The coefficients k1, k2, k3 correspond to the radial distortion coefficients of the camera, while p1, p2 represent the tangential distortion coefficients, x d and d It can be understood as the coordinates before camera distortion calibration, x u and u It can be understood as the coordinates after camera distortion calibration

[0133] Therefore, if the system parameters (such as camera parameters, including internal parameters and distortion coefficients, etc.) and laser plane parameters are known, the 3D point cloud of the two projected laser lines can be obtained. The calibration process of these parameters is described in detail in the experimental section below.

[0134] In another embodiment, a coordinate alignment step is also included. Based on the three-dimensional measurement model constructed above, the three-dimensional coordinates of the laser feature points can be obtained from the camera's field of view. However, in actual scanning scenarios, the camera is often not parallel to the scanning surface, such as Figure 5 As shown in . In order to transform the 3D point cloud of the two projected laser lines into a measurable depth profile, it is necessary to restore the measurement reference plane for coordinate alignment. This process involves restoring the measurement reference plane using two sets of internal feature points identified by the laser baseline recovery algorithm. The measurement reference plane fitting equation can be expressed as follows:

[0135] A0X+B0Y+C0Z+D0=0 (17)

[0136] Where X, Y and Z are the global point cloud coordinates of the two sets of laser internal points, and A0, B0, C0 and D0 are the coefficients of the surface plane equation. Once the measurement reference plane is constructed, the aligned laser coordinates P 0(X,Y,Z) can be restored by applying the following transformation equation:

[0137]

[0138] Where T1 is the transformation matrix that defines the orientation and position information of the measurement reference plane. The above transformation equation actually orients the point cloud to a non-tilted direction. In this way, after the de-tilting process, the aligned point cloud can be used to separate defect points from the normal surface. Points that satisfy the plane equations mentioned above are considered to be normal surfaces, while other points are considered to be part of the structural defects. It can be seen that the alignment process can obtain the depth profile of the out-of-plane building defects, and further obtain the measurement parameters related to various defects on the scanned surface.

[0139] The above process of processing the laser scanning image to obtain the measurement parameters can be executed by the controller of the scanning device, and can also be completed by an external computing device that is operably connected to the scanning device.

[0140] In another embodiment of the present application, a method for measuring structural defects using the parallel laser scanning device described above is also provided. The method mainly includes: step 1) using a laser source to emit two beams of laser light parallel to each other to project onto the scanned surface; step 2) using a camera to capture an image of the scanned surface; step 3) processing the captured image of the scanned surface to obtain measurement parameters related to the structural defects in the scanned surface. Wherein, in step 3), a first neural network is first used to detect a structural defect region containing laser stripes from the captured image of the scanned surface; then, a laser stripe image is segmented from the structural defect region using a second neural network, and a laser center line is extracted from the laser stripe image; and for the laser stripe image, two parallel laser baselines are restored, and then a three-dimensional measurement model as described above is constructed, and two groups of laser point clouds corresponding to the two laser center lines extracted from the laser stripe image are generated based on the three-dimensional measurement model, and the three-dimensional coordinates of the point to be measured are obtained based on the laser point cloud.

[0141] In another embodiment of the present application, a measurement system based on parallel laser line scanning is also provided, which includes a parallel laser line scanning device and a computing device communicatively coupled thereto. The parallel laser line scanning device includes the above-mentioned laser source, which is configured to emit two beams of laser light parallel to each other to project the scanned surface; the above-mentioned digital camera including at least a lens and an image sensor, which is configured to collect images of the scanned surface; the above-mentioned positioning rod, which includes a laser positioning arm and a retractable main positioning arm rotatably connected to each other; and a controller, which is configured to send the collected image of the scanned surface and the internal parameters of the scanning device to the computing device. In the parallel laser line scanning device, the camera is detachably arranged at the end of the retractable main positioning arm away from the laser positioning arm, and the laser source is detachably mounted on the laser positioning arm. The computing device can communicate with the controller of the parallel laser line scanning device in a wired or wireless manner, and based on the information received from the controller, obtains the measurement parameters related to the scanned surface by processing the collected image of the scanned surface. In addition, the training and execution of each model mentioned above can also be performed on the computing device.

[0142] Experiment and analysis

[0143] In order to evaluate the measurement accuracy of the parallel laser line scanning device of the present application, the inventors also conducted tests in laboratory and field environments. The scanning device used for the experimental test is as follows: Figure 6As shown in Figure 1, it includes a digital camera, two laser diodes, a battery for powering the laser diodes, and a retractable positioning rod. Table 1 gives the specific hardware device parameters. The scanning device is specially designed to meet the requirements of building scanning and can perform accurate measurements within a range of 1m to 4m. The processing of the collected laser scanning images is implemented in Python and runs on a computer equipped with an Intel i7-6700, 3.40GHz CPU and 16GB RAM.

[0144] Table 1 Hardware equipment parameters

[0145]

[0146] 1) System calibration

[0147] The scanning device is customized to accommodate predetermined measurement spans from 1m to 4m. For optimal measurement results and ease of operation, the parameters l, α and θ are set to 0.2m, 36° and 60°, respectively. For measurement distances of 1m-2m and 2m-4m, L0 adopts values ​​of 0.5m and 1.5m, respectively. These parameter settings enhance the adaptability of the scanning device, making it conducive to scanning a wide range of measurement environments. In addition, by ensuring that the captured laser line is almost in the center of the scanned image, the adverse effects of image distortion on the measurement results can be minimized. This strategy guarantees consistent, accurate and trustworthy defect detection in different scanning scenarios.

[0148] Before performing any measurement task, the scanning device should be fully calibrated so that the subsequent image processing process can accurately generate a measurable depth profile based on the scanned image. The system calibration process involves calibrating the parameters of the image model, including both the camera and laser plane parameters. Camera calibration is performed by placing a chessboard in different positions and orientations. Then the camera intrinsic parameters and distortion coefficients are determined using a camera calibrator toolbox based on least squares fitting and maximum likelihood estimation (Zhang, Z. (2000). A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (11), 1330-1334.). Thereafter, a single laser line chessboard is used for laser plane calibration. By adjusting the position and orientation of the chessboard, a set of 3D lines defining the laser plane is established. The results of the calibration are listed in detail in Table 2.

[0149] Table 2

[0150]

[0151] 2) Laser stripe processing

[0152] 2a) Stripe area detection

[0153] (1) Dataset Collection

[0154] Images of typical out-of-plane building defects were collected via laser scanning. These defects, such as bulges and spalling, originated from various aging sites, including old residential buildings and parking lots. To ensure model generalization and avoid overfitting, images were taken at different distances and lighting settings. A total of 850 images were collected for the YOLOv7 model, each with a resolution of 6000×4000 and resized to 640×640. The laser stripe area was then cropped into smaller slices and resized to 512×512 to train the U-Net model. Only 350 laser stripe area images were manually segmented due to the difficult and time-consuming labeling process. To improve training performance and enrich the dataset, these images were increased by 1000 by image mirroring and random rotation. Both deep learning models were divided into training, validation, and test sets in a ratio of 6:2:2. The laser stripe images in the test set were collected from different construction sites from the training set to test the generalization ability.

[0155] (2) Training and testing process

[0156] TensorFlow was used as a deep learning framework to train and test the network model. Training was performed on Google Colab using an NVIDIA T4 GPU with 16GB RAM. The optimization algorithm used during training was Adaptive Moment Estimation (Adam). The learning rate was set to 0.001, the maximum number of iterations was 20,000, and each batch contained 10 training samples. The intersection over union (IOU) threshold of non-maximum suppression (NMS) was set to 0.7. The entire training process lasted about 6 hours. Figure 7(a) shows a schematic diagram of the results of detecting the laser stripe area. Figure 7(b) shows a schematic diagram of the results of segmenting the laser stripes.

[0157] Both the YOLOv7 and U-Net models show effective convergence after the corresponding training cycles. After evaluation on the test set, the performance of the YOLOv7 model is evaluated using the precision-recall (PR) curve. As shown in Figure 9(a), the model achieves an impressive average precision (AP) of 0.99 at an IoU of 0.5, highlighting its accurate detection of the laser stripe region. On the other hand, the performance of the U-Net model is represented by the confusion matrix in Figure 9(b). The matrix quantifies the model's effectiveness in distinguishing the laser line and background regions, with IoU scores of 0.910 and 0.996, respectively. The combined average IoU of the two classes is 0.953. Assuming an IoU threshold of 0.9 for successful detection, the U-Net model can effectively segment the laser stripes. The results show that both the YOLOv7 and U-Net models can accurately detect and segment the laser stripe region, respectively. In addition, YOLOv7 is used to specifically detect the laser line region near the structural defect. The experimental results show that the AP values ​​of bulges, delamination and peeling are 0.63, 0.69 and 0.90 respectively. Please refer to Figure 8(a)-(c) for the specific detection results. As more data is collected, the detection performance is expected to be further improved. It is worth mentioning that by projecting laser lines on the structural defects, even structural defects with less obvious features (such as bulges) can be accurately identified by the deep learning model.

[0158] 2b) Stripe center extraction

[0159] To evaluate the effectiveness of the proposed deep convolutional neural network-based stripe center extraction algorithm, comparative experiments were conducted using scan samples captured under different lighting conditions. Figure 7(a) shows the YOLOv7 detection results for scanned images captured under bulb lighting, strong light, and cloudy conditions, respectively. Figure 7(b) shows the U-net segmentation results for scanned images captured under bulb lighting, strong light, and cloudy conditions, respectively. Figure 7(c) shows the center extraction results of deep learning combined with the Steger algorithm, while Figure 7(d) depicts the results obtained using the threshold method and the Steger algorithm. The stripe center extraction performance can be evaluated by comparing Figure 7(c) and Figure 7(d). It is obvious that under bulb lighting, the extraction results in Figure 7(d) contain noise information near the bulb that deviates from the actual center of the laser stripe. Under strong light conditions, the Steger algorithm cannot extract the center of the stripes in the dark defect area due to its weak intensity and inability to calculate the Hessian matrix. The extraction results under cloudy lighting conditions are similar because the relatively bright laser stripes eliminate the background noise under cloudy lighting conditions. Comparing the extraction speed using the Steger algorithm, it is obvious that the addition of the laser stripe recognition algorithm can significantly improve the extraction speed. Comparative experiments show that the proposed fringe center extraction algorithm based on deep convolutional neural network exhibits higher efficiency and better noise suppression ability, especially when dealing with weak laser fringe information or strong interference caused by additional light sources.

[0160] 2c) Laser baseline restoration

[0161] Using the data set of extracted laser stripe center points, RANSAC and HO are used to restore the two laser baselines respectively. RANSAC is used for baseline restoration in Figure 10(a). HO is used for baseline restoration in Figure 10(b). The RANSAC method is considered indirect because the baselines are not generated simultaneously, but through two independent RANSAC processes. The second RANSAC process operates on a cropped subset of data calculated based on the first prediction. It is worth noting that RANSAC does not enforce parallel constraints between lines, nor does it use orthogonal distance regression to fit the laser baselines. In contrast, the HO method meets these criteria, making it a suitable optimization technique for baseline reconstruction.

[0162] By conducting baseline restoration experiments on 30 field defect samples using RANSAC and HO, it was observed that the average convergence time of the two methods was 0.07s and 3.84s, respectively. This indicates the superior efficiency of RANSAC. Nevertheless, HO ensures consistent and trustworthy convergence to the expected solution. As shown in Figure 10(a), RANSAC successfully identifies and parameterizes two laser baselines, as shown in the orange and green lines. This shows the robustness of RANSAC in dealing with outliers as they have little impact on the final prediction. The RANSAC method exhibits local error exclusion as it accurately fits the baseline without biasing towards the distorted part. However, at the right end of the lower laser line, there is a deviation from the true fringe center, indicating the presence of image distortion that affects the baseline recovery process. Despite using calibrated camera parameters, the distortion still affects the overall baseline recovery of the entire image. A possible solution to reduce this error is to limit the analysis to the middle part of the image, where the distortion is minimal.

[0163] Using the optimization method, the samples were further processed using the HO method to achieve baseline recovery. The parameters were updated for more than 5000 cycles and the optimal historical prediction was selected as the final result, as shown in Figure 10(b), which is similar to Figure 10(a) and is not affected by gross errors and local error distortions. It is worth noting that the prediction in Figure 10(b) exhibits more accurate alignment without upward tilt. This can be attributed to the simultaneous fitting of the two laser lines in the BP neural network, which helps to eliminate the errors caused by distortion. This observation highlights the importance of ensuring parallel projection of the laser lines during hardware calibration, as it can effectively reduce the algorithmic error in the pipeline when using the HO method.

[0164] 3) Measurement and discussion

[0165] 3a) Verification of measurement results

[0166] The depth measurement capability of the scanning device was first evaluated through laboratory testing. As shown in Figure 10(a), the test involved the measurement of a stepped gypsum block. The depth of this stepped gypsum block varied by 10 mm, reflecting the characteristics of a typical building defect. The evaluation was carried out under controlled indoor lighting conditions. The stepped block was gradually moved from 1m to 4m to verify the suitability of the scanning device for on-site building scanning.

[0167] from Figure 11(a) and 11(b)It is obvious from the visual results presented in that both the fringe center and the laser baseline are accurately detected and reconstructed. In addition, the depth profile is effectively reconstructed, demonstrating the feasibility of the proposed image processing method. In Figure 11(b), small oscillations in depth are observed along the laser line, which can be regarded as measurement errors. By measuring samples at different distances from 1m to 4m, it is inferred that the maximum fluctuation observed remains within 0.5mm. The evaluated errors confirm that the designed measurement system achieves the required accuracy for detecting early-stage building defects.

[0168] Figures 12(a) and (b) show the object surface used for field measurement and the test results, respectively. A high-precision laser rangefinder with a precision of 0.1 mm was used to verify the field measurement results using the proposed system. An example is shown in Figure 12(a), where a tiny ceiling bulge was scanned using the proposed setup. At the same time, the laser rangefinder moved along the level and performed point-to-point measurements at seven points along the projected laser line. The seven-point measurement results were then aligned with the reconstructed depth profile, as shown in Figure 12(b). It is worth noting that the difference between the measurement results of the two methods remained within 0.5 mm. Such a close alignment demonstrates the effectiveness of the scanning device of the present application in measuring building defects. By iterating the algorithm for 30 field scan samples, the calculation time on the GPU was approximately 4.5 seconds and the calculation time on the CPU was approximately 6.2 seconds.

[0169] 3b) Error analysis

[0170] The inventors have also evaluated the measurement errors caused by the handheld scanning device when operating in the field. The handheld nature of the parallel laser line scanning device introduces the possibility of changes in the relative position between the camera and the laser diode. Factors such as the assembly of the instrument, inaccurate positioning of the rod, or the swinging of the arm during scanning can cause their relative position to change. This can cause measurement errors in the scanning system. Figure 1 It is evident from the scanning device prototype shown in that the parameters L0, α and θ can affect the relative position of the laser diode and the camera. By considering small variations of L0, α and θ, such as ΔL0 = ±10 mm, Δα = ±2°, and Δθ = ±2°, the error caused by the relative position variation between the laser diode and the camera can be analyzed using equation (9). Two different setup configurations are considered for scanning distances of 1-2 m and 2-4 m, respectively. It is worth noting that an increase in α will result in a decrease in θ.

[0171] from Fig.13It can be seen that when the scanning distance is 1 to 2m, the measurement error increases linearly with the increase of L0, α and θ variance, with growth rates of 0.15%, 2% and -1.5% respectively. Similarly, for the setting of scanning distance of 2-4m, the measurement error also increases linearly with the increase of L0, α and θ variance, with growth rates of 0.06%, 2.4% and -1.8% respectively (as shown in Figure 2). Fig.14 ). In contrast, when the rod length is increased to obtain longer distances, the system is less sensitive to changes in rod length but more sensitive to rod tilt angle and laser angle.

[0172] Assumptions When H>>h should be verified. Considering n op The variance of , can be calculated using equation (6) to calculate the corresponding d value. Then the error in d can be calculated. The comparison results are as follows Fig.15 As shown in Represents the measurement error of laser distortion pixels (n op ) on the defect depth measurement error (d). It is observed that As the distance gets closer to the depth, the shorter the measurement distance, the smaller it is. For 1-2m and 2-4m measurement settings, The values ​​of are over 95.5% and 97% respectively. Fig.13 and Fig.14 The conclusions drawn are solid and supported by data.

[0173] From the above experiments, it can be seen that the scanning device and measurement method for out-of-plane building defect measurement provided in the embodiments of the present application, by combining parallel laser line scanning with an image processing algorithm based on machine learning, demonstrate the ability to accurately measure defects, with an error range of less than 0.5 mm, meeting the requirements for detecting and measuring early bulges. Through the above evaluation of the scanning device and its image processing process under laboratory and field conditions, it can be seen that it is robust against different lighting conditions and complex backgrounds. Even under challenging lighting conditions, the scanning device using the embodiments of the present application can accurately detect and quantify building defects.

[0174] Although the present invention has been described through preferred embodiments, the present invention is not limited to the embodiments described herein but includes various changes and modifications that may be made without departing from the scope of the present invention.

Claims

1. A parallel laser line scanning device, comprising: a laser source configured to emit two laser beams parallel to each other to project onto the scanned surface; a digital camera configured to capture an image of the scanned surface, the camera comprising a lens and an image sensor; a positioning rod, comprising a laser positioning arm and a telescopic main positioning arm rotatably connected to each other, the camera being arranged at an end of the telescopic main positioning arm away from the laser positioning arm, and the laser source being mounted on the laser positioning arm; and The controller is configured to obtain measurement parameters related to the scanning surface by processing the acquired image of the scanning surface.

2. The parallel laser line scanning device according to claim 1, wherein: The measurement parameter is the defect depth of the scanned surface, and the controller obtains the measurement parameter according to the following formula: Among them, n op is the number of pixels between the two parallel laser lines on the image, L0 and l are the sizes of the main positioning arm and the laser positioning arm respectively, θ is the laser projection angle, f is the focal length of the lens, s is the horizontal size of the image sensor, r is the total number of pixels included in the image sensor, H is the measurement distance, and d is the defect depth.

3. The parallel laser line scanning device according to claim 1, wherein: The laser source is a dual-line laser diode or includes two lasers.

4. The parallel laser line scanning device according to claim 1, wherein: The controller is configured to perform the following steps: Step S1: extracting the laser centerline of the image; Step S2: recovering two parallel laser baselines from the image; Step S3: constructing a three-dimensional measurement model, wherein the three-dimensional measurement model is used to represent the relative position relationship between object points on the scanned surface and image pixels in the image sensor; as well as Step S4: generating two groups of laser point clouds corresponding to the two laser center lines in the image based on the three-dimensional measurement model, and obtaining the three-dimensional coordinates of the points to be measured based on the laser point clouds.

5. The parallel laser line scanning device according to claim 4, wherein: The step S1 includes the following steps: S1a, detecting a region containing laser stripes from the image using a first neural network; S1b, segmenting a laser stripe image from the region containing the laser stripes using a second neural network; S1c, extracting the laser center line of the laser stripe image.

6. The parallel laser line scanning device according to claim 4, wherein: The step S1 includes the following steps: S1a, detecting a structural defect region including laser stripes from the image using a first neural network; S1b, segmenting a laser stripe image from the structural defect region using a second neural network; S1c, extracting the laser center line of the laser stripe image.

7. The parallel laser line scanning device according to claim 5 or 6, wherein in step S2, two parallel laser baselines are restored from the image by a pre-trained 3-layer back-propagation neural network, wherein the loss function used in the training of the neural network determines whether the laser point belongs to its nearest laser baseline based on whether the distorted pixel of the laser point is less than half of the distance between the two parallel laser lines.

8. The parallel laser line scanning device according to claim 5 or 6, wherein: In step S3, the following relative position relationship between the object point and the image pixel in the image sensor is established: Where (X c ,Y c ,Z c ,1) is the homogeneous coordinate of the object point in the world coordinate system, (u,v,1) is the homogeneous coordinate in the pixel coordinate system, s is the random scale factor, and A is the 3×3 intrinsic parameter matrix of the camera, including the effective focal length, image sensor size, and principal point.

9. The parallel laser line scanning device according to claim 8, wherein: The three-dimensional coordinates of the point to be measured are obtained based on the equation of the laser plane and the equation of the relative position relationship, wherein the equation of the laser plane is: A i X c +B i Y c +C i Z c +D i =0 Among them A i , B i , C i and D i are the physical coefficients describing the single laser plane equation.

10. A method for measuring structural defects using the parallel laser scanning device according to any one of claims 1 to 9, comprising: A laser source is used to emit two laser beams parallel to each other to project onto the scanned surface; Acquire an image of the scanned surface with a camera, the camera comprising a lens and an image sensor; The acquired image of the scanned surface is processed to obtain measurement parameters related to the structural defects in the scanned surface.

Images