Structural dynamic displacement monitoring method based on YOLOv5 network model
Through ring-encoded marking and subpixel-level edge detection based on YOLOv5 network model, the problem of insufficient displacement monitoring accuracy in complex environments in traditional methods is solved, and high-precision dynamic structural displacement monitoring is achieved, which is suitable for civil infrastructure.
Patent Information
- Application Number
- CN202510566772.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2025-08-08
AI Technical Summary
Traditional contact displacement monitoring methods are time-consuming and labor-intensive, and video-based monitoring technology reduces measurement accuracy in complex environments. Traditional object detection algorithms are susceptible to light and occlusion, resulting in large errors in structural vibration displacement detection.
The structure dynamic displacement monitoring method based on the YOLOv5 network model is adopted, and the data sets in complex environments are established by introducing ring coding marks, and the YOLOv5 model is trained for initial positioning, and the subpixel-level edge detection and ellipse fitting algorithm are used for precise positioning to realize subpixel-level dynamic displacement monitoring.
It improves image recognition robustness in complex environments, realizes subpixel dynamic displacement monitoring of structures, is suitable for dynamic displacement monitoring of civil infrastructure, and reduces detection errors.
Smart Images
Figure CN120451214A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of dynamic monitoring of civil engineering structures, and in particular to a method for dynamic displacement monitoring of structures based on a YOLOv5 network model. Background Art
[0002] Dynamic displacement is a key indicator of a structure's dynamic characteristics and safety. Traditional contact-based displacement monitoring methods are cumbersome in terms of wiring design and physical placement, and are time-consuming and labor-intensive, especially for large structures. Digital cameras, as non-contact methods, are relatively low-cost, flexible, and capable of simultaneous measurement with very high spatial resolution. Combined with computer vision algorithms, video-based measurement techniques have been successfully used to measure vibration displacement on various types of structures.
[0003] Due to factors such as environmental complexity, video-based monitoring technology may result in reduced measurement accuracy or even failure during actual measurement. To improve measurement accuracy and efficiency, a fast and accurate target detection algorithm is needed to determine the approximate position of the object to be measured, thereby enabling visual displacement measurement. A target detection algorithm is an algorithm that automatically detects the object to be detected and outputs relevant information after inputting an image. The main indicators for evaluating the performance of a target detection algorithm are accuracy and detection speed. Traditional target detection algorithms, such as scale-invariant feature transforms, Haar-like features, and transformed local binary patterns, are easily affected by factors such as illumination and occlusion, which can reduce detection effectiveness and increase errors in structural vibration displacement detection.
[0004] Solving the above problems has become a top priority. Summary of the Invention
[0005] In view of this, the present invention provides a structural dynamic displacement monitoring method based on the YOLOv5 network model.
[0006] The technical solution is as follows:
[0007] The first aspect of the present application relates to a method for monitoring structural dynamic displacement based on a YOLOv5 network model, which is performed according to the following steps:
[0008] S1. Introduce ring coding markers with affine transformation invariance and establish a ring coding dataset considering different complex environmental backgrounds;
[0009] S2. Select the YOLOv5 model to perform image conversion and network training on the circular coded mark pictures in complex environments, and design the parameters of the YOLOv5 network model;
[0010] S3. Arrange circular coded markers at monitoring points of the structure to be measured and obtain a motion video of the structure;
[0011] S4, using the YOLOv5 network model trained in step S2 to perform preliminary positioning of the circular coding mark of each frame image in the motion video;
[0012] S5, performing sub-pixel precision positioning of the center coordinates of the annular coding mark of each frame image;
[0013] S6. Gather the center coordinates of the annular coding mark of each frame of the video image obtained in step S5 to realize dynamic displacement monitoring of the structure.
[0014] The above structural dynamic displacement monitoring method based on the YOLOv5 network model has the following beneficial effects:
[0015] 1. This method uses the YOLOv5 network model to select the object area instead of the traditional target detection algorithm. Compared with traditional monitoring methods, the YOLOv5 network model shows greater robustness when dealing with image recognition problems in complex environments.
[0016] 2. Based on the target selected by the YOLOv5 network model, this method uses sub-pixel edge detection and ellipse fitting algorithm to solve the center position of the ring code and correct the centrifugal deviation, realizing sub-pixel dynamic displacement monitoring of the structure.
[0017] Therefore, the method of the present invention is more suitable for dynamic displacement monitoring of actual structures, and provides a more applicable new method for dynamic displacement monitoring of civil infrastructure. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 Pictures of ring-coded marks taken in different complex environments;
[0019] Figure 2 Schematic diagram of the YOLOv5 network model;
[0020] Figure 3 This is a graph showing the structure of the YOLOv5 network model training;
[0021] Figure 4 A test result diagram for multiple circular coding marks;
[0022] Figure 5 This is the detection result diagram of the Canny operator within the ring coding mark;
[0023] Figure 6 is the perspective projection transformation diagram of the space ring;
[0024] Figure 7 It is a ring-shaped coding mark pasted on the surface of the cantilever beam;
[0025] Figure 8This is a comparison picture of the tracking effect of the circular coded mark using the method of the present invention and the template matching algorithm;
[0026] Figure 9 It is the time history curve of the x-axis vibration displacement of the cantilever beam end;
[0027] Figure 10 A spectrum obtained by the method of the present invention;
[0028] Figure 11 It is a flowchart of the present invention. DETAILED DESCRIPTION
[0029] The present invention will be further described below with reference to the embodiments and accompanying drawings.
[0030] like Figure 11 As shown in FIG, a structural dynamic displacement monitoring method based on the YOLOv5 network model is performed in the following steps:
[0031] S1. Establish a circular coding dataset.
[0032] Specifically, a circular coded marker with affine transformation invariance is introduced. Pictures with circular coded markers are taken at different distances, angles and lighting environments to establish a circular coded dataset. When constructing the circular coded dataset, the captured pictures are enhanced by data augmentation methods such as conversion, flipping and rotation to improve the richness of the dataset, which can greatly enhance the recognition performance of the network.
[0033] See Figure 1 , Figure 1 (a) is a picture of a circular code mark taken from a distance. Figure 1 (b) is a picture of the ring-coded mark taken at an oblique angle. Figure 1 (c) Image of the ring-coded mark taken in a bright light environment.
[0034] S2. Select the YOLOv5 model to perform image conversion and network training on the circular coded mark pictures in complex environments, and design the parameters of the YOLOv5 network model;
[0035] Specifically, step S2 is performed according to the following steps:
[0036] S21. Select the YOLOv5 network model as the initial positioning model of the ring coding marker, and design the parameters of the YOLOv5 network model.
[0037] In this embodiment, select Figure 2 The YOLOv5 network model shown is used as the initial positioning model for marking, and the parameters of the designed YOLOv5 network model are shown in Table 1.
[0038] Table 1 Specific network parameters of YOLOv5
[0039]
[0040]
[0041] The model consists of two parts: a backbone for feature extraction and a head module for object detection. The backbone consists of a Focus unit, four convolutional layers, four CSP layers, and one SPP layer. The Focus unit splits the 640×640×3 input into 320×320×12 and converts it into a 320×320×64 feature map using 64 3×3 filters, achieving downsampling and feature extraction without losing information. Each convolutional layer contains a pooling layer; each CSP layer contains two convolution kernels, 1×1 and 3×3; and the SPP layer is designed to improve the model's robustness to object deformation. The head consists of four convolutional layers, two upsampling layers, four aggregation units, and one detection unit. Upsampling and aggregation fuse features at different levels; the detection unit is performed at three scales: 20×20, 40×40, and 80×80.
[0042] S22: Import the circular code data set established in step S1 into the YOLOv5 network model for image conversion and training to achieve initial positioning of the circular code marker.
[0043] The batch size is set to 16. The overall training environment is carried out in Pycharm. The experimental results are as follows Figure 3 As shown, Figure 3 (a) is the training loss curve under different training rounds. Figure 3 (b) is the detection accuracy curve under different numbers of training rounds.
[0044] Figure 3 The horizontal axis represents the number of training rounds, and each training round represents that the model is trained once on all training data; the vertical axis represents the loss value of the model on the training set and the accuracy value on the test set after each training round. After 200 rounds of training, the accuracy of the network validation set reached 99.68%. In the early stage of training, the cross entropy loss of the residual network dropped rapidly, and the recognition accuracy continued to rise. After 30 rounds of training, the changes in the loss and accuracy of training and validation tended to slow down and gradually converged. The detection results of the ring-coded mark are shown in Figure 2. Figure 4 As shown in Figure 3, the initial positioning of the ring code based on YOLOv5 shows good accuracy.
[0045] S3. Arrange circular coding markers at monitoring points of the structure to be measured and obtain a motion video of the structure.
[0046] S4. Use the YOLOv5 network model trained in step S2 to initially locate the ring-coded markers of each frame image in the motion video.
[0047] S5. Perform sub-pixel precision positioning on the center coordinates of the annular coding mark of each frame image.
[0048] Specifically, step S5 is performed according to the following steps:
[0049] S51. Within the identified annular code mark frame of each frame image, a step-by-step precision strategy is used to locate the center point of the annular code mark.
[0050] S52. First, use the Canny operator to perform pixel-level edge detection on the annular coding mark in the mark frame.
[0051] In step S52, the image I(x,y) is first smoothed using the Gaussian function G(x,y) to obtain a smoothed image G0(x,y):
[0052] G0(x,y)=G(x,y)*I(x,y) (1)
[0053] In formula (1), * represents convolution, x is the horizontal coordinate of the pixel-level detection image, and y is the vertical coordinate of the pixel-level detection image;
[0054] The Gaussian function G(x,y) in formula (1) selects the Gaussian kernel G(x,y) = [0.075 0.123 0.075; 0.123 0.204 0.123; 0.075 0.123 0.075] with a size of 3×3 and a standard deviation of 1. The horizontal gradient P(x,y) and vertical gradient Q(x,y) of the pixel are obtained as follows:
[0055]
[0056]
[0057] The magnitude g(x,y) and direction θ(x,y) of the gradient are:
[0058] (x,y)=arctan(Q(x,y) / P(x,y))(4)
[0059]
[0060] The point with the local maximum of g(x,y) is taken as the candidate point, and the edge is refined using non-maximum suppression and double threshold segmentation to obtain the pixel-level edge of the ring-coded mark, as shown in Figure 5 As shown, Figure 5 (a) is the original image, Figure 5(b) is the Canny operator Figure 5 (a) Pixel-level edge image after processing of the original image.
[0061] S53 , performing sub-pixel edge detection on the pixel-level edge of the annular code mark obtained in step S52 using a Zernike moment template.
[0062] In step S53, a Zernike moment template is used to perform sub-pixel edge detection on the pixel-level edge of the annular code mark obtained in step S52. The sub-pixel coordinate (x', y') is calculated as follows:
[0063]
[0064] In formula (6), N is the size of the Zernike moment template, x' is the horizontal coordinate of the sub-pixel detection image, y' is the vertical coordinate of the sub-pixel detection image, α and d are parameters, and their calculation formulas are:
[0065]
[0066] In formula (7) and formula (8), Im[A 11 ] and Re[A 11 ] refers to A 11 The imaginary and real parts of j are the imaginary units, A 11 and A 20 Represents Zernike moment; for digital images, the calculation of Zernike moment can be achieved by convolution of template and image; here we construct two templates with size (ie N = 7) of 7 × 7 M 11 and M 20 , A 11 and A 20 Can be connected to M 11 and M 20 The convolution calculations are:
[0067] A 11 = I(x,y)* M 11 (9)
[0068] A 20 =I(x,y)*M 20 (10)
[0069] According to the definition of Zernike moment, the template coefficient M is obtained by dividing the unit circle with a uniform grid. 11 The real part, M 11 The imaginary part and M 20 As shown in Tables 2 to 4 respectively.
[0070] Table 2M11 real number template
[0071]
[0072] Table 3M11 imaginary number template
[0073]
[0074] Table 4M20 template
[0075]
[0076] S54. Implement sub-pixel coordinate positioning of the center point of the annular coding mark by using a least squares ellipse fitting algorithm.
[0077] Specifically, the circular ring-coded marker point appears in an elliptical shape after being imaged by the camera, so the goal of extracting the center of the ring-coded marker point is to obtain the center of the ellipse on the image plane.
[0078] The sub-pixel edge of the detected annular code mark is composed of a set of discrete points. In step S54, the center of the annular code mark is located using a least squares ellipse fitting algorithm to obtain the ellipse center coordinates, which are the center coordinates of the annular code mark. The steps include:
[0079] Combined with the general equation for a plane ellipse:
[0080] x 2 +Axy+By 2 +Cx+Dy+E=0 (11)
[0081] In formula (11), A, B, C, D, and E are all ellipse parameters, and x and y are variables in the Cartesian coordinate system;
[0082] The least squares method is used to determine the values of the ellipse parameters A, B, C, D, and E by minimizing the following equations:
[0083]
[0084] In formula (12), x i and y i is a variable in the Cartesian coordinate system;
[0085] Then the five basic parameters of the ellipse are obtained by the following formula:
[0086]
[0087] In formula (13), x0 is the abscissa of the center of the ellipse, y0 is the ordinate of the center of the ellipse, a is the length of the major axis, b is the length of the minor axis, and θ is the attitude angle.
[0088] S55. Based on the pinhole imaging and perspective projection transformation principles, the center point coordinates of the annular coding mark are corrected for eccentric deviation.
[0089] For concentric rings, according to the pinhole imaging and perspective projection transformation principle, the concentric rings in the plane appear as two ellipses with separated centers in the image captured by the camera, such as Figure 6 As shown, Figure 6 (a) is the ring on the object plane, Figure 6 (b) is the ring of the imaging plane.
[0090] In step S55, the correction equation for performing centrifugal deviation correction on the center coordinates of the annular coding mark in each frame of video image is:
[0091]
[0092] In formula (14), k1 = r1 2 / (r1 2 -r2 2 ), k2=r2 2 / (r1 2 -r2 2 ), r1 and r2 are the radii of the large and small concentric circles respectively; C' is the center point of the concentric circles after correction, and its coordinates are (u C’ ,v C’ ); B1' and B2' are the center points of the two ellipses, and their coordinates are (u B1’ ,v B1’ ) and (u B2’ ,v B2’ ), and then the coordinates of the center point C' after the concentric circle correction (u C’ ,v C’ ).
[0093] S6. Gather the center coordinates of the annular coding mark of each frame of the video image obtained in step S5. The collection of the center coordinates reflects the dynamic displacement of the structure, thereby realizing dynamic displacement monitoring of the structure.
[0094] Reference example:
[0095] This reference example uses a cantilever beam as the research object, verifies the method through random vibration test experiments, and compares it with the traditional template matching algorithm. First, build a visual recognition system and stick several Figure 7 Circular coding mark shown.
[0096] Figure 8 The following figure compares the tracking effect of the ring code using the method proposed by the present invention and the template matching algorithm under single background and complex background respectively. Figure 8 (a) is a tracking picture of the method of the present invention under a single background, Figure 8 (b) is the tracking image of the template matching algorithm under a single background. Figure 8 (c) is a tracking picture of the method of the present invention under a complex background, Figure 8 (d) is the tracking image of the template matching algorithm under complex background.
[0097] Depend on Figure 8 It can be seen that under a single background, both the proposed method and the template matching algorithm can stably track the annular coded marker. However, under the influence of a complex background, the template matching algorithm will often cause the tracking frame to shift or the recognition frame to change significantly, affecting the subsequent positioning of the center of the annular coded marker. This is especially prone to such problems at the ends of the cantilever beam, and less so at the middle and base of the cantilever beam. Compared to the template matching algorithm, the proposed method has excellent accuracy in tracking the annular coded marker under complex backgrounds.
[0098] After the initial positioning of the ring-coded mark is completed, the center of the ring-coded mark is solved by using the sub-pixel edge detection and ellipse fitting algorithm proposed in the present invention to obtain the sub-pixel coordinates of the ring-coded mark in each frame of the image, and the x-axis vibration displacement time history curve of the cantilever beam end is plotted as shown in the figure. Figure 9 As shown. Among them, Figure 9 (a) is a curve graph under a single background. Figure 9 (b) is a curve graph under complex background.
[0099] Depend on Figure 9 (a) It can be seen that under a single background, the amplitude of the vibration measured by the method proposed in this invention and the template matching algorithm will gradually weaken over time, and the motion trajectory of the target feature point between each frame image will become smaller; Figure 9 (b) It can be seen that due to the obvious lag of template matching during the tracking process, a large error occurs.
[0100] In addition, the displacement time history measured by the method proposed in the present invention is subjected to fast Fourier transform. Figure 10 The spectrum signals of the five measurement points are displayed. The first-order frequency peaks obtained at all five measurement points are 1.5Hz, which indicates that the first-order vibration frequency of the cantilever beam is 1.5Hz. The spectrum analysis verifies that the displacements calculated at the measurement points are accurate.
[0101] Therefore, in cantilever beam experiments, by comparing the displacement calculation accuracy of the proposed method with that of traditional target detection algorithms, we can see from the tracking results and displacement time history comparison that the proposed method effectively avoids the problems of template matching algorithms, namely lag and frame loss. It also demonstrates high accuracy and stability when calculating the displacement of objects in complex environments.
[0102] Furthermore, by performing fast Fourier transform on the displacement time-history information of the five points measured on the cantilever beam surface to obtain frequency domain information, the results show that the frequency peaks of the five measuring points are the same, which verifies that the displacement of the measuring points calculated by the method proposed in the present invention is accurate.
[0103] Finally, it should be noted that the above description is only a preferred embodiment of the present invention. Under the guidance of the present invention, ordinary technicians in this field can make various similar expressions without violating the purpose and claims of the present invention. Such changes fall within the scope of protection of the present invention.
Claims
1. A method for monitoring structural dynamic displacement in complex environments based on the YOLOv5 network model, characterized in that: Follow these steps: S1. Introduce ring coding markers with affine transformation invariance and establish a ring coding dataset considering different complex environmental backgrounds; S2. Select the YOLOv5 model to perform image conversion and network training on the circular coded mark pictures in complex environments, and design the parameters of the YOLOv5 network model; S3. Arrange circular coded markers at monitoring points of the structure to be measured and obtain a motion video of the structure; S4, using the YOLOv5 network model trained in step S2 to perform preliminary positioning of the circular coding mark of each frame image in the motion video; S5, performing sub-pixel precision positioning of the center coordinates of the annular coding mark of each frame image; S6. Gather the center coordinates of the annular coding mark of each frame of the video image obtained in step S5 to realize dynamic displacement monitoring of the structure.
2. The structural dynamic displacement monitoring method based on the YOLOv5 network model according to claim 1 is characterized in that: In step S1, pictures with circular coding marks are taken at different distances, angles and lighting environments and a circular coding dataset is established. When constructing the circular coding dataset, data enhancement methods such as conversion, flipping and rotation are used on the captured pictures to improve the richness of the dataset.
3. The structural dynamic displacement monitoring method based on the YOLOv5 network model according to claim 2 is characterized in that: The step S5 is performed according to the following steps: S51, within the annular code mark frame identified in each frame of the image, using a step-by-step precision strategy to locate the center point of the annular code mark; S52, firstly use the Canny operator to perform pixel-level edge detection on the annular coding mark in the mark frame; S53, performing sub-pixel edge detection on the pixel-level edge of the annular code mark obtained in step S52 using a Zernike moment template; S54, achieving sub-pixel coordinate positioning of the center point of the annular coding mark by a least squares ellipse fitting algorithm; S55. Based on the pinhole imaging and perspective projection transformation principles, the center point coordinates of the annular coding mark are corrected for eccentric deviation.
4. The structural dynamic displacement monitoring method based on the YOLOv5 network model according to claim 3 is characterized in that: In step S52, the image I(x,y) is first smoothed using the Gaussian function G(x,y) to obtain a smoothed image G0(x,y): G0(x,y)=G(x,y)*I(x,y) (1) In formula (1), * represents convolution, x is the horizontal coordinate of the pixel-level detection image, and y is the vertical coordinate of the pixel-level detection image; The Gaussian function G(x,y) in formula (1) selects the Gaussian kernel G(x,y) = [0.075 0.123 0.075; 0.123 0.204 0.123; 0.075 0.123 0.075] with a size of 3×3 and a standard deviation of 1. The horizontal gradient P(x,y) and vertical gradient Q(x,y) of the pixel are obtained as follows: The magnitude g(x,y) and direction θ(x,y) of the gradient are: (x,y)=arctan(Q(x,y) / P(x,y))(4) The point with the local maximum of g(x,y) is taken as the candidate point, and the edge is refined using non-maximum suppression and double threshold segmentation to obtain the pixel-level edge of the ring-coded mark.
5. The structural dynamic displacement monitoring method based on the YOLOv5 network model according to claim 4 is characterized in that: In step S53, a Zernike moment template is used to perform sub-pixel edge detection on the pixel-level edge of the annular code mark obtained in step S52, and the sub-pixel coordinate (x', y') is calculated as follows: In formula (6), N is the size of the Zernike moment template, x' is the horizontal coordinate of the sub-pixel detection image, y' is the vertical coordinate of the sub-pixel detection image, α and d are parameters, and their calculation formulas are: In formula (7) and formula (8), Im[A 11 ] and Re[A 11 ] refers to A 11 The imaginary and real parts of j are the imaginary units, A 11 and A 20 Represents Zernike moment; for digital images, the calculation of Zernike moment is realized by convolution of template and image; construct two templates with size of 7×7 M 11 and M 20 , A 11 and A 20 Can be connected to M 11 and M 20 The convolution calculations are: A 11 = I(x,y)* M 11 (9) A 20 =I(x,y)* M 20 (10) 6. The structural dynamic displacement monitoring method based on the YOLOv5 network model according to claim 5 is characterized in that: In step S54, the least squares ellipse fitting algorithm is used to locate the center of the annular code mark to obtain the ellipse center coordinates, which are the center coordinates of the annular code mark, including the following steps: Combined with the general equation for a plane ellipse: x 2 +Axy+By 2 +Cx+Dy+E=0 (11) In formula (11), A, B, C, D, and E are all ellipse parameters, and x and y are variables in the Cartesian coordinate system; The least squares method is used to determine the values of the ellipse parameters A, B, C, D, and E by minimizing the following equations: In formula (12), x i and y i is a variable in the Cartesian coordinate system; Then the five basic parameters of the ellipse are obtained by the following formula: In formula (13), x0 is the abscissa of the center of the ellipse, y0 is the ordinate of the center of the ellipse, a is the length of the major axis, b is the length of the minor axis, and θ is the attitude angle.
7. The structural dynamic displacement monitoring method based on the YOLOv5 network model according to claim 6 is characterized in that: In step S55, the correction equation for performing centrifugal deviation correction on the center coordinates of the annular coding mark in each frame of the video image is: In formula (14), k1 = r1 2 / (r1 2 -r2 2 ), k2=r2 2 / (r1 2 -r2 2 ), r1 and r2 are the radii of the large and small concentric circles respectively; C' is the center point of the concentric circles after correction, and its coordinates are (u C ',v C '); B1' and B2' are the center points of the two ellipses, and their coordinates are (u B1 ',v B1 ') and (u B2 ',v B2 '), and then the coordinates of the center point C' after the concentric circle correction (u C ',v C ').