Full-field deformation measurement method for annular structure

Abstract

The present invention discloses a method for measuring the full-field deformation of a circular structure. The method uses a single panoramic camera as the data acquisition device, and the data processing process is divided into three steps: panoramic image unfolding and calibration, automatic extraction of structure nodes, and sub-pixel calculation of node center coordinates. In step one, a distortion-free planar image covering the full-field range of the circular structure is obtained by decomposing and projecting in multiple directions according to the panoramic camera imaging model and the cube projection method. In step two, a deep learning network with a micro-target detection layer and an attention mechanism added is used to automatically extract the range of each node. In step three, the perceptual hashing method is used for clustering, and a method based on sub-pixel line detection and fitting is used to accurately calculate the node coordinates. The method proposed by the present invention overcomes the drawback of the commonly used full-field deformation measurement method relying on a complex multi-camera measurement system, and has the advantages of being convenient to implement, low cost, and high robustness.

Description

Technical Field

[0001] The present invention belongs to the field of structural health monitoring and measurement, and more specifically, relates to a method for full-field deformation measurement of a circular structure based on a panoramic camera and deep learning-assisted positioning. Background Art

[0002] For important large stadiums, conducting load tests and simulating special construction conditions on their scaled-down models before construction are important means to analyze the bearing capacity of the stadiums and reasonably plan the construction sequence. In load tests and construction condition simulations, it is necessary to accurately measure the deformation within the entire structure to analyze the deformation state and mechanical properties of the structure. Traditional deformation measurement methods such as pasting strain gauges and arranging displacement sensors can accurately measure the structural deformation and are widely used in component experiments (especially concrete components). However, the measurement range of this method is limited, and a large number of sensors need to be arranged in advance, which cannot meet the requirements of full-field deformation measurement of large-scale scaled-down models under large deformation conditions. Vision-based measurement methods are a low-cost and high-precision measurement method that has received extensive attention in recent years. It uses a camera to capture the pixel changes of the structure on the imaging plane, and then obtains the true deformation of the structure according to the corresponding relationship between the imaging plane and the three-dimensional object. Therefore, generally only a simple measurement system composed of a few cameras and a computer with embedded algorithms is required to measure the deformation of a large-scale structure. In this paper, aiming at the problem of full-field deformation measurement of a large-scale scaled-down model of a circular stadium, a low-cost and rapid measurement method for circular structure deformation based on a panoramic camera and machine vision algorithm is proposed.

[0003] With the rapid improvement of camera performance and the gradual improvement of computer vision-related algorithms in recent years, vision-based measurement methods have developed various measurement methods for different measurement objects. These methods have been applied in deformation measurements from small-scale microelectronic detection to large-scale kilometer-level bridges. Generally, they can be divided into target-based measurement methods and target-free measurement methods. Among them, target-based measurement methods generally require pre-pasting specially designed targets on the structure, such as coded points of various shapes, infrared targets, and active light source targets. After the camera captures an image containing the target, the position of the target in the image is automatically identified through an algorithm for identifying coded points or a lens with a filter for special wavelength light, so as to calculate the displacement of the target. Target-free measurement methods generally set feature detectors according to the geometric features or texture features of the measured object itself, and then calculate the change of the detected features according to the matching relationship between the front and back images, so as to obtain the pixel displacement. Such methods include methods of extracting the target edge by using edge detection and line detection to calculate the displacement of the target in the front and back two frames of images, and methods based on digital image correlation. Among them, digital image correlation has become one of the most commonly used vision measurement methods due to its good stability and accuracy.

[0004] For the full-field deformation measurement of a scaled model of a circular stadium, there are two problems to be solved urgently in applying the vision measurement method. 1. The field of view of a single camera is limited, and it is impossible to simultaneously capture all the nodes to be measured on a large-sized circular scaled model; 2. The scaled model consists of slender rods and cables, with a complex structure and few textures at the positions of the nodes to be measured. It is easy to lose the target or generate false matches when using the existing vision measurement method in a complex test environment.

[0005] Regarding the measurement problem of large-sized annular structures, a measurement system composed of multiple cameras can obtain the large-range full-field deformation of the structure. Some scholars have proposed two spatial data stitching strategies for multi-camera digital image correlation systems and discussed the feasibility of applying the two strategies to the deformation measurement of large-sized industrial installations. Some scholars have proposed a multi-camera digital image correlation method and system for measuring large engineering objects with distributed and non-overlapping regions of interest and applied it to the three-dimensional deformation measurement of a structure with a span of 18 m. Some scholars have established a four-camera vision system and studied local calibration and global calibration methods for multi-camera correlation and a point cloud correction method for optimizing point cloud stitching, providing a basis for the visual application of multi-camera systems. Some scholars have studied the performance of object-based and calibration-based multi-camera stereo DIC methods applied to the deformation measurement of slender members, built a measurement system consisting of 9 cameras, and applied it to the deformation measurement of a 900-mm-long concrete beam. The above studies have proven that multi-camera measurement systems can take into account the advantages of large field of view and high precision when measuring the deformation of large-sized structures. However, multi-camera systems have problems such as complex composition, high cost, and the need to complete complex calibration before measurement. The optimal measurement field of view size of a single camera is fixed. To increase the measurement field of view, the number of cameras must be increased exponentially. For a reduced-scale model of an annular stadium with a diameter of more than ten meters, a multi-camera measurement system with overlapping fields of view composed of more than 10 cameras is required. This method is undoubtedly complex and costly. A panoramic camera is a camera that can simultaneously obtain the surrounding 360° range of the environment. With the increasing progress of imaging technology, low-cost and highly integrated panoramic cameras have been widely applied to the fields of photography and VR. In structural health detection, some scholars have also proposed applying panoramic cameras or 360° panoramic images to overcome the limitations of traditional cameras. Some scholars have proposed a method for evaluating the damage of post-disaster buildings based on 360° panoramic images of street view services, and automatically identify and extract buildings in the panoramic images by using region-based convolutional neural networks. Some scholars have designed a structural surface damage detection network based on panoramic images, overcoming the problem that traditional deep learning networks cannot process panoramic images with high resolution and high distortion. Panoramic cameras have a field of view size and high resolution far exceeding those of ordinary cameras. However, so far, there are very few studies on the application of panoramic cameras in structural health detection, and the measurement of the full-field deformation of structures using panoramic cameras has not been studied yet. Summary of the Invention

[0006] Based on the above situation, the present invention provides a full-field deformation measurement method for a circular structure based on a panoramic camera and deep learning-assisted positioning. The core of this method includes two parts: (1) For the problem of full-field deformation measurement of a large-scale circular reduced-scale model, it is proposed to use a single panoramic camera to obtain a 360° full-field image. According to the imaging principle of the panoramic camera, the significantly distorted panoramic image is projected onto six directions of front, back, left, right, up, and down using a hexahedron model, and then the undistorted images for measurement are obtained through calibration in each direction; (2) For the problem of measuring node deformation, it is proposed to use the improved YOLO v5 model based on the attention mechanism to solve the problem of range detection of small nodes. Then, the perceptual hashing algorithm is used to cluster images for the detected large number of node ranges. Finally, the center point coordinates of the nodes are located by the method of sub-pixel straight line detection and fitting, realizing the rapid measurement of the full-field node displacement.

[0007] To solve the above technical problems, the technical solution of the present invention is as follows:

[0008] In the first aspect of the present invention, a full-field deformation measurement method for a circular structure is provided, including the following steps: S1. Unfolding and calibration of panoramic images; S2. Automatic extraction of measurement position ranges; S3. Calculation of actual displacements at each range position.

[0009] The specific steps of S1 are as follows: According to the panoramic camera imaging model and the method of decomposing and projecting in multiple directions, an undistorted planar image covering the full-field range of the circular structure is obtained.

[0010] In S1, for the projection plane tangent to the panoramic sphere, the field of view angle FOV = 90° is set, and the size of the projected planar graph is (w, h). First, the normalized focal length is estimated:

[0011]

[0012] For the point (u, v) on the projection plane, the conversion to the spherical coordinate system is

[0013]

[0014] The conversion to polar coordinates (θ, r)

[0015]

[0016] Then the coordinates (U, V) of the panoramic image are

[0017]

[0018] The specific steps of S2 are as follows: for the pictures projected and unfolded in six directions of the panoramic image, independent calibration methods are used for calibration respectively; multiple images are taken in the front, back, left, and right directions of the panoramic camera using a checkerboard calibration board, and after the above-mentioned projection segmentation of the obtained panoramic image, the Zhang's calibration method is used for calibration of the images in the four directions respectively.

[0019] The specific steps of S3 are as follows: the perceptual hashing method is used for clustering and a method based on sub-pixel line detection and fitting is used to accurately calculate the node coordinates.

[0020] The purpose of the unfolding and calibration of the panoramic image is to convert the panoramic image from the spherical projection unfolded image recorded by the camera into the orthographic projection images of the front, back, left, right, up, and down six faces according to the cylinder projection and cube projection methods. Based on this, it can be considered that the panoramic camera is simplified to a pinhole model camera in the six face directions. Therefore, the Zhang's calibration method is used for calibration respectively, and the calibrated results are used for distortion correction of the images of the six faces. After the above operations are completed, when measuring the displacement with the panoramic camera, the homography matrix can be used for the calibrated image to obtain the proportional relationship between the three-dimensional object to be measured and the projection image of any one face. Therefore, the displacement of the object with a real scale can be calculated using the image.

[0021] The automatic extraction of the measurement position range is to automatically find the key point positions to be measured from the images obtained in the previous step. Taking the annular cable net structure in this article as an example, it contains dozens of connection nodes. Automatically extracting the node positions can improve the automation degree of the whole set of methods. However, the node positions of the cable net structure are small and occupy few pixels in the whole image. Therefore, the key problem in this part is how to achieve high-precision recognition of small node targets. A YOLO v5 model fused with an attention mechanism is proposed to identify small node targets in the image. By adding a Transformer prediction head on the basis of the YOLO v5 model to increase the sensitivity of the model to tiny objects, each node in the cable net can be accurately identified and framed.

[0022] After determining the calculation range of the node positions, the last step is to calculate the displacement of each ROI respectively. The key is to accurately locate the coordinates of the node center from the ROI. Since each node is the intersection point of the structural members, the method of line detection is used to detect the members and fit the intersection points of the lines, and the coordinates of the intersection points can be obtained. Combining with the image scale parameters calculated before, the displacement of each node with a real scale can be obtained.

[0023] According to another aspect of the present invention, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in the full-field deformation measurement method of the annular structure of the present invention are implemented.

[0024] According to another aspect of the present invention, there is provided a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps in the ring structure full-field deformation measurement method of the present invention are implemented.

[0025] Compared with the prior art, the present invention has at least the following beneficial effects:

[0026] The invented full-field deformation measurement method for the structure is a low-cost and non-contact measurement method. Traditional sensor-based deformation measurement methods, such as pasting strain gauges and arranging displacement sensors, can accurately measure the structural deformation. However, the measurement range of this method is limited, and a large number of sensors need to be arranged in advance, which cannot meet the requirements of full-field deformation measurement of large-scale reduced models under large deformation conditions. Existing vision measurement methods for collecting full-field images of structures rely on a complex measurement system composed of multiple cameras and have the problem of difficultly accurately measuring the displacement of small targets with low texture.

[0027] Compared with the multi-camera measurement system, the proposed panoramic camera solution has the advantages of low complexity and low cost. The invented node displacement calculation method improves the accuracy of the convolutional neural network in small target recognition and overcomes the disadvantage that the digital image correlation method is prone to losing low-texture targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description only relate to some embodiments of the present invention and do not limit the present invention.

[0029] Figure 1 is a schematic diagram of the inventive method;

[0030] Figure 2 is the imaging principle of the panoramic camera of the method of the present invention;

[0031] Figure 3 is the method of projecting and unfolding the panoramic image of the method of the present invention in six directions;

[0032] Figure 4 is the structural diagram of the improved node detection network in the method of the present invention;

[0033] Figure 5 is the flowchart of the calculation method of node coordinates in the method of the present invention;

[0034] Figure 6 is the result diagram of the training of the node detection network in the established dataset;

[0035] Figure 7It is a comparison chart of the test results of the method of the present invention in the embodiment and the test results of the total station. Specific embodiments

[0036] The present invention will be further described below through specific embodiments, but the protection scope of the present invention should not be limited thereby.

[0037] The test example is a scaled model of a stadium, and its structure is composed of a fixed main body part on the outer layer and a movable dome on the top. The movable dome is connected to the fixed main body through a flexible cable structure, and the up and down movement of the dome is controlled by 6 groups of brackets with pulleys. The length of the outer main body part is 18m, the width is 16m, and the height is 4.5m. The inner circle length of the dome at the top of the inner layer is 8.6m, and the width is 5.8m.

[0038] The purpose of the test is to simulate the process of the middle dome descending during the use of the stadium. The experimental condition is to gradually lower the dome in two steps, with each stage descending about 10 cm. Since the descending process is to manually control the six pulley groups to relax the traction chain, it is necessary to measure the accurate displacement of each node of the middle dome. During the descending process, the displacements of 40 nodes at the upper, middle and bottom of the middle dome structure are measured. During the test, a single panoramic camera measurement system is used to measure the displacement. For comparison, a total station is used to measure the coordinates of the nodes simultaneously to verify the results of the panoramic measurement.

[0039] The specific steps of the method of the present invention are as follows:

[0040] First, panoramic images of the model during the test are collected at equal time intervals for analysis. The preprocessing of the panoramic images includes the unfolding and distortion removal of the panoramic images. After completing this step, projection images in the front, back, left and right directions are obtained. Then, the trained node detection model is used to detect the node range, and according to the detection results, the nodes are cut out from the original image to obtain a large number of node images. For the large number of node images obtained after cutting, first calculate the hash fingerprint images for all nodes and calculate the Hamming distance for clustering. Then, the sub-pixel line detection algorithm is used to detect lines, the skeleton line extraction and midline calculation fitting method is used to obtain the skeleton line at the node, and finally, the intersection point closest to the point in the image is selected as the midpoint of the node. The above processing can obtain the pixel coordinates of each node, and by comparing the image results at different times, the pixel displacement of each node can be obtained.

[0041] The results measured by the proposed method are compared with the results measured point by point by the total station to analyze the accuracy of the proposed method. In each test stage, a panoramic image is selected, the displacements of each measurement node are calculated, and the results are compared with those measured by the total station. Since the coordinates of the nodes are measured by the total station four times in the stable state before measurement, the initial state before the test starts, the first descent, and the second descent during the test, four panoramic photos in four corresponding states are selected. By subtracting the results of each measurement from the results of the first measurement, three displacement curves can be obtained: the displacement of the nodes between the initial state and the stable state in Step 1, the displacement of the nodes between the first descent and the initial state in Step 2, and the displacement of the nodes between the second descent and the initial state in Step 3, as Figure 7 shown. The displacement measurement results of the nodes show that the results of the proposed method in the three stages are in good agreement with those of the total station, and can correctly reflect the vertical displacement of the nodes. The results of the error analysis show that the average error between the proposed method and the total station data is 3.7 mm, the maximum error is 8 mm, the displacement of the node when the maximum error occurs is 272 mm, and the measurement error is 2.9%, meeting the requirements of industrial measurement.

[0042] Example 2:

[0043] The computer-readable storage medium of this embodiment stores a computer program, which when executed by a processor implements the steps in the full-field deformation measurement method of the annular structure in Example 1.

[0044] The computer-readable storage medium of this embodiment can be the internal storage unit of the terminal, such as the hard disk or memory of the terminal; the computer-readable storage medium of this embodiment can also be the external storage device of the terminal, such as the plug-in hard disk, smart memory card, secure digital card, flash card, etc. equipped on the terminal; further, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the terminal.

[0045] The computer-readable storage medium of this embodiment is used to store the computer program and other programs and data required by the terminal, and the computer-readable storage medium can also be used to temporarily store the data that has been output or will be output.

[0046] Example 3:

[0047] The computer device of this embodiment includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in the full-field deformation measurement method of the annular structure in Example 1.

[0048] In this embodiment, the processor may be a central processing unit, or may also be other general-purpose processors, digital signal processors, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.; the memory may include read-only memory and random access memory, and provide instructions and data to the processor. A part of the memory may also include non-volatile random access memory. For example, the memory may also store information about the device type.

[0049] Those skilled in the art should understand that the content disclosed in the embodiments can be provided as a method, a system, or a computer program product. Therefore, the present solution can be implemented in the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present solution can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0050] This solution is described with reference to the flowcharts and / or block diagrams of the methods and computer program products according to the embodiments of this solution. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions; these computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for the functions specified in one block or multiple blocks.

[0051] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for the functions specified in one block or multiple blocks.

[0052] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for the functions specified in one block or multiple blocks.

[0053] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above various methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0054] The examples described in the present invention are only descriptions of the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various deformations and improvements made by those skilled in the art to the technical solutions of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for measuring the full-field deformation of an annular structure, characterized in that, It includes the following steps: S1. Unfolding and calibration of the panoramic image; S2. Automatic extraction of the measurement position range; S3. Calculation of the actual displacement at each range position; The specific steps of S1 are as follows: According to the panoramic camera imaging model and the cube projection method, a method of decomposing and projecting in multiple directions is used to obtain a de-distorted planar image covering the full range of the annular structure; The specific steps of S2 are as follows: For the images projected and unfolded in six directions of the panoramic image, independent calibration methods are used for calibration respectively; Multiple images are taken using a checkerboard calibration board in the front, back, left, and right directions of the panoramic camera respectively. After the obtained panoramic image is segmented by the above projection, the Zhang's calibration method is used for calibration of the images in the four directions respectively; The specific steps of S3 are as follows: The perceptual hashing method is used for clustering and the method based on sub-pixel line detection and fitting is used to accurately calculate the node coordinates.

2. The method according to claim 1, wherein In S1, for the projection plane tangent to the panoramic sphere, the field of view angle FOV = 90°, and the size of the projected planar graph is (w, h). First, the normalized focal length is estimated: For the point (u, v) on the projection plane, it is converted to the spherical coordinate system as Converted to the polar coordinate (θ, r) Then the coordinates (U, V) of the panoramic image are 3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the steps in the full-field deformation measurement method of the annular structure described in any one of claims 1 to 2.

4. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the full-field deformation measurement method of the annular structure described in any one of claims 1 to 2.

Images