Lightweight multi-target visual perception method and system for high-formwork safety monitoring

Abstract

The invention discloses a lightweight multi-target visual perception method and system for high-formwork safety monitoring, and belongs to the technical field of structure safety monitoring, and the method comprises the steps: continuously collecting shot images of different positions of a to-be-detected high-formwork; for each position of the high-support formwork, infrared target tracking is carried out on continuously shot images of the high-support formwork through a Camshift-Gaussian-Centroid lightweight multi-target detection algorithm, the displacement change of the mass center of each infrared target is obtained, and monitoring of the dynamic change of the high-support formwork is achieved; wherein when the infrared target is tracked, if the infrared target is shielded, the mass center of the infrared target is obtained according to the shielding degree of the infrared target through a CamShift algorithm and a target robust extraction method of area consistency constraint and shape consistency constraint; and if the infrared target is not shielded, the mass center of the target is obtained through a CamShift algorithm and a two-dimensional Gaussian fitting mode. The problem of light path shielding of optical measurement can be solved, and the displacement of the high formwork can be monitored in real time and at high precision.

Description

Technical Field

[0001] The present invention belongs to the technical field of structural safety monitoring, and particularly relates to a lightweight multi-objective visual perception method and system for high formwork support safety monitoring. Background Art

[0002] With the continuous increase in the number of urban viaducts and long-span structures, the demand for high formwork support systems has also increased accordingly. The formwork support system is used in reinforced concrete buildings and is used during the installation, maintenance, access or inspection of the building system. It usually consists of steel pipes, connectors and plates. When the formwork support system has a height exceeding 8m, or a span exceeding 18m, and the total construction load is greater than 10kN / m 2 It is also called a high formwork support system, abbreviated as high formwork. Investigations into the collapse of formwork support systems have shown that 74% of the support system collapses occur during concrete pouring operations, usually resulting in the instantaneous collapse and overturning of the reinforced concrete structure during the construction stage, with catastrophic consequences.

[0003] The failure modes of the formwork support system often include buckling failure of vertical poles, bending failure of horizontal bars, and initial defect failure. In a tall formwork support system, due to the greater clear height of the vertical poles, buckling failure of the vertical poles is the main form. On the other hand, in order to avoid the failure of high formwork, monitoring it is an important means to solve safety accidents. Currently, in the prior art, Moon et al. used a sensor network to monitor the deflection of the temporary support structure and compare it with the allowable limit; Yuan et al. used a cyber-physical system (CPS) to monitor the load information, displacement of the support plate, and the inclination of the vertical pole in three directions, and then developed a real-time monitoring and alarm system for temporary structures based on CPS, constructing a virtual support structure for remote real-time monitoring through multiple sensor data. The above methods have achieved the monitoring and early warning of high formwork, but they require a large number of sensors to be deployed, at a huge cost. At the same time, laying lines on site easily brings potential safety hazards.

[0004] Optical imaging and computer vision - assisted technologies have developed rapidly in the field of civil engineering and have great potential in the dynamic measurement of displacements of civil infrastructure. Some scholars have used optical measurement methods to monitor the displacements of high - formwork supports. For example, Lu Songyao et al. periodically monitored the displacements of the supporting formwork during the pouring process; Jung et al. applied video analysis technology to the monitoring of concrete scaffolding and used the Hidden Markov Model (HMM) to understand possible defects in the supports and infer the reasons. The limitation of this method is that it cannot quantify the deformation of the supports and can only monitor local areas; Feng et al. attempted to measure the deformation of the shape of structural materials based on a stereo image - matching algorithm to promote the monitoring of the integrity and safety of the supporting structure. Later, various image - processing methods were compared, and the feasibility of using group - based intelligent digital image correlation (DIC) technology for the monitoring of high - formwork support systems was discussed, but no relevant research has been carried out; Xu et al. measured the displacements at the nodes of small - scale temporary structures in the laboratory based on computer vision to determine the safe stage of the structure; Jung et al. proposed applying image - processing technology to automatically detect small changes in the supporting structure and verified the great potential of the proposed method through small - scale experiments; Luo et al. established a pipe - axis model of a scaled - down support system based on laser point clouds and realized 3D deformation monitoring of the support system by comparing the pipe - axis models at different times. Since the light path will be blocked by pipes and plates, only the displacements at the edges of the supporting formwork system can be observed, which makes it difficult to prevent failures in the middle of the support system.

[0005] In summary, the method based on optical measurement has great potential for dynamic displacement monitoring of high - formwork vertical poles. However, there is no effective solution to the problem of target occlusion in the complex on - site environment, which hinders the application of optical imaging and computer vision measurement methods in the safety monitoring of high - formwork. Summary of the Invention

[0006] In view of the deficiencies in the prior art, the present invention provides a lightweight multi - target visual perception method and system for high - formwork safety monitoring, which can overcome the problem of light - path occlusion in optical measurement and monitor the displacement of high - formwork in real - time with high precision.

[0007] The present invention provides the following technical solutions:

[0008] In the first aspect, a lightweight multi - target visual perception method for high - formwork safety monitoring is provided, including:

[0009] Continuously collect captured images of different positions of the high - formwork to be detected at a preset sampling frequency; a number of infrared targets are arranged on the high - formwork to be monitored, and there are four light points on each infrared target;

[0010] For each position of the high formwork, the infrared target is tracked in the continuously captured images through the Camshift-Gaussian-Centroid lightweight multi-target detection algorithm, and the displacement change of the centroid of each infrared target is obtained to realize the monitoring of the dynamic changes of the high formwork. Among them, when tracking the infrared target, if the infrared target is occluded, the centroid of the infrared target is obtained according to the occlusion degree of the infrared target through the CamShift algorithm and the robust target extraction method with area consistency constraint and shape consistency constraint. If the infrared target is not occluded, the centroid of the target is obtained through the CamShift algorithm and two-dimensional Gaussian fitting.

[0011] Optionally, for each position of the high formwork, the infrared target is tracked in the continuously captured images through the Camshift-Gaussian-Centroid lightweight multi-target detection algorithm, and the displacement change of the centroid of each infrared target is obtained to realize the monitoring of the dynamic changes of the high formwork. The specific process is as follows:

[0012] For the current frame image of the continuously captured images, obtain the initial tracking window for target tracking of each target in it.

[0013] According to the initial tracking window, use the pyramid iteration method to obtain the best tracking window of the current frame image and the centroid of the best tracking window.

[0014] In the best tracking window of the current frame image, track the light spot of the infrared target in the current tracking window through the CamShift algorithm to obtain the centroid of the infrared target. Specifically: if the infrared target is not occluded, four light spots are tracked, and the centroid of the infrared target is obtained according to the two-dimensional Gaussian fitting method. If one light spot of the infrared target is occluded, three light spots are tracked, and the centroid of the infrared target is obtained according to the area consistency constraint method. If two light spots of the infrared target are occluded, two light spots are tracked, calculate the average value of the relative displacements of the two light spots between the reference frame image and the current frame image, and correct the centroid of the reference frame image in combination with the shape consistency constraint to obtain the centroid of the infrared target in the current frame. If three light spots of the infrared target are occluded, one light spot is tracked, calculate the relative displacement of one light spot between the reference frame image and the current frame image, and correct the centroid of the reference frame image in combination with the shape consistency constraint to obtain the centroid of the infrared target in the current frame. If all four light spots of the infrared target are occluded, zero light spots are tracked, and the centroid of the infrared target in the current frame is estimated through the extended Kalman filter method according to the centroid positions of the infrared targets in a set number of historical frame images.

[0015] Obtain the best tracking window and the centroid of the tracking window of the next frame image, and track the infrared target in the next frame image until the centroid of the infrared target in each frame image is obtained.

[0016] Calculate the displacement change of the centroids of the infrared targets in all frame images to achieve the monitoring of the dynamic changes of high-order modes.

[0017] Optionally, in the initial tracking window for target tracking of each target in the current frame image of the continuously captured images,

[0018] If the current frame image is the first frame image of the continuously captured images, perform binary processing on the first frame image through the Pyramid-Histogram-Otsu multi-threshold segmentation method to segment out all the light points in the first frame image; use the ROI segmentation method based on 4-connected-region to distinguish the connected regions of each light point, and use the minimum circumscribed rectangle of the connected region of each light point as the initial tracking window of the first frame image;

[0019] If the current frame image is not the first frame image of the continuously captured images, the initial tracking window of the current frame image is the best tracking window of the previous frame image.

[0020] Optionally, the specific process of performing binary processing on the first frame image through the Pyramid-Histogram-Otsu multi-threshold segmentation method to segment out all the light points in the first frame image is as follows:

[0021] Perform pyramid processing on the first frame image to generate image representations at different scales;

[0022] At each scale, calculate the grayscale histogram of the image representation, and reduce the dimension of the grayscale histogram according to the scaling factor to construct a histogram pyramid;

[0023] On any layer a of the histogram pyramid, use the Otsu method to initially determine two thresholds corresponding to layer a to distinguish three different regions of the image; the three different regions are the background, the halo of the infrared target light points, and the light points of the infrared target;

[0024] Iteratively and finely adjust the two initially determined thresholds to obtain two optimal thresholds of the histogram pyramid;

[0025] According to the two optimal thresholds, segment the first frame image to obtain the background, the halo of the infrared target light points, and the light points of the infrared target.

[0026] Optionally, the specific process of iteratively and finely adjusting the two initially determined thresholds to obtain two optimal thresholds of the histogram pyramid is as follows:

[0027] Step I: Multiply the two initially determined thresholds by the reciprocal of the scaling factor respectively to obtain two roughly optimal thresholds corresponding to the layer i with a larger size in the upper layer of pyramid layer a;

[0028] Step II: For each rough optimal threshold corresponding to the current layer i, search within the set pixel range around it, calculate the between-class variance corresponding to each pixel value, and select the pixel value with the largest between-class variance as the exact optimal threshold for this rough optimal threshold of the current layer i;

[0029] Step III: Take the exact optimal threshold of the current layer i as the rough optimal threshold of the previous layer i - 1, and repeat Step II to obtain the exact optimal threshold of the previous layer i - 1;

[0030] Step IV: Let i = i - 1, repeat the operations of Step II and Step III until the optimal threshold of the layer with the largest size in the histogram pyramid is obtained, and take the optimal threshold of the layer with the largest size in the histogram pyramid as the optimal threshold of the histogram pyramid.

[0031] Optionally, the process of obtaining the optimal tracking window and the centroid of the tracking window of the current frame image according to the initial tracking window by using the pyramid iteration method is as follows:

[0032] Take the initial tracking window as the candidate tracking window, calculate the centroid position of the candidate tracking window, set (x, y) as the pixel point of the candidate tracking window, I(x, y) as the pixel value of the pixel point (x, y), and the 0th moment M 00 and the 1st moment M 10 , M 01 are:

[0033]

[0034] The centroid position (x c , y c ) of the candidate tracking window is:

[0035]

[0036] Move the center of the candidate tracking window to the centroid position;

[0037] Judge whether it converges. If it converges, the current candidate tracking window is the optimal tracking window, and the centroid position of the current candidate tracking window is the centroid of the optimal tracking window; if not, reduce the size of the current candidate tracking window according to the set requirements, and recalculate the centroid within the candidate tracking window after the size reduction until convergence.

[0038] Optionally, the area consistency constraint is:

[0039] A i ≥ a * A 0

[0040] where A iThe area enclosed by the light points tracked within the optimal tracking window of the current frame image, A 0 is the area enclosed by the four light points in the reference image, and a is the area coefficient;

[0041] The shape consistency constraint is:

[0042]

[0043] where (a 0 , b 0 , α 0 ) is the original shape of the reference image, and (a i , b i , α i ) is the shape enclosed by the light points tracked within the optimal tracking window of the current frame image, and ε a , ε b and ε α are three set thresholds.

[0044] Optionally, the specific process of calculating the displacement change of the centroid of the infrared target in all frame images is as follows:

[0045] Calculate the pixel displacement d of the centroid of the infrared target between the current frame image and the previous frame image y ;

[0046]

[0047] where y 0 is the value of the centroid of the infrared target on the y-axis in the previous frame image, y i is the value of the centroid of the infrared target on the y-axis in the current frame image, y i ' is the value of the center point of the minimum circumscribed rectangle of the centroid of the infrared target on the y-axis in the current frame image, A 0 is the area enclosed by the four light points in the reference image, and A i is the area enclosed by the light points tracked within the optimal tracking window of the current frame image;

[0048] Convert the pixel displacement into the displacement v of the high-order mode infrared target in the world coordinate system;

[0049]

[0050] where D is the size of the infrared target in the world coordinate system, d is the pixel size of the infrared target in the camera coordinate system, and l 0 is the pixel size of the acquisition camera;

[0051] Remove the target vibration displacement and installation angle displacement in the displacement v of the high-order mode infrared target in the world coordinate system to obtain the true displacement v y ;

[0052] v y = v - l * (cosβ - cosα)

[0053] Wherein, l is the height of the installation bracket of the infrared target, β is the installation angle deviation between the infrared target and the y-axis, and α is the rotation angle between the infrared target and the y-axis caused by vibration.

[0054] Optionally, it further includes: judging whether there is an abnormality in the high formwork according to the displacement change of the centroid of all infrared targets, and when there is an abnormality, sending corresponding warning information according to the abnormal situation;

[0055] The abnormal types of the high formwork include: F1, formwork settlement caused by ground settlement; F2, instability caused by excessive pressure on the jack; F3, formwork settlement caused by local fastener failure; F4, structural overturning caused by overall displacement;

[0056] If the displacement of the mass point of one infrared target reaches 80% of the set value, abnormal situations of abnormal types F2 and F3 may occur, and a first-level warning is issued;

[0057] If the displacements of the mass points of multiple unrelated infrared targets all reach 80% of the set value, abnormal situations of abnormal types F1 and F4 may occur, and a second-level warning is issued;

[0058] If the displacements of the mass points of multiple related infrared targets all reach 80% of the limit value, abnormal situations of abnormal types F2 and F3 may occur, and a second-level warning is issued;

[0059] If the displacement of the mass point of a certain infrared target reaches the set value, abnormal situations of abnormal types F2 and F3 may occur, and a second-level danger warning is issued.

[0060] In a second aspect, a lightweight multi-target visual perception system for high formwork safety monitoring is provided, including:

[0061] An acquisition module, configured to continuously acquire captured images of different positions of the high formwork to be detected according to a preset sampling frequency; a plurality of infrared targets are arranged on the high formwork to be monitored, and there are four light spots on each infrared target;

[0062] The management and control module is used to track the infrared target in the continuously shot images of each position of the high-support formwork through the Camshift-Gaussian-Centroid lightweight multi-target detection algorithm, obtain the displacement change of the center of mass of each infrared target, and realize the monitoring of the dynamic change of the high-support formwork; wherein, when tracking the infrared target, if the infrared target is blocked, the center of mass of the infrared target is obtained according to the degree of blockage of the infrared target through the CamShift algorithm and the target robust extraction method with area consistency constraint and shape consistency constraint; if the infrared target is not blocked, the center of mass of the target is obtained through the CamShift algorithm and two-dimensional Gaussian fitting.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] (1) In order to solve the optical path occlusion problem caused by the complex environment such as the flow of workers and the parking of construction machinery in the construction scene, the targets are divided into no occlusion, 1-3 levels of occlusion and complete occlusion according to the degree of occlusion. A robust target extraction method based on Photsu's multi-threshold segmentation and area and shape constraints is proposed. An adaptive centroid solution method is used for each occlusion type to solve the occlusion problem of the target during the monitoring process.

[0065] (2) The Camshift-Gaussian-Centroid lightweight multi-target detection algorithm is adopted, which has excellent tracking effect when the shape and size of the tracking target changes or the position changes significantly. At the same time, this method greatly reduces the cost of single-point monitoring and solves the problem that traditional displacement sensors require the deployment of a large number of sensors and the slow deployment of measurement points. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a framework diagram of high-support formwork displacement monitoring of the present invention;

[0067] Figure 2 It is a schematic diagram of the Otsu multi-threshold segmentation method based on histogram pyramid of the present invention;

[0068] Figure 3 It is a schematic diagram of accurate positioning and occlusion classification based on area and shape consistency constraints of the present invention;

[0069] Figure 4 It is a model diagram of the infrared target displacement of the present invention;

[0070] Figure 5 It is a flow chart of target tracking by infrared target of the present invention;

[0071] Figure 6 is a structural block diagram of the online camera monitoring and early warning system of the present invention;

[0072] Figure 7 It is a diagram of the monitoring range and positional relationship of the on-line camera in Embodiment 2 of the present invention. Specific Embodiments

[0073] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention. It should be noted that the terms "including" in the description and claims of the present invention and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0074] Embodiment 1

[0075] As Figure 1 shown, a lightweight multi-target visual perception method for high-bay formwork safety monitoring is provided, including the following steps:

[0076] Step S1: Continuously collect captured images of different positions of the high-bay formwork to be detected at a preset sampling frequency; a number of infrared targets are provided on the high-bay formwork to be monitored, and each infrared target has four light points.

[0077] Due to the complex background at the construction site, sufficient sunlight during the day and insufficient sunlight at night, infrared targets are used as observation targets. Usually, there are a number of infrared targets on the high-bay formwork. Optionally, there are 9 infrared targets, and 9 infrared targets can be monitored by three on-line cameras, that is, each camera monitors three infrared targets.

[0078] The captured images of different positions are the captured images of each camera, and the sampling frequency of each camera can be determined according to expert experience; the four light points are distributed in a rectangle to form an infrared target.

[0079] Step S2: As Figure 5 shown, for each position of the high-bay formwork, the Camshift-Gaussian-Centroid lightweight multi-target detection algorithm is used to track the infrared targets in its continuously captured images, obtain the displacement changes of the centroids of each infrared target, and realize the monitoring of the dynamic changes of the high-bay formwork; among them, when tracking the infrared targets, if the infrared target is occluded, the centroid of the infrared target is obtained according to the occlusion degree of the infrared target by the CamShift algorithm and the robust target extraction method with area consistency constraint and shape consistency constraint; if the infrared target is not occluded, the centroid of the target is obtained by the CamShift algorithm and two-dimensional Gaussian fitting.

[0080] In this embodiment, step S2 includes the following sub-steps:

[0081] Step S21: For the current frame image of the continuously captured images, obtain an initial tracking window for target tracking of each target thereon.

[0082] Specifically, step S21 includes the following sub-steps:

[0083] Step S211: If the current frame image is the first frame image of the continuously captured images, perform binarization processing on the first frame image by using the Pyramid-Histogram-Otsu multi-threshold segmentation method to segment all the light points in the first frame image; use the ROI segmentation method based on 4-connected-region to distinguish the connected regions of each light point, and use the minimum circumscribed rectangle of the connected region of each light point as the initial tracking window of the first frame image.

[0084] As Figure 2 shown, in this embodiment, perform binarization processing on the first frame image by using the Pyramid-Histogram-Otsu multi-threshold segmentation method to segment all the light points in the first frame image. The specific process is as follows:

[0085] Perform pyramid processing on the first frame image to generate image representations of different scales; at each scale, calculate the grayscale histogram of the image representation, and reduce the dimension of the grayscale histogram according to the scaling factor to construct a histogram pyramid; at any layer a of the histogram pyramid, use the Otsu method to preliminarily determine two thresholds corresponding to layer a to distinguish three different regions of the image; the three different regions are the background, the halo of the infrared target light point, and the light point of the infrared target; perform iterative fine-tuning on the two preliminarily determined thresholds to obtain two optimal thresholds of the histogram pyramid; according to the two optimal thresholds, segment the first frame image to obtain the background, the halo of the infrared target light point, and the light point of the infrared target.

[0086] Since the target centroid drifts or even the positioning fails due to the occlusion of the target by workers and mechanical equipment at the construction site, the method of multi-level thresholds is first used to segment the image and remove the halo of the target, which is convenient for subsequent image processing. The maximum inter-class variance method is to calculate the inter-class variance between the foreground and the background by traversing each pixel in the grayscale histogram to obtain the maximum inter-class variance.

[0087] The method of using the ROI segmentation method based on 4-connected-region to distinguish the connected regions of each light point can refer to the prior art.

[0088] The present invention proposes Otsu multi-threshold segmentation based on histogram pyramid, which has high algorithm efficiency and good segmentation effect; compared with the simple use of Otsu method for multi-threshold segmentation in the prior art, it can reduce the complexity of calculating the between-class variance.

[0089] Furthermore, in this embodiment, the two preliminarily determined thresholds are iteratively and finely adjusted to obtain two optimal thresholds of the histogram pyramid. The specific process is as follows:

[0090] Step I: Multiply the two preliminarily determined thresholds by the reciprocal of the scaling factor respectively to obtain two corresponding rough optimal thresholds for the layer i with a larger size in the upper layer a of the pyramid.

[0091] Step II: For each rough optimal threshold corresponding to the current layer i, search within the set pixel range around it, calculate the between-class variance corresponding to each pixel value, and select the pixel value with the largest between-class variance as the exact optimal threshold of this rough optimal threshold for the current layer i.

[0092] Step III: Take the exact optimal threshold of the current layer i as the rough optimal threshold of the upper layer i-1, and repeat Step II to obtain the exact optimal threshold of the upper layer i-1.

[0093] Step IV: Let i = i-1, repeat the operations of Step II and Step III until the optimal threshold of the layer with the largest size of the histogram pyramid is obtained, and take the optimal threshold of the layer with the largest size of the histogram pyramid as the optimal threshold of the histogram pyramid.

[0094] Step S212: If the current frame image is not the first frame image of the continuously captured images, the initial tracking window of the current frame image is the optimal tracking window of the previous frame image.

[0095] Step S22: According to the initial tracking window, use the pyramid iteration method to obtain the optimal tracking window of the current frame image and the centroid of the optimal tracking window.

[0096] In this embodiment, Step S22 includes the following sub-steps:

[0097] Step S221: Take the initial tracking window as the candidate tracking window, calculate the centroid position of the candidate tracking window, set (x, y) as the pixel point of the candidate tracking window, I(x, y) as the pixel value of the pixel point (x, y), and the 0th moment M 00 and the 1st moment M 10 、M 01 are:

[0098]

[0099] The centroid position (xc , y c ) is as follows:

[0100]

[0101] Step S222: Move the center of the candidate tracking window to the centroid position.

[0102] Step S223: Determine whether convergence occurs. If it converges, the current candidate tracking window is the best tracking window, and the centroid position of the current candidate tracking window is the centroid of the best tracking window; if not, reduce the size of the current candidate tracking window according to the set requirements, and recalculate the centroid within the candidate tracking window after the size reduction until convergence occurs.

[0103] Although other simpler algorithms only search for the brightest pixels, the pyramid method iteratively calculates the centroid of the current sub-window, and for the next iteration, it relocates a smaller sub-window within the current centroid and repeats the search. In each iteration, the maximum value of the pixels within the scanned area is calculated. When calculating the centroid, subtract from each pixel, which can converge to the centroid of the light spot faster. When calculating the centroid for the i-th iteration, the window edge is reduced by 1 pixel for the next iteration until the minimum scanned window of 3×3 is reached, defining the centroid of the searched window (x trac , y trac ) and the center of Rect mini (x rec , y rec ) has an error of W rec_trac .

[0104]

[0105] If W rec_trac is large, re-determine Track_window i . This method has sub-pixel accuracy because the centroid is calculated using floating-point values, and the pixel boundaries consider weighted values to exclude sub-pixel regions that fall outside the scanned area.

[0106] Furthermore, move the center of the candidate tracking window to the centroid position. If the distance between the center of the candidate tracking window and the centroid position is greater than the set threshold, re-determine the candidate tracking window and calculate the centroid position of the candidate tracking window.

[0107] Step S23: Within the best tracking window of the current frame image, track the light spot of the infrared target within the current tracking window through the Camshift algorithm to obtain the centroid of the infrared target.

[0108] Specifically: As Figure 3As shown in the figure, (1) if the infrared target is not blocked, four light points are traced, and the centroid of the infrared target is obtained by the method of two-dimensional Gaussian fitting.

[0109] When the target is blocked and the area A of the connected region i decreases, the centroid position will shift. As the degree of occlusion increases, the positioning accuracy of the centroid will drop sharply. In an ideal situation, the intensity distribution of the light spot should conform to the Gaussian distribution. The Gaussian surface fitting method has many advantages such as high accuracy and being applicable in the case of complex and variable-shaped images.

[0110]

[0111] In the formula, A is the amplitude of the light spot; (x 0 , y 0 ) is the centroid of the target; δ x , δ y are the standard deviations in the x and y directions respectively; I(x, y) is the intensity value. Taking the logarithm of both sides of the above formula gives:

[0112]

[0113] Converting the above formula into a polynomial:

[0114] z = ax 2 + by 2 + cx + dy + f

[0115] Then the centroid coordinates and amplitude can be used to obtain the centroid coordinates of the infrared target by the least squares method:

[0116]

[0117] (2) If one light point of the infrared target is blocked, three light points are traced, and the centroid of the infrared target is obtained by the method of area consistency constraint.

[0118] The area consistency constraint is:

[0119] A i ≥ a * A 0

[0120] Among them, A i is the area enclosed by the light points traced within the best tracking window of the current frame image, A 0 is the area enclosed by the four light points in the reference image, and a is the area coefficient; optionally, the value of a is 3 / 4, indicating that the area of the occlusion region is less than 1 / 4.

[0121] (3) If two light points of the infrared target are occluded, then two light points are tracked, the average value of the relative displacements of the two light points between the reference frame image and the current frame image is calculated, and the centroid of the reference frame image is corrected in combination with the shape consistency constraint to obtain the centroid of the infrared target in the current frame.

[0122] The shape consistency constraint is:

[0123]

[0124] Among them, (a 0 , b 0 , α 0 ) is the original shape of the reference image, and (a i , b i , α i ) is the shape of the minimum circumscribed rectangle enclosed by the light points tracked within the optimal tracking window of the current frame image. ε a , ε b and ε α are three set thresholds.

[0125] The reference frame image is usually the previous frame image.

[0126] (4) If three light points of the infrared target are occluded, then one light point is tracked, the relative displacement of one light point between the reference frame image and the current frame image is calculated, and the centroid of the reference frame image is corrected in combination with the shape consistency constraint to obtain the centroid of the infrared target in the current frame.

[0127] (5) If all four light points of the infrared target are occluded, then 0 light points are tracked, and the centroid of the infrared target in the current frame is estimated by the extended Kalman filter method based on the centroid positions of the infrared targets in a set number of historical frame images.

[0128] The method for estimating the centroid of the infrared target in the current frame by the extended Kalman filter method can refer to the prior art.

[0129] Step S24: Obtain the optimal tracking window and the centroid of the tracking window of the next frame image, and perform tracking on the infrared target in the next frame image until the centroid of the infrared target in each frame image is obtained;

[0130] Step S25: Calculate the displacement changes of the centroids of the infrared targets in all frame images to realize the monitoring of the dynamic changes of the high-mode.

[0131] Step S25 includes the following sub-steps:

[0132] As Figure 4 shown, Figure 4 in (a) is the displacement model diagram of the target under ideal installation conditions, Figure 4 in (b) is the displacement model diagram of the target when there is vibration,Figure 4 In (c), it is the displacement model diagram of the target when there are vibration and installation angle errors. Step S251: Calculate the pixel displacement d of the centroid of the infrared target in the current frame image and the previous frame image. y ;

[0133]

[0134] where y 0 is the value of the centroid of the infrared target in the previous frame image on the y-axis, y i is the value of the centroid of the infrared target in the current frame image on the y-axis, y i ' is the value of the center point of the minimum circumscribed rectangle of the centroid of the infrared target in the current frame image on the y-axis, A 0 is the area enclosed by the four light points in the reference image, A i is the area enclosed by the light points tracked within the best tracking window of the current frame image;

[0135] Step S252: Convert the pixel displacement into the displacement v of the high-mode infrared target in the world coordinate system;

[0136]

[0137] where D is the size of the infrared target in the world coordinate system, d is the pixel size of the infrared target in the camera coordinate system, l 0 is the pixel size of the acquisition camera;

[0138] Step S253: Remove the target vibration displacement and installation angle displacement in the displacement v of the high-mode infrared target in the world coordinate system to obtain the true displacement v of the high-mode infrared target y ;

[0139] v y = v - l * (cosβ - cosα)

[0140] where l is the height of the installation bracket of the infrared target, β is the installation angle deviation of the infrared target from the y-axis, and α is the rotation angle of the infrared target from the y-axis caused by vibration.

[0141] In this embodiment, further, it also includes step S26: According to the displacement changes of the centroids of all infrared targets, determine whether there is an abnormality in the high-mode, and when there is an abnormality, send corresponding warning information according to the abnormal situation.

[0142] The abnormal types of the high-mode include: F1, formwork settlement caused by ground settlement; F2, instability caused by excessive pressure on the jacks; F3, formwork settlement caused by local fastener failure; F4, structural overturning caused by overall displacement;

[0143] If the particle displacement of an infrared target reaches 80% of the set value, abnormal situations of abnormal types F2 and F3 may occur, and a first-level warning is issued;

[0144] If the particle displacements of multiple unrelated infrared targets all reach 80% of the set value, abnormal situations of abnormal types F1 and F4 may occur, and a second-level warning is issued;

[0145] If the particle displacements of multiple related infrared targets all reach 80% of the limit value, abnormal situations of abnormal types F2 and F3 may occur, and a second-level warning is issued;

[0146] If the particle displacement of a certain infrared target reaches the set value, abnormal situations of abnormal types F2 and F3 may occur, and a second-level danger warning is issued.

[0147] Embodiment 2

[0148] A lightweight multi-target visual perception method for high-branch formwork safety monitoring is given. An online camera is used to monitor the vertical displacement of the high-branch formwork. The current overall situation of the construction site is as Figure 7 shown. In the figure, the red area 1 is the construction monitoring area, and the area 2 is the placement position of the online camera. A total of three online cameras are used, and each online camera monitors multiple target points (independently powered). This project is a cast-in-place slab. The plane size of the support system of the slab is 58.596m x 20m, and the overall height is 22.5m, belonging to the high-branch formwork system. During the construction process, the vertical displacement of the high-branch formwork is monitored, and early warning is given when the displacement reaches the limit value to prevent accidents.

[0149] In the monitoring plan, the distances from the monitoring areas of axes ①, ②, and ③ to the camera are 20m, 28m, and 35m respectively. The three online cameras are placed on the same side. A total of 9 measuring points on axes ① and ② are measured simultaneously by the online camera and the total station (corresponding to Figure 7 the red dotted line range in the figure). The measuring points on axis ③ are monitored by the total station. The monitoring content includes the settlement amount of each measuring point, and the accuracy requirement is 1mm. The 9 measuring points within the red frame are measured by the online camera and rechecked by the total station, and the other measuring points are supplemented by the total station. The online camera is installed on the concrete beam of axis 9, and the concrete beam is connected to the base of the online camera through a steel bracket (bolt or welding), at the same horizontal plane position as the target, ensuring that the angle between the optical axis of the camera and the plane where the target is located is less than 5°. When the optical axis is not perpendicular to the image plane, the calibration magnification factor of the oblique optical axis is: SF 2 . The low formwork of the slab has been laid. Since the height of the cast-in-place slab beam is 7.9m, in order to achieve uninterrupted real-time measurement, an extension bracket needs to be designed to extend from the low formwork to the camera's field of view, and at the same time, the infrared target is stably connected to the extension bracket (using welding).

[0150] (2) Data Collection

[0151] Before pouring the platform, steel bars are tied and holes are reserved at the monitoring point to install the target bracket. The target plane position is checked in advance with the design drawings, and the designed bracket is installed and firmly fixed. A total of 9 monitoring points are set up in this monitoring, and three online cameras are used, each of which monitors three targets. At the same time, a total station is used for on-site monitoring, and the target position monitored by the online camera is re-measured.

[0152] The online camera has a sampling frequency of 2Hz and can work around the clock after installation, performing real-time calculation and data transmission, and issuing an early warning when the high-support formwork settlement reaches the early warning value. The total station monitors every two hours, with one person responsible for operating the total station and one person responsible for recording data and analysis.

[0153] The present invention is applied to the displacement monitoring of high-support formwork nodes and has been preliminarily verified in the laboratory. The sub-pixel center positioning technology of the coded marker point and the oblique optical axis calibration method when the position of the camera and the measuring point remains unchanged have been preliminarily verified in the laboratory. The IDS industrial camera model UI-3370CP is used, and the pixel size of the image sensor CMOS sensor is 5.5um, and the focal length of the lens is 35mm. The sub-pixel center of mass positioning method can accurately identify the center of mass position, and the magnification factor is calibrated in combination with the oblique optical axis calibration method. The measurement results are compared with the total station results. The data is processed in real time during the measurement process and can be displayed in the form of a line graph. A wireless module is installed to upload the processing results to the cloud in xlsx format. The displacement limit is set online, and an early warning is issued when the displacement of the measuring point reaches the limit.

[0154] The present invention is applied to the displacement monitoring of a large-scale cast-in-place reinforced concrete structure supporting formwork system with a length of 58.596m, a width of 20m and a height of 22.5m. The comparison results with the total station verify that the displacement monitoring accuracy of the visual perception method using the present invention reaches 0.74mm, indicating that the lightweight multi-target visual perception method for high-support formwork safety monitoring of the present application can realize real-time measurement and safety warning of long-term displacement of high-support formwork.

[0155] Example 3

[0156] like Figure 6 As shown, a lightweight multi-target visual perception system for high-rise formwork safety monitoring is provided, including

[0157] The acquisition module is used to continuously acquire images of different positions of the high-support formwork to be detected according to a preset sampling frequency; a plurality of infrared targets are arranged on the high-support formwork to be monitored, and each infrared target has four light spots;

[0158] The management and control module is used to track the infrared targets in the continuously captured images of each position of the high formwork through the Camshift-Gaussian-Centroid lightweight multi-target detection algorithm, obtain the displacement changes of the centroids of each infrared target, and realize the monitoring of the dynamic changes of the high formwork. Among them, when tracking the infrared target, if the infrared target is occluded, the centroid of the infrared target is obtained according to the occlusion degree of the infrared target through the CamShift algorithm and the robust target extraction method with area consistency constraint and shape consistency constraint. If the infrared target is not occluded, the centroid of the target is obtained through the CamShift algorithm and two-dimensional Gaussian fitting.

[0159] The core hardware in the acquisition module is the online camera host, which is fixed on the reference point through a stable mechanical device. The target target is imaged on the image sensor through the long focal length optical system based on the industrial camera, and the image processing technology in the embedded micro industrial computer is used to determine the displacement time history data of the target target.

[0160] The monitoring center of the management and control module is equipped with a host and a network storage device NAS. The host can operate the outdoor measurement end online camera through the remote control software for target tracking calculation, and at the same time collect the relevant videos and save them locally in the online camera for subsequent retrieval. The calculated displacement data can be saved in real time on the network virtual disk mapped by NAS or on the host disk. The monitoring center host can also read the disk displacement data for subsequent in-depth structural analysis of the structure.

[0161] In this embodiment, a lightweight multi-target visual perception system for high formwork safety monitoring further includes a communication and transmission module. The communication and transmission module uses a Dandelion industrial router to form a local area network between each online camera at the outdoor measurement end and the monitoring center, and can transmit the measurement results to the monitoring center through a wireless transmission system such as 4G or wireless network or a wired transmission system such as optical cable.

[0162] The various embodiments in this specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0163] Those skilled in the art can clearly understand that the technologies in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0164] The above are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements made without departing from the principle of the present invention should be regarded as within the protection scope of the present invention.

Claims

1. A lightweight multi-target visual perception method for high-rise formwork safety monitoring, characterized in that: include: According to the preset sampling frequency, continuously collect images of different positions of the high-support formwork to be inspected; A plurality of infrared targets are arranged on the high support formwork to be monitored, and each infrared target has four light spots; For each position of the high-support formwork, the Camshift-Gaussian-Centroid lightweight multi-target detection algorithm is used to track the infrared target in its continuously captured images, and the displacement change of the center of mass of each infrared target is obtained to realize the monitoring of the dynamic changes of the high-support formwork; when tracking the infrared target, if the infrared target is blocked, the center of mass of the infrared target is obtained according to the degree of occlusion of the infrared target through the CamShift algorithm and the target robust extraction method with area consistency constraint and shape consistency constraint; if the infrared target is not blocked, the center of mass of the target is obtained through the CamShift algorithm and two-dimensional Gaussian fitting.

2. The lightweight multi-target visual perception method for high-rise formwork safety monitoring according to claim 1 is characterized in that: For each position of the high-support formwork, the Camshift-Gaussian-Centroid lightweight multi-target detection algorithm is used to track the infrared target in its continuously shot images, obtain the displacement change of the center of mass of each infrared target, and realize the monitoring of the dynamic change of the high-support formwork. The specific process is as follows: For the current frame image of the continuously shot images, an initial tracking window for tracking each target is obtained; According to the initial tracking window, the pyramid iteration method is used to obtain the best tracking window and the centroid of the best tracking window of the current frame image; In the best tracking window of the current frame image, the light spots of the infrared target in the current tracking window are tracked by the Camshift algorithm to obtain the center of mass of the infrared target; specifically: if the infrared target is not blocked, four light spots are tracked, and the center of mass of the infrared target is obtained by two-dimensional Gaussian fitting; if one light spot of the infrared target is blocked, three light spots are tracked, and the center of mass of the infrared target is obtained by area consistency constraint; if two light spots of the infrared target are blocked, two light spots are tracked, the average value of the relative displacement of the two light spots of the reference frame image and the current frame image is calculated, and the center of mass of the reference frame image is corrected in combination with the shape consistency constraint to obtain the center of mass of the infrared target of the current frame; If the three light spots of the infrared target are blocked, one light spot is tracked, the relative displacement of one light spot between the reference frame image and the current frame image is calculated, and the center of mass of the reference frame image is corrected in combination with the shape consistency constraint to obtain the center of mass of the infrared target in the current frame; If all four light spots of the infrared target are blocked, zero light spots are tracked, and the center of mass of the infrared target in the current frame is estimated by the extended Kalman filter method based on the center of mass positions of the infrared target in the set number of historical frame images; Obtain the best tracking window and the centroid of the tracking window for the next frame of image, and track the infrared target for the next frame of image until the centroid of the infrared target for each frame of image is obtained; Calculate the displacement change of the center of mass of the infrared target in all frame images to monitor the dynamic changes of the high-rise model.

3. The lightweight multi-target visual perception method for high-rise formwork safety monitoring according to claim 2 is characterized in that: The current frame image of the continuously shot images is used to obtain an initial tracking window for tracking each target. If the current frame image is the first frame image of the continuously shot images, the first frame image is binarized by the Pyramid-Histogram-Otsu multi-threshold segmentation method to segment all the light spots of the first frame image; the ROI segmentation method based on 4-connected-region is used to distinguish the connected area of ​​each light spot, and the minimum bounding rectangle of the connected area of ​​each light spot is used as the initial tracking window of the first frame image; If the current frame image is not the first frame image of the continuously captured images, the initial tracking window of the current frame image is the best tracking window of the previous frame image.

4. The lightweight multi-target visual perception method for high-rise formwork safety monitoring according to claim 3 is characterized in that: The first frame image is binarized by the Pyramid-Histogram-Otsu multi-threshold segmentation method to segment all the light spots of the first frame image. The specific process is as follows: Perform pyramid processing on the first frame image to generate image representations of different scales; At each scale, the grayscale histogram of the image representation is calculated, and the grayscale histogram is reduced in dimension according to the scaling factor to construct a histogram pyramid; On any layer a of the histogram pyramid, the Otsu method is used to preliminarily determine two thresholds corresponding to layer a to distinguish three different areas of the image; the three different areas are the background, the halo of the infrared target light spot, and the infrared target light spot; Iteratively fine-tune the two initially determined thresholds to obtain two optimal thresholds of the histogram pyramid; According to the two optimal thresholds, the first frame image is segmented to obtain the background, the halo of the infrared target light spot and the light spot of the infrared target.

5. The lightweight multi-target visual perception method for high-rise formwork safety monitoring according to claim 4 is characterized in that: The two thresholds initially determined are iteratively fine-tuned to obtain two optimal thresholds of the histogram pyramid. The specific process is as follows: Step I: multiply the two thresholds initially determined by the inverse of the scaling factor respectively to obtain two rough optimal thresholds corresponding to the previous larger layer i of the pyramid layer a; Step II: For each rough optimal threshold corresponding to the current layer i, search within the set pixel range around it, calculate the inter-class variance corresponding to each pixel value, and select the pixel value with the largest inter-class variance as the precise optimal threshold of the rough optimal threshold of the current layer i; Step III: Use the precise optimal threshold of the current layer i as the rough optimal threshold of the previous layer i-1, and repeat step II to obtain the precise optimal threshold of the previous layer i-1; Step IV: let i=i-1, repeat the operations of steps II and III until the optimal threshold of the largest layer of the histogram pyramid is obtained, and the optimal threshold of the largest layer of the histogram pyramid is used as the optimal threshold of the histogram pyramid.

6. The lightweight multi-target visual perception method for high-rise formwork safety monitoring according to claim 2 is characterized in that: The optimal tracking window and the centroid of the tracking window of the current frame image are obtained by using the pyramid iteration method according to the initial tracking window. The specific process is as follows: The initial tracking window is used as the candidate tracking window, the centroid position of the candidate tracking window is calculated, (x, y) is set as the pixel point of the candidate tracking window, I(x, y) is the pixel value of the pixel point (x, y), and the 0th order moment M of the candidate tracking window is 00 and the first-order moment M 10 、M 01 for: The centroid position of the candidate tracking window (x c ,y c )for: Move the center of the candidate tracking window to the centroid position; Determine whether it converges. If it converges, the current candidate tracking window is the best tracking window, and the centroid position of the current candidate tracking window is the centroid of the best tracking window; If not, the size of the current candidate tracking window is reduced according to the set requirements, and the centroid is recalculated within the candidate tracking window after the size reduction until convergence.

7. The lightweight multi-target visual perception method for high-rise formwork safety monitoring according to claim 2 is characterized in that: The area consistency constraint is: TO i ≥a*A0 Among them, A i is the area enclosed by the light spots tracked in the optimal tracking window of the current frame image, A0 is the area enclosed by the four light spots in the reference image, and a is the area coefficient; The shape consistency constraint is: Among them, (a0, b0, α0) is the original shape of the reference image, (a i ,b i ,α i ) is the shape of the light spot tracked in the best tracking window of the current frame image, ε a , ε b and ε α Three thresholds are set.

8. The lightweight multi-target visual perception method for high-rise formwork safety monitoring according to claim 2 is characterized in that: The specific process of calculating the displacement change of the center of mass of the infrared target of all frame images is as follows: Calculate the pixel displacement d of the center of mass of the infrared target between the current frame image and the previous frame image y ; Among them, y0 is the value of the infrared target mass center on the y axis in the previous frame image, and y i is the value of the infrared target mass center on the y-axis of the current frame image, y i ' is the value of the center point of the minimum circumscribed rectangle of the infrared target centroid in the current frame image on the y-axis, A0 is the area enclosed by the four light spots in the reference image, A i is the area enclosed by the light spots tracked in the best tracking window of the current frame image; Convert the pixel displacement into the displacement v of the high-support infrared target in the world coordinate system; Where D is the size of the infrared target in the world coordinate system, d is the pixel size of the infrared target in the camera coordinate system, and l0 is the pixel size of the acquisition camera; Remove the target vibration displacement and installation angle displacement from the displacement v of the high-support infrared target in the world coordinate system to obtain the real displacement v of the high-support infrared target. y ; v y =vl*(cosβ-cosα) Among them, l is the height of the mounting bracket of the infrared target, β is the installation angle deviation between the infrared target and the y-axis, and α is the rotation angle between the infrared target and the y-axis caused by vibration.

9. The lightweight multi-target visual perception method for high-rise formwork safety monitoring according to claim 1 is characterized in that: Also includes: According to the displacement changes of the centroids of all infrared targets, it is determined whether there is any abnormality in the high-support formwork, and when there is an abnormality, corresponding warning information is issued according to the abnormal situation; The abnormal types of high-support formwork include: F1, formwork settlement caused by ground settlement; F2, instability caused by excessive pressure on the top rod; F3, formwork settlement caused by failure of local fasteners; F4, structural overturning caused by overall displacement; If the displacement of a particle of an infrared target reaches 80% of the set value, an abnormal situation of abnormal type F2 and F3 may occur, and a first-level warning is issued; If the particle displacements of multiple unrelated infrared targets all reach 80% of the set value, abnormal situations of abnormal types F1 and F4 may occur, and a level 2 warning will be issued; If the particle displacements of multiple related infrared targets all reach 80% of the limit value, abnormal situations of abnormal types F2 and F3 may occur, and a level 2 warning will be issued; If the particle displacement of a certain infrared target reaches the set value, abnormal situations of abnormal types F2 and F3 may occur, and a secondary danger warning will be issued.

10. A lightweight multi-target visual perception system for high-rise formwork safety monitoring, characterized in that: include: The acquisition module is used to continuously acquire images of different positions of the high-support formwork to be inspected according to a preset sampling frequency; A plurality of infrared targets are arranged on the high support formwork to be monitored, and each infrared target has four light spots; The management and control module is used to track the infrared target in the continuously shot images of each position of the high-support formwork through the Camshift-Gaussian-Centroid lightweight multi-target detection algorithm, obtain the displacement change of the center of mass of each infrared target, and realize the monitoring of the dynamic change of the high-support formwork; wherein, when tracking the infrared target, if the infrared target is blocked, the center of mass of the infrared target is obtained according to the degree of blockage of the infrared target through the CamShift algorithm and the target robust extraction method with area consistency constraint and shape consistency constraint; if the infrared target is not blocked, the center of mass of the target is obtained through the CamShift algorithm and two-dimensional Gaussian fitting.

Images