Bridge dynamic displacement high-precision identification method based on digital image processing

Abstract

The application relates to the technical field of image enhancement, in particular to a bridge dynamic displacement high-precision identification method based on digital image processing, which comprises the following steps: taking any frame video image in a video image sequence of a bridge to be identified as a target video image; determining each fuzzy area of the target video image according to the LBP value and the gradient amplitude of each pixel point in the target video image; determining the motion law index of each fuzzy area of the target video image in an adjacent frame and the motion track stability index in a continuous frame, and then determining the influence degree of each fuzzy area on the bridge displacement identification precision; performing enhancement processing on the target video image by using the influence degree, obtaining the target video image after the enhancement processing, and then obtaining each frame video image after the enhancement processing, so as to identify the bridge dynamic displacement. The application can overcome the noise influence of uneven light and bridge vibration in the video image to a certain extent, and is beneficial to the identification of the bridge dynamic displacement.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and more specifically to a high-precision method for identifying the dynamic displacement of bridges based on digital image processing. Background Technology

[0002] During long-term service, bridge structures experience a continuous decrease in load-bearing capacity and durability due to environmental erosion and cyclic loads, thus affecting the safety of road and bridge operations. Numerous factors influence the structural performance of bridges, and detailed and comprehensive safety inspections are fundamental for appropriate maintenance, reinforcement, and modification. In bridge safety monitoring, displacement change is one of the most important indicators, reflecting the vertical stiffness of the bridge beams and thus determining the stability and safety of the bridge structure.

[0003] In bridge dynamic displacement monitoring, image acquisition equipment is used to obtain image sequences of the bridge structure, and image processing algorithms are used to calculate the displacement changes of the structure. However, due to changes in natural lighting conditions, such as cloud cover, and bridge vibrations caused by vehicles traveling on the bridge, errors can easily occur in the identification and matching of key points of the bridge in consecutive frame images, making high-precision identification of bridge dynamic displacement inaccurate. Summary of the Invention

[0004] To address the aforementioned technical problem of inaccurate high-precision identification of bridge dynamic displacement in existing methods, the present invention aims to provide a high-precision identification method for bridge dynamic displacement based on digital image processing. The specific technical solution adopted is as follows:

[0005] One embodiment of the present invention provides a high-precision method for identifying the dynamic displacement of bridges based on digital image processing. The method includes the following steps:

[0006] Obtain the video image sequence of the bridge to be identified;

[0007] Taking any frame of the video image sequence as the target video image, each blurred region of the target video image is determined based on the LBP value and gradient magnitude of each pixel in the target video image; the blurred region is the region affected by external environmental factors.

[0008] Determine the motion pattern index of each blurred region in the target video image in adjacent frames and the motion trajectory stability index in consecutive frames; fuse the motion pattern index and the motion trajectory stability index to obtain the degree of influence of each blurred region on the bridge displacement recognition accuracy;

[0009] The blurred region in the target video image is enhanced using the degree of influence to obtain the enhanced target video image; then, each frame of the enhanced video image is acquired to identify the dynamic displacement of the bridge.

[0010] Further, determining the various blurred regions of the target video image based on the LBP value and gradient magnitude of each pixel in the target video image includes:

[0011] The target video image is divided into several local regions;

[0012] Based on the LBP value of each pixel in each local region, the degree of texture difference between each two local regions is analyzed to determine the local regions corresponding to each type of bridge structure.

[0013] Determine each target point and its matching value in each local region; the matching value is the minimum value among all LBP difference values ​​corresponding to two local regions of the same bridge structure, and the target point is the two pixel points corresponding to the minimum value;

[0014] Based on each target point and its matching value in each local region, the blur level of each local region is determined; based on the gradient magnitude of each pixel in each local region, the average gradient magnitude of each local region is determined.

[0015] Based on the degree of ambiguity and the average gradient magnitude of each local region, the interference factor for bridge displacement identification in each local region is determined.

[0016] The local region where the interference factor is not less than the preset interference threshold is defined as the fuzzy region.

[0017] Further, the step of analyzing the texture difference between every two local regions based on the LBP value of each pixel in each local region, and determining the local regions corresponding to each bridge structure, includes:

[0018] Calculate the difference between the LBP value of the target pixel in the first local region and the LBP value of each pixel in the second local region, and record it as the LBP difference value. Select the minimum LBP difference value. Take the average of the minimum LBP difference values ​​corresponding to all pixels in the first local region as the degree of texture difference between the first local region and the second local region.

[0019] The first local region and the second local region are two different local regions in the target video image, and the target pixel is any pixel in the first local region;

[0020] Based on the degree of texture difference between each pair of local regions, all local regions in the target video image are clustered to obtain each cluster; each local region in the same cluster is identified as a local region corresponding to the same bridge structure.

[0021] Further, determining the blur level of each local region based on each target point and its matching value in each local region includes:

[0022] For each local region, the target points that appear repeatedly in the target point set corresponding to the local region are identified as feature points. The number of target points corresponding to each feature point, as well as the number of target points and feature points in the local region, are counted.

[0023] The degree of blurring in the local region is determined based on the number of target points and the minimum matching value corresponding to each feature point, as well as the first ratio of the number of feature points to the number of target points in the local region.

[0024] The degree of ambiguity is negatively correlated with the number of target points and the first ratio, and positively correlated with the minimum matching value.

[0025] Further, determining the interference factor for bridge displacement identification for each local region based on the degree of ambiguity and the average gradient magnitude of each local region includes:

[0026] For each local region, the degree of texture difference between every two local regions corresponding to the bridge structure to which the local region belongs is obtained, and the minimum degree of texture difference is selected.

[0027] The fuzziness level, the mean gradient magnitude, and the minimum texture difference level of the local region are fused and analyzed to determine the interference factors of the local region on bridge displacement recognition.

[0028] The interference factor is positively correlated with the degree of blurring, and negatively correlated with both the mean gradient magnitude and the minimum texture difference.

[0029] Further, determining the motion pattern indices of each blurred region of the target video image in adjacent frames and the motion trajectory stability indices in consecutive frames includes:

[0030] Each feature point in each blurred region of the target video image is obtained, and then the matching point corresponding to each feature point in the next frame of the target video image is determined.

[0031] Determine the motion distance and direction between each feature point and its matching point to form a motion vector; based on the motion vector and position coordinates of each feature point in each blurred region, determine the motion law index of each blurred region in adjacent frames;

[0032] Matching analysis is performed on each feature point in each continuously distributed video frame to obtain the number of consecutive successfully matched frames corresponding to each feature point in each blurred region.

[0033] Based on the number of consecutively successfully matched frames and the degree of blurriness of each blurred region in each consecutively successfully matched video image, the motion trajectory stability index of each blurred region in consecutive frames is determined.

[0034] Furthermore, determining the motion pattern index of each blurred region in adjacent frames based on the motion vector and position coordinates of each feature point in each blurred region includes:

[0035] For each fuzzy region, based on the motion vector and position coordinates of each feature point in the fuzzy region, the magnitude difference of the motion vector between each pair of feature points, the angle between the motion vectors, and the Euclidean distance between each pair of feature points are determined.

[0036] The clustering distance between each pair of feature points is determined based on the difference in modulus, the included angle, and the Euclidean distance; the clustering distance is then used to cluster all feature points in the fuzzy region to obtain various clusters.

[0037] Based on the angle between the motion vectors of every two feature points in each cluster and the magnitude of the motion vector of each feature point, the motion pattern index of the blurred region in adjacent frames is determined.

[0038] Furthermore, determining the motion pattern index of the blurred region in adjacent frames based on the angle between the motion vectors of every two feature points in each cluster and the magnitude of the motion vector of each feature point includes:

[0039] Calculate the average value of all included angles and the variance of the motion vector magnitude in each cluster to obtain the motion consistency index of each cluster; select the clusters whose motion consistency index is greater than a preset consistency threshold as reference clusters; the motion consistency index is negatively correlated with both the average value and the variance.

[0040] Based on the number of clusters corresponding to the blurred region, the motion consistency index of each reference cluster, and the number of feature points, the motion pattern index of the blurred region in adjacent frames is determined; the motion pattern index is negatively correlated with the number of clusters, and positively correlated with both the motion consistency index and the number of feature points.

[0041] Further, the step of determining the motion trajectory stability index of each blurred region in consecutive frames based on the number of consecutively successfully matched frames and the blur degree of each blurred region in each consecutively successfully matched video image frame includes:

[0042] Obtain the number of pixels in each blurred region in each frame of video image that is continuously matched successfully, and use the second ratio of the number of pixels in the blurred region to the total number of pixels in the video image as the weight.

[0043] The degree of blurring is weighted and summed according to the weight of each blurred region in each video frame to obtain the degree of influence of each video frame on the accuracy of bridge displacement recognition.

[0044] For each feature point, a continuous matching index is determined based on the degree of influence of each video image frame on the accuracy of bridge displacement recognition and the third ratio of the number of consecutively successfully matched frames to the total number of frames. The continuous matching index is negatively correlated with the degree of influence and positively correlated with the third ratio.

[0045] By comprehensively analyzing the continuous matching index of all feature points within the same blurred region in the target video image, the motion trajectory stability index of each blurred region in consecutive frames is obtained.

[0046] Furthermore, the fusion of the motion law index and the motion trajectory stability index to obtain the degree of influence of each fuzzy region on the bridge displacement recognition accuracy includes:

[0047] For each fuzzy region, the product of the motion law index and the motion trajectory stability index of the fuzzy region is calculated; the product is then subjected to negative correlation normalization to obtain the degree of influence of the fuzzy region on the accuracy of bridge displacement recognition.

[0048] The present invention has the following beneficial effects:

[0049] This invention provides a high-precision method for identifying bridge dynamic displacement based on digital image processing. This method overcomes the influence of uneven lighting and bridge vibration noise in video images to a certain extent, enhances the accuracy of key point identification and matching in the process of bridge dynamic displacement identification, facilitates the identification of bridge dynamic displacement, and improves the accuracy of bridge dynamic displacement identification. First, by combining the LBP value and gradient magnitude of each pixel in the video image, each blurred region is determined. Blurred regions are areas affected by external environmental factors. Selecting these regions allows for targeted analysis of the impact of uneven illumination on bridge dynamic displacement recognition, reducing the amount of data analysis and facilitating adaptive image enhancement processing to mitigate the influence of uneven illumination in the video image. Second, the motion law index of each blurred region in the video image in adjacent frames and the motion trajectory stability index in consecutive frames are determined. This demonstrates that the invention considers not only the influence of illumination but also the impact of bridge vibration caused by vehicle movement. The motion law index and motion trajectory stability index are then used to quantify the degree of influence of blurred regions on the accuracy of bridge displacement recognition. Finally, the enhanced video image obtained by enhancing the corresponding blurred regions in the video image using the degree of influence results in a more accurate bridge dynamic displacement recognition, effectively ensuring the structural safety of the bridge. Attached Figure Description

[0050] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating a high-precision bridge dynamic displacement recognition method based on digital image processing, as provided in one embodiment of the present invention;

[0052] Figure 2 This is a flowchart illustrating the steps for determining various blurred regions of a target video image in an embodiment of the present invention.

[0053] Figure 3 This is a flowchart illustrating the steps of determining the motion law index of each blurred region of the target video image in adjacent frames and the motion trajectory stability index in consecutive frames in an embodiment of the present invention. Detailed Implementation

[0054] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solution proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0056] The application scenarios targeted by this invention can be:

[0057] During their service life, bridges bear various loads from vehicles and pedestrians, and also need to cope with changes in the natural environment, leading to fatigue damage and deformation of the bridge structure. To ensure the load-bearing capacity and safety of bridges, it is necessary to identify bridge displacement. However, during the process of detecting bridge dynamic displacement, the video images captured by cameras may contain noise, resulting in unclear areas in the video images, thus making it difficult to identify bridge dynamic displacement.

[0058] To enhance video image quality, specifically by eliminating noise and achieving high-precision recognition of bridge dynamic displacement, one embodiment of this invention provides a method for high-precision recognition of bridge dynamic displacement based on digital image processing. Figure 1 As shown, it includes the following steps:

[0059] S1, Obtain the video image sequence of the bridge to be identified.

[0060] In one embodiment, within a preset time period, high frame rate, high resolution industrial cameras or drones are used to continuously acquire images of the bridge structure of the bridge to be identified, and the acquired video images are preprocessed to obtain a video image sequence.

[0061] The preset time period can be the current 7 days, and the video image acquisition frame rate can be 25 frames per second or higher. The implementer can set it according to the specific actual situation. Image preprocessing includes, but is not limited to, image segmentation, image denoising and other processing methods.

[0062] In one embodiment, each frame of the video image sequence can be a grayscale image processed by a segmentation neural network, which can effectively remove the influence of background factors such as the sky area of ​​the non-bridge structure.

[0063] The segmentation neural network can be Mask R-CNN (Mask Regional Convolutional Neural Network), and the loss function used is the cross-entropy loss function.

[0064] Of course, the acquired video images can also be processed to remove noise, enhance contrast, and remove lighting changes, making the video images clearer and providing accurate image data for subsequent analysis. Common preprocessing methods include Gaussian blur.

[0065] It's worth noting that current noise suppression and deblurring techniques, such as Gaussian blur removal and image sharpening, primarily target static and relatively simple blurs. For complex blurs caused by external environmental factors (such as weather changes), traditional image processing methods are currently unable to fully recover image details and may even introduce new artifacts. Therefore, based on the noise characteristics caused by external environmental factors in the video image sequence, adaptive denoising and enhancement processing is performed on each frame of the video image sequence to ensure the accuracy of bridge dynamic displacement recognition based on the video image sequence.

[0066] Thus, this embodiment has obtained a video image sequence of the bridge structure to be identified.

[0067] S2, take any frame of video image in the video image sequence as the target video image, and determine each blurred region of the target video image based on the LBP value and gradient magnitude of each pixel in the target video image.

[0068] Here, the blurred area is the region in the target video image that is affected by external environmental factors.

[0069] In actual bridge dynamic monitoring, cameras are often located in external environments where lighting conditions are affected by time, weather, season, and the surrounding environment. For example, the brightness of the bridge surface can vary significantly between day and night, sunny and cloudy days, and whether the bridge is in direct sunlight or in the shade. Especially between consecutive frames, changes in lighting can cause significant changes in the brightness or contrast of certain areas, resulting in complex noise in the video images captured by the camera. This can lead to loss of detail or blurring, creating blurred or unclear areas in the image.

[0070] Each frame of a video image sequence needs to be analyzed. However, for the sake of understanding the solution, we will take any frame of video image as an example to perform image enhancement analysis. Any frame of video image in the video image sequence can be used as the target video image.

[0071] As an exemplary implementation, each blurred region of the target video image is determined as follows: Figure 2As shown, it includes:

[0072] S21, divide the target video image into several local regions.

[0073] In video images of bridge structures, some areas may become blurry due to noise, uneven lighting, or other factors. Analyzing these areas together with areas of clear texture can lead to errors in the image sharpness assessment. Therefore, based on the structural characteristics of bridges, regions with similar textures in the video image are clustered to avoid interference from noise in the sharpness assessment results and improve the accuracy of bridge displacement identification.

[0074] The structural characteristics of bridges refer to the presence of a large number of areas with similar textures. For example, bridges typically contain many repeating components, such as beams, columns, and trusses, which appear to have similar textures in the image.

[0075] In one embodiment, a superpixel segmentation algorithm is used to divide the target video image into several regions with similar texture, color, or structural features, denoted as local regions.

[0076] In this model, one local region corresponds to one component region. The implementation process of the superpixel segmentation algorithm is existing technology and will not be elaborated here. Of course, implementers can also use other image segmentation methods, such as region growing algorithms or gray-level co-occurrence matrices, etc., which are not specifically limited here.

[0077] S22, based on the LBP value of each pixel in each local region, analyze the degree of texture difference between each two local regions, and determine the local regions corresponding to each type of bridge structure.

[0078] Here, determining the local regions corresponding to each type of bridge structure refers to determining the local regions belonging to the same type of bridge structure. This is beneficial for analyzing the bridge displacement identification interference in the local regions of the same type of bridge structure.

[0079] In one embodiment, the Local Binary Pattern (LBP) is used to obtain the LBP value of each pixel in each local region.

[0080] Local Binary Patterns (LBP) are used to generate binary patterns from the values ​​of neighboring pixels, describing texture features. Since the LBP algorithm focuses on the relative size relationship between a pixel and its neighbors, rather than its absolute grayscale value, the LBP value remains unaffected even if the overall brightness changes, as long as the relative size relationship between adjacent pixels remains constant. Therefore, the LBP value of each pixel is used for data analysis here.

[0081] As an exemplary implementation, the various local regions corresponding to each type of bridge structure are determined, including:

[0082] The first step is to calculate the difference between the LBP value of the target pixel in the first local region and the LBP value of each pixel in the second local region, and record it as the LBP difference value. The minimum LBP difference value is selected. The average of the minimum LBP difference values ​​corresponding to all pixels in the first local region is taken as the degree of texture difference between the first local region and the second local region.

[0083] Here, the degree of texture difference refers to the dissimilarity of textures in two local regions. The first local region and the second local region are two different local regions in the target video image, and the target pixel is any pixel in the first local region.

[0084] In one embodiment, the first local region is denoted as the q-th region, the second local region as the p-th region, and the target pixel as the k-th pixel, where q, p, and k are all positive integers. The LBP difference between the k-th pixel in the q-th region and the LBP values ​​of all pixels in the p-th region is obtained. The minimum LBP difference is used as the texture difference factor of the k-th pixel in the q-th region, and the average texture difference factor of all pixels in the q-th region is used as the texture difference degree between the q-th region and the p-th region. Here, the LBP difference value is the absolute value of the difference between the LBP values ​​of two pixels.

[0085] The second step is to cluster all local regions in the target video image according to the degree of texture difference between each pair of local regions to obtain each cluster; and to identify each local region in the same cluster as the local region corresponding to the same bridge structure.

[0086] In one embodiment, the degree of texture difference is used as the clustering distance. All local regions in the target video image are clustered using the clustering distance between every two local regions, resulting in various clusters, each representing the same bridge structure. The implementation process of the clustering algorithm is existing technology and will not be elaborated here.

[0087] It is worth noting that since the local regions within each cluster are physically of the same type of components, their texture features should be highly consistent. This can effectively avoid interference from texture differences between different components and improve the accuracy of identifying ambiguous regions.

[0088] S23, determine each target point and its matching value in each local region.

[0089] Here, the matching value is the minimum of all LBP difference values ​​corresponding to two local regions of the same bridge structure, and the target points are the two pixels corresponding to the minimum value. The larger the matching value, the more obvious the texture difference between the two target points.

[0090] In one embodiment, for the j-th bridge structure, the minimum value among the LBP differences between all pixels in the q-th region and all pixels in the p-th region is obtained; the two pixels corresponding to the minimum value are denoted as a target point in the q-th region and a target point in the p-th region; and the minimum value is used as the matching value between the target points in the q-th region and the p-th region. Here, j is a positive integer.

[0091] By referring to the method for determining the target point in the q-th region and the target point in the p-th region, and their matching values, we can obtain each target point and its matching value in each local region.

[0092] S24, determine the blur level of each local region based on each target point and its matching value; determine the average gradient magnitude of each local region based on the gradient magnitude of each pixel in each local region.

[0093] Here, the degree of ambiguity refers to the degree of unclearness in a local area. The greater the degree of ambiguity, the greater the interference of the local area on bridge displacement identification.

[0094] As an exemplary implementation, determining the blur level of each local region includes:

[0095] The first step is to identify the recurring target points in the target point set corresponding to each local region as feature points, and then count the number of target points corresponding to each feature point, as well as the number of target points and feature points in the local region.

[0096] Here, a feature point refers to a target point in a local area that forms the minimum LBP difference value with a target point in multiple local areas of the same bridge structure. The more feature points in a local area, the higher the clarity of the local area, and vice versa.

[0097] The second step is to determine the degree of ambiguity in the local area based on the number of target points and the minimum matching value corresponding to each feature point, as well as the first ratio of the number of feature points to the number of target points in the local area.

[0098] Here, the minimum matching value refers to the minimum value among all matching values ​​of all target points corresponding to the feature point. If no minimum value exists, any matching value is randomly selected as the minimum matching value. The degree of ambiguity is negatively correlated with the number of target points and the first ratio, and positively correlated with the minimum matching value. Negative correlation means that the larger the independent variable, the smaller the dependent variable, while positive correlation means that the larger the independent variable, the larger the dependent variable.

[0099] As an example, the formula for calculating the blur level of the q-th region can be:

[0100] In the formula, Let represent the degree of blur in the q-th region, exp represent an exponential function with the natural constant as the base, exp(-) is used to implement negative correlation processing, L represent the number of feature points in the q-th region, and N represent the number of target points in the q-th region. This represents the first ratio of the number of feature points to the number of target points in the q-th region, where l represents the index of the feature point. This represents the number of target points corresponding to the l-th feature point in the q-th region. This represents the minimum matching value corresponding to the l-th feature point in the q-th region.

[0101] In the formula for calculating the degree of ambiguity, the ratio of the number of feature points to the number of target points is referred to as the first ratio to distinguish it from the other ratios in this embodiment, namely the second and third ratios; the first ratio The smaller the value, the fewer the number of feature points matched in the q-th region, the less clear the q-th region is, and the greater the degree of blurriness; the number of target points corresponding to feature points. The larger the value, the higher the clarity and the lower the blur of the q-th region, and the smaller the minimum matching value corresponding to the feature point. The smaller the value, the better the target point in the q-th region matches the target points in other regions of the same bridge structure, and the higher the clarity of the q-th region.

[0102] As an exemplary implementation, determining the average gradient magnitude of each local region includes:

[0103] The first step is to perform edge detection on each local region in the target video image to determine the gradient magnitude of each pixel within each local region.

[0104] In one embodiment, Canny edge detection is used to perform edge detection on a local region.

[0105] The second step is to calculate the average gradient magnitude of all pixels in the same local region based on the gradient magnitude of each pixel in each local region.

[0106] In this embodiment, the smaller the average gradient magnitude, the smoother the brightness edge of the corresponding local area, the more likely it is to be disturbed by noise and uneven lighting, lacking obvious edge information, and the more blurred the corresponding local area is.

[0107] S25. Based on the degree of ambiguity and the average gradient magnitude of each local region, determine the interference factor for bridge displacement identification in each local region.

[0108] Here, the interference factor refers to the influence of local image features on the bridge displacement recognition process.

[0109] As an exemplary implementation, determining the interference factor for bridge displacement identification in each local region includes:

[0110] The first step is to obtain the degree of texture difference between every two local regions corresponding to the bridge structure to which the local region belongs, and select the minimum degree of texture difference.

[0111] In this embodiment, the greater the minimum texture difference, the smaller the structural texture similarity between all local regions corresponding to the bridge structure to which the local region belongs. This further indicates that the clustering results when determining the bridge structure are less reliable, and the less confidence the fuzziness of the local region and the mean gradient magnitude are, the less trustworthy they are.

[0112] The second step involves a fusion analysis of the ambiguity, average gradient magnitude, and minimum texture difference in local areas to determine the interference factors of local areas on bridge displacement identification.

[0113] Here, the interference factor is positively correlated with the degree of blurring, and negatively correlated with the mean gradient magnitude and the minimum texture difference.

[0114] As an example, the formula for calculating the interference factor of the q-th region of the j-th bridge structure for bridge displacement identification can be:

[0115] In the formula, Let represent the interference factor of the q-th region of the j-th bridge structure in the z-th frame of the video image for bridge displacement identification, and norm represent the normalization function. This indicates the degree of blur in the q-th region. This represents the minimum texture difference corresponding to the j-th bridge structure in the z-th video frame. This represents the average gradient magnitude of the q-th region of the j-th type of bridge structure in the z-th frame of the video image.

[0116] In the formula for calculating the interference factor, and Generally, it is impossible for the value to be zero. However, to avoid extreme cases, a non-zero constant is added to the denominator of the fraction, such as taking an empirical value of 0.001. Similarly, this also applies to fractions in other calculation formulas.

[0117] S26, the local area where the interference factor is not less than the preset interference threshold is regarded as the fuzzy area.

[0118] Here, the larger the interference factor, the greater the probability that the local area is a fuzzy area.

[0119] In one embodiment, the interference factor ranges from 0 to 1, and the preset interference threshold can be set to 0.7. Local areas with an interference factor not less than 0.7 are considered as blurred regions, meaning local areas with an interference factor greater than or equal to 0.7 are considered as blurred regions, thus obtaining the various blurred regions in the target video image. The preset interference threshold can be set by the implementer according to specific circumstances, and is not specifically limited here.

[0120] Thus, this embodiment has obtained each blurred region in the target video image.

[0121] S3, determine the motion law index of each blurred region of the target video image in adjacent frames and the motion trajectory stability index in consecutive frames; fuse the motion law index and the motion trajectory stability index to obtain the degree of influence of each blurred region on the bridge displacement recognition accuracy.

[0122] During the video image acquisition process of the bridge structure, vehicles may be traveling on the bridge deck. When vehicles travel on the bridge, their own movement and the relative motion between the wheels and the bridge deck cause high-frequency vibrations in local structures of the bridge (such as the beams and bridge deck that the vehicles pass over). If the acquired video image is directly enhanced at this time, the noise in the blurred area may be amplified, interfering with the subsequent displacement calculation.

[0123] Noise manifests as random, discontinuous variations. In temporally continuous video images, noise typically exhibits random patterns of change within each frame, making it impossible to predict using simple temporal consistency. Therefore, if the changes in blurred areas are irregular or lack continuity, it indicates a greater impact of local areas on the accuracy of bridge displacement recognition.

[0124] As an exemplary implementation, the motion pattern index of each blurred region of the target video image in adjacent frames is determined as the motion trajectory stability index in consecutive frames, such as... Figure 3 As shown, it includes:

[0125] S31, obtain each feature point in each blurred region of the target video image, and then determine the matching point corresponding to each feature point in the next frame of the target video image.

[0126] In one embodiment, a feature matching algorithm is used to determine the matching point in the next frame of the target video image where each feature point has a similar descriptor. The implementation process of the feature matching algorithm is prior art and will not be described in detail here.

[0127] It is worth noting that feature points that do not have matching points will not participate in subsequent calculations and analyses.

[0128] S32, determine the motion distance and direction between each feature point and its matching point to form a motion vector; based on the motion vector and position coordinates of each feature point in each blurred region, determine the motion law index of each blurred region in adjacent frames.

[0129] In one embodiment, each feature point in the blurred region of the z-th frame video image is mapped to the z+1-th frame video image, and the Euclidean distance and direction between each feature point and the matched point are determined, which are respectively used as the motion distance and motion direction.

[0130] If the motion vector of a feature point exhibits random or inconsistent changes in adjacent frames, it means that the feature point may be affected by noise. Therefore, by analyzing the motion consistency of multiple feature points in a blurred region, the motion law index of the blurred region in adjacent frames can be determined.

[0131] As an exemplary implementation, determining the motion pattern index of each blurred region in adjacent frames includes:

[0132] The first step is to determine the magnitude difference of the motion vectors between each pair of feature points, the angle between the motion vectors, and the Euclidean distance between each pair of feature points, based on the motion vectors and position coordinates of each feature point in the fuzzy region.

[0133] In one embodiment, for every two feature points in the fuzzy region, the magnitude between each motion vector is first determined, and then the difference in magnitude between each two motion vectors is determined; the angle between the directions of each two motion vectors is obtained; and the Euclidean distance between each two feature points is calculated based on the position coordinates of each two feature points.

[0134] The second step is to determine the clustering distance between each pair of feature points based on the difference in modulus, the included angle, and the Euclidean distance. Then, the clustering distance is used to cluster all feature points in the fuzzy region to obtain each cluster.

[0135] In one embodiment, the modulus difference, included angle value, and Euclidean distance are first standardized to unify the dimensions, and then the clustering distance between every two feature points is calculated based on the standardized modulus difference, included angle value, and Euclidean distance.

[0136] As an example, the formula for calculating the cluster distance between every two feature points can be:

[0137] In the formula, D represents the clustering distance between two feature points. This represents the standardized value of the difference in modulus between two feature points. This represents the standardized value of the angle between two feature points. This represents the standardized value of the Euclidean distance between two feature points.

[0138] In one embodiment, based on the clustering distance between every two feature points within the same fuzzy region, the K-medoids clustering algorithm is used to perform clustering analysis on all feature points within the fuzzy region, resulting in several clusters. The implementation process of the K-medoids clustering algorithm is existing technology and will not be described in detail here.

[0139] The third step is to determine the motion pattern index of the blurred region in adjacent frames based on the angle between the motion vectors of every two feature points in each cluster and the magnitude of the motion vector of each feature point.

[0140] In bridge structures, actual displacements (such as the vertical vibration of the main beam) usually have consistent motion characteristics, that is, the motion direction and amplitude of feature points within the same cluster are not significantly different. Based on this, it can be known that if the feature points within a certain cluster fluctuate greatly, it may be local noise or non-structural motion with poor motion regularity.

[0141] As an exemplary implementation, determining the motion pattern index of the blurred region in adjacent frames includes:

[0142] The first sub-step involves calculating the average value of all included angles and the variance of the motion vector magnitude in each cluster to obtain the motion consistency index for each cluster; clusters with motion consistency indices greater than a preset consistency threshold are used as reference clusters.

[0143] Here, the larger the average value of all included angles in a cluster, the more chaotic the movement directions of the feature points in the cluster, the lower the consistency of movement directions, and the smaller the movement consistency index. Conversely, the larger the variance of the magnitude of the movement vectors of all feature points in a cluster, the greater the fluctuation of the movement distance of all feature points in the cluster, and the smaller the movement consistency index. Therefore, the movement consistency index is negatively correlated with both the average value and the variance.

[0144] As an example, the formula for calculating the motion consistency index of each cluster can be:

[0145] In the formula, Y represents the motion consistency index of the cluster, exp represents the exponential function with the natural exponent e as the base, and exp(-) is used to normalize the data for negative correlation. This represents the average of all included angle values ​​in the cluster. This represents the variance of the magnitudes of all motion vectors in the cluster.

[0146] In one embodiment, in order to filter out clusters with strong consistency in motion direction and motion amplitude in the fuzzy region, so as to facilitate the subsequent calculation of motion law index, clusters with motion consistency index greater than a preset consistency threshold are used as reference clusters. The preset consistency threshold can be an empirical value of 0.7.

[0147] The second sub-step involves determining the motion pattern index of the blurred region in adjacent frames based on the number of clusters corresponding to the blurred region, the motion consistency index of each reference cluster, and the number of feature points.

[0148] Here, the motion pattern index is negatively correlated with the number of clusters, and positively correlated with both the motion consistency index and the number of feature points.

[0149] As an example, the formula for calculating the motion pattern index of the blurred region in adjacent frames can be:

[0150] In the formula, This represents the motion pattern index of the m-th blurred region in the z-th video frame across adjacent frames, where m represents the index of the blurred region and norm represents the normalization function. This represents the number of clusters within a local region, where C represents the number of reference clusters and c represents the cluster index. This represents the motion consistency index of the c-th reference cluster. represents the number of feature points in the c-th reference cluster, and n represents the number of pixels in the c-th reference cluster.

[0151] In the formula for calculating the motion law index, The larger the value, the more feature points with inconsistent motion within the fuzzy area, the worse the motion regularity between feature corner points, and the greater the impact on the accuracy of bridge displacement recognition. The larger the value, the higher the motion consistency index. The greater the credibility, the stronger the motion regularity of the blurred area in adjacent frames.

[0152] It is worth noting that the blurred areas in the last frame of the video image can be excluded from the subsequent dynamic displacement analysis process.

[0153] It should be noted that, because blurred regions may unexpectedly possess similar descriptors or motion patterns between two adjacent video frames, they may be misjudged as having consistent motion. However, during the matching process with continuous temporal distribution, the randomness of noise points will gradually be exposed, leading to matching failures or trajectory interruptions. Therefore, it is necessary to further track the motion trajectory of blurred regions in consecutive frames and analyze the degree of influence of blurred regions on the accuracy of bridge displacement recognition, which corresponds to subsequent steps S33 to S34.

[0154] S33, perform matching analysis on each feature point in each continuously distributed video frame to obtain the number of consecutive successfully matched frames corresponding to each feature point in each blurred region.

[0155] Here, the larger the number of consecutive successful matching frames corresponding to a feature point, the higher the continuous matching rate of the feature point, and the smaller the impact on the accuracy of bridge displacement recognition.

[0156] In one embodiment, for each feature point within each blurred region in the target video image, the continuous matching coordinates of each feature point are recorded in each continuously distributed video frame, and the trajectory sequence of each feature point is constructed. If a feature point fails to match in a certain video frame, that is, there is no similar descriptor for the feature point in the video image, it is handled by interpolation or null value processing. The number of consecutively matched frames is counted from the trajectory sequence of each feature point to obtain the number of consecutively matched frames corresponding to each feature point in each blurred region.

[0157] It should be noted that the time-continuously distributed video images analyzed in this embodiment refer to the target video image and each frame of video image located after the acquisition time of the target video image.

[0158] S34. Based on the number of consecutively successfully matched frames and the degree of blurriness of each blurred region in each consecutively successfully matched video image, determine the motion trajectory stability index of each blurred region in consecutive frames.

[0159] As an exemplary implementation, based on the number of consecutively successfully matched frames and the blurriness of each blurred region in each consecutively successfully matched video frame, a motion trajectory stability index for each blurred region in consecutive frames is determined, including:

[0160] The first step is to obtain the number of pixels in each blurred region of each consecutively matched video frame, and use the second ratio of the number of pixels in the blurred region to the total number of pixels in the video image as the weight.

[0161] The larger the weight, the greater the proportion of the blurred region in the video image, and the higher the reliability of the blur degree of the blurred region, which facilitates subsequent weighted summation analysis of the blur degree of the blurred region based on the weight.

[0162] The second step involves weighted summation of the blur levels based on the weights of each blurred region in each video frame to obtain the degree of influence of each video frame on the accuracy of bridge displacement recognition.

[0163] As an example, the formula for calculating the influence of the z-th frame video image on the accuracy of bridge displacement recognition can be:

[0164] In the formula, This represents the degree of influence of the z-th frame of the video image on the accuracy of bridge displacement recognition, where norm represents the linear normalization function, M represents the number of blurred regions in the z-th frame of the video image, and m represents the index of the blurred region. This represents the degree of blur in the m-th blurred region of the z-th video frame. This represents the number of pixels in the m-th blurred region of the z-th video frame. This represents the total number of pixels in the z-th frame of the video image. This represents the second ratio, i.e., the weight.

[0165] By referring to the influence of the z-th video image on the accuracy of bridge displacement recognition, the influence of each video image on the accuracy of bridge displacement recognition can be obtained.

[0166] The third step involves determining the continuous matching index for each feature point based on the impact of each video frame on the accuracy of bridge displacement recognition and the third ratio of the number of consecutively successfully matched frames to the total number of frames.

[0167] Here, the continuous matching index is negatively correlated with the degree of influence, but positively correlated with the third ratio.

[0168] As an example, the formula for calculating the continuous matching index for each feature point can be:

[0169] In the formula, R represents the continuous matching index of feature points, and norm represents the normalization function. Z represents the number of consecutive successfully matched frames, and Z represents the total number of frames in the video image sequence. This represents the third ratio of the number of consecutive successfully matched frames to the total number of frames. This represents the cumulative value indicating the degree of influence of each consecutively successfully matched video frame on the accuracy of bridge displacement recognition.

[0170] In the formula for calculating the continuous matching index, the number of consecutively successfully matched frames corresponding to different feature points may vary, and the number of consecutively successfully matched video images may also differ. Therefore, the cumulative value of the influence varies. Possibly different; the cumulative value of the degree of influence. The larger the value, the more severe the blurring of the successfully matched video image, and the worse the image quality. This may reduce the stability and reliability of feature point matching, meaning that even if the feature points are successfully matched, there may still be a high degree of error or interference. (Third ratio) Its credibility is low.

[0171] The fourth step is to conduct a comprehensive analysis of the continuous matching index of all feature points in the same blurred region in the target video image to obtain the motion trajectory stability index of each blurred region in consecutive frames.

[0172] In one embodiment, the average value of the continuous matching index of all feature points within the same fuzzy region is calculated, and the average value of the continuous matching index of the fuzzy region is used as the motion trajectory stability index of the fuzzy region in consecutive frames.

[0173] As an exemplary implementation, by integrating motion law indicators and motion trajectory stability indicators, the degree of influence of each fuzzy region on the accuracy of bridge displacement recognition is obtained, including:

[0174] Here, both the motion law index and the motion trajectory stability index are dimensionless indices. The larger the motion law index and the motion trajectory stability index, the more stable the feature points in the fuzzy area are in continuous frames and can be continuously tracked. The more stable the motion trajectory of the feature points, the more likely it is to be the true displacement of the bridge. The less the fuzzy area affects the accuracy of bridge displacement recognition, the less the denoising enhancement of the fuzzy area will be.

[0175] In one embodiment, for each fuzzy region, the product of the motion law index and the motion trajectory stability index of the fuzzy region is calculated; the exp(-) function can be used to perform negative correlation normalization on the product to obtain the degree of influence of the fuzzy region on the accuracy of bridge displacement recognition.

[0176] Thus, this embodiment has obtained the degree of influence of each blurred region in the target video image on the accuracy of bridge displacement recognition.

[0177] S4, enhance the corresponding blurred area in the target video image according to the degree of influence to obtain the enhanced target video image; then obtain each frame of the enhanced video image to identify the dynamic displacement of the bridge.

[0178] As an exemplary implementation, the corresponding blurred region in the target video image is enhanced based on the degree of influence to obtain an enhanced target video image, including:

[0179] The first step is to determine the sharpening intensity of each pixel in the target video image based on the degree of influence of each blurred region in the target video image on the accuracy of bridge displacement recognition.

[0180] In one embodiment, the sharpening intensity of each pixel within each blurred region of the target video image is set to the influence level of the corresponding blurred region, and the sharpening intensity of each pixel outside the blurred regions of the target video image is set to zero, thereby obtaining the sharpening intensity of each pixel in the target video image. Specifically, the sharpening intensity of each pixel within the same blurred region is the same.

[0181] The second step is to sharpen the target video image according to the sharpening intensity of each pixel to obtain the enhanced target video image.

[0182] The process of sharpening video images based on sharpening intensity is an existing technology and will not be described in detail here.

[0183] As an exemplary implementation, acquiring each frame of video image after enhancement processing to identify the dynamic displacement of the bridge includes:

[0184] The first step is to obtain each frame of the enhanced video image by referring to the process of acquiring the enhanced target video image.

[0185] The second step is to identify the dynamic displacement of the bridge based on each frame of the enhanced video image.

[0186] As an exemplary implementation, the identification of dynamic displacement of the bridge is performed based on each frame of the enhanced video image, including:

[0187] The first sub-step involves using feature point detection algorithms such as AKAZE, SIFT, and BRIEF to extract key points from each frame of the enhanced video image, and then using algorithms such as FLANN and PROSAC to perform key point matching in order to determine the positional changes of key points in the video image that is continuously distributed over time.

[0188] Obtain the coordinates of the key points in the z-th frame of the enhanced video image, denoted as . After matching using FLANN and PROSAC, the corresponding point is obtained in the (z+1)th frame of the enhanced video image, denoted as . Then the expression for the displacement of the key point can be:

[0189] ; In the formula, Indicates the lateral displacement of key points. This indicates the longitudinal displacement of the key point.

[0190] The second sub-step involves calculating the displacement between keypoints using a sub-pixel matching algorithm, and then converting the pixel displacement into physical displacement using a scaling factor. The expression for this is:

[0191] ; In the formula, Indicates the lateral physical displacement of key points. Indicates the scaling factor. This represents the longitudinal physical displacement of the key point.

[0192] The third sub-step involves reconstructing the dynamically changing physical displacement by fusing accelerometer data or using multi-scale filtering methods to determine the dynamic displacement result of the bridge.

[0193] It should be noted that reconstructing dynamically changing physical displacements can smooth and optimize displacement data, eliminate instability caused by local errors in image processing or feature matching, and effectively extract the true dynamic change trend of the object, rather than local short-term errors, thereby improving the accuracy of bridge unique identification.

[0194] Thus, this embodiment completes the accurate identification of the dynamic displacement of the bridge.

[0195] This invention divides a single frame of video image into several blurred regions, analyzes the impact of different blurred regions on the accuracy of bridge displacement recognition, and uses this to adaptively sharpen and enhance the video image, thereby improving the accuracy of bridge dynamic displacement recognition and ensuring the structural safety of the bridge.

[0196] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A high-precision bridge dynamic displacement recognition method based on digital image processing, characterized in that, The method comprises the following steps: obtaining a video image sequence of a bridge to be identified; taking any frame video image in the video image sequence as a target video image, and determining each fuzzy region of the target video image according to the LBP value and gradient amplitude of each pixel point in the target video image; the fuzzy region is a region affected by external environmental factors; determining the motion law index of each fuzzy region in the adjacent frame and the motion trajectory stability index of each fuzzy region in the continuous frame of the target video image; and fusing the motion law index and the motion trajectory stability index to obtain the influence degree of each fuzzy region on the bridge displacement identification accuracy; performing enhancement processing on the fuzzy region in the target video image by using the influence degree to obtain an enhanced target video image; and then obtaining each frame video image after enhancement processing to identify the dynamic displacement of the bridge; the method of determining each fuzzy region of the target video image according to the LBP value and gradient amplitude of each pixel point in the target video image comprises: dividing the target video image into a plurality of local regions; analyzing the texture difference degree between each two local regions according to the LBP value of each pixel point in each local region, and determining each local region corresponding to each bridge structure; determining each target point and its matching value in each local region; the matching value is the minimum value of all LBP difference values corresponding to two local regions of the same bridge structure, and the target point is the two pixel points corresponding to the minimum value; determining the fuzzy degree of each local region according to each target point and its matching value of each local region; and determining the gradient amplitude mean value of each local region according to the gradient amplitude of each pixel point in each local region; determining the interference factor of each local region on the bridge displacement identification according to the fuzzy degree and the gradient amplitude mean value of each local region; regarding the local region with the interference factor not less than a preset interference threshold as a fuzzy region.

2. The high-precision bridge dynamic displacement recognition method based on digital image processing according to claim 1, characterized in that, the method of analyzing the texture difference degree between each two local regions according to the LBP value of each pixel point in each local region, and determining each local region corresponding to each bridge structure comprises: calculating the difference value between the LBP value of a target pixel point in a first local region and the LBP value of each pixel point in a second local region, denoted as LBP difference value, and selecting the minimum LBP difference value; and taking the average value of the minimum LBP difference value corresponding to all pixel points in the first local region as the texture difference degree between the first local region and the second local region; the first local region and the second local region are two different local regions in the target video image, and the target pixel point is any pixel point in the first local region; performing clustering processing on all local regions in the target video image according to the texture difference degree between each two local regions to obtain each cluster; and determining each local region in the same cluster as each local region corresponding to the same bridge structure.

3. The high-precision bridge dynamic displacement recognition method based on digital image processing according to claim 2, characterized in that, the method of determining the fuzzy degree of each local region according to each target point and its matching value of each local region comprises: For each local region, determine the target points repeatedly appearing in the target point set corresponding to the local region as feature points, and count the number of target points corresponding to each feature point, as well as the number of target points and the number of feature points in the local region; Determine the blur degree of the local region according to the number of target points corresponding to each feature point and the minimum matching value, and the first ratio of the number of feature points and the number of target points in the local region. The blur degree is negatively correlated with the target point number and the first ratio, and positively correlated with the minimum matching value.

4. The high-precision bridge dynamic displacement recognition method based on digital image processing according to claim 3, characterized in that, The determination of the interference factor of each local region to the bridge displacement recognition according to the blur degree and the average gradient amplitude of each local region comprises: For each local region, obtain the texture difference degree between each two local regions corresponding to the bridge structure to which the local region belongs, and select the minimum texture difference degree; Fuse and analyze the blur degree, the average gradient amplitude and the minimum texture difference degree of the local region to determine the interference factor of the local region to the bridge displacement recognition; The interference factor is positively correlated with the blur degree, and negatively correlated with the average gradient amplitude and the minimum texture difference degree.

5. The high-precision bridge dynamic displacement recognition method based on digital image processing according to claim 3, characterized in that, The determination of the motion rule index of each blur region in the adjacent frame and the motion trajectory stability index of each blur region in the continuous frame of the target video image comprises: Obtain each feature point in each blur region of the target video image, and then determine the matching point corresponding to each feature point in the next frame of the target video image; Determine the motion distance and motion direction between each feature point and its matching point to form a motion vector, and determine the motion rule index of each blur region in the adjacent frame according to the motion vector and the position coordinates of each feature point in each blur region; Perform matching analysis on each feature point in each frame of the continuously distributed video image to obtain the number of continuous matching success frames corresponding to each feature point in each blur region; Determine the motion trajectory stability index of each blur region in the continuous frame according to the number of continuous matching success frames and the blur degree of each blur region in each frame of the continuously matching success video image.

6. The high-precision bridge dynamic displacement recognition method based on digital image processing according to claim 5, characterized in that, The determination of the motion rule index of each blur region in the adjacent frame according to the motion vector and the position coordinates of each feature point in each blur region comprises: For each blur region, determine the length difference of the motion vector, the angle value of the motion vector and the Euclidean distance between each two feature points according to the motion vector and the position coordinates of each feature point in the blur region; Determine the clustering distance of each two feature points according to the length difference, the angle value and the Euclidean distance corresponding to each two feature points, and cluster all feature points in the blur region by using the clustering distance to obtain each cluster; Determine the motion rule index of the blur region in the adjacent frame according to the angle value between the motion vectors of each two feature points in each cluster and the length of the motion vector of each feature point.

7. The high-precision bridge dynamic displacement recognition method based on digital image processing according to claim 6, characterized in that, The motion rule index of the blur area in the adjacent frame is determined according to an included angle value between motion vectors of each two feature points in each cluster and a motion vector module length of each feature point, and the method comprises the following steps: An average value of all included angle values and a variance of motion vector module lengths in each cluster are calculated to obtain a motion consistency index of each cluster; a cluster with a motion consistency index greater than a preset consistency threshold is regarded as a reference cluster; the motion consistency index is negatively correlated with the average value and the variance; A motion rule index of the blur area in the adjacent frame is determined according to the number of clusters corresponding to the blur area, the motion consistency index of each reference cluster and the number of feature points; the motion rule index is negatively correlated with the number of clusters and positively correlated with the motion consistency index and the number of feature points.

8. The high-precision bridge dynamic displacement recognition method based on digital image processing according to claim 5, characterized in that, The motion trajectory stability index of each blur area in the continuous frames is determined according to the number of continuous matching successful frames and the blur degree of each blur area in each video image with continuous matching success, and the method comprises the following steps: The number of pixel points of each blur area in each video image with continuous matching success is obtained, and a second ratio of the number of pixel points of the blur area to the total number of pixel points of the video image is taken as a weight; The blur degree is weighted and summed according to the weight of each blur area in each video image to obtain the influence degree of each video image on the bridge displacement recognition accuracy; The continuous matching index of each feature point is determined according to the influence degree of each video image on the bridge displacement recognition accuracy, a third ratio of the number of continuous matching successful frames to the total number of frames, respectively for each feature point; the continuous matching index is negatively correlated with the influence degree and positively correlated with the third ratio; The continuous matching indexes of all feature points in the same blur area in the target video image are comprehensively analyzed to obtain the motion trajectory stability index of each blur area in the continuous frames.

9. The high-precision bridge dynamic displacement recognition method based on digital image processing according to claim 1, characterized in that, The influence degree of each blur area on the bridge displacement recognition accuracy is obtained by fusing the motion rule index and the motion trajectory stability index, and the method comprises the following steps: The product of the motion rule index and the motion trajectory stability index of each blur area is calculated, respectively for each blur area; the product is negatively correlated and normalized to obtain the influence degree of the blur area on the bridge displacement recognition accuracy.

Images