Method and device for monitoring feature point displacement based on video and adaptive environmental changes

Abstract

The present invention discloses a method and device for monitoring feature point displacement based on adaptive environmental changes of video, the method comprising: setting a target structure to be monitored, obtaining a long-term detection video of the target structure, pre-processing it, and detecting feature points in a designated region of interest of an initial video frame of the target structure using the Shi-Tomasi corner point detection method; based on the feature points initially detected, using the Lucas-Kanade optical flow method to calculate the displacement size of the feature points initially detected, and dynamically adjusting feature point detection parameters according to the displacement size of the feature points; eliminating feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold, and finally obtaining a target feature point displacement set after adaptive adjustment and optimization, and calculating the average value of all target feature point displacements as the pixel displacement of the target structure in the region of interest. The present invention can improve the robustness and accuracy of physical displacement acquisition.

Description

Technical Field

[0001] The present invention relates to the technical field of structural health monitoring, and in particular to a method and device for monitoring characteristic point displacement based on video and adaptive environmental changes. Background Art

[0002] In civil engineering, structural displacement is one of the important indicators for evaluating the health of dynamic structures. Traditional contact sensor measurement methods are difficult to meet the requirements of high speed, high precision and full field of view, and the accuracy of measurement may be affected due to physical contact. In contrast, non-contact measurement methods based on computer vision have significant advantages, which can achieve multi-point measurement of target structures and avoid the problems caused by physical contact. However, existing vision-based measurement methods still have large errors when environmental factors change (such as lighting, viewing angle, noise, rain, etc.).

[0003] Patent document CN115690150A discloses a video-based multi-target displacement tracking monitoring method and device, the method comprising: obtaining the structural parameters of the target building and establishing a corresponding structural three-dimensional model, obtaining the corresponding camera parameters; setting the monitoring target to be monitored in the target building, and outputting an area of ​​interest containing at least one target feature point; using the pyramid streamer method to track the displacement of all target feature points in the area of ​​interest, eliminating the target feature points with abnormal displacement data in the tracking results, and calculating the displacement average of all uneliminated target feature points as the pixel displacement of the monitoring target in the area of ​​interest; according to the camera parameters, the pixel displacement of all monitoring targets is reversely mapped to the three-dimensional model to obtain the physical displacement of the monitoring target in the real environment. This method cannot well resist the influence of changes in environmental factors on the structural target under long-term monitoring.

[0004] Patent document CN118032773A discloses an alarm device based on bridge structure health detection, including: a limit plate is fixedly connected to one side of the mounting frame, the bottom outer wall of the sliding rod at the top of the limit plate is rotatably connected to a rotating frame, a rangefinder is fixedly connected to one side of the rotating frame, and a waist-shaped hole is opened on one side of the detection plate on one side of the rotating frame for use with the infrared rays emitted by the rangefinder; a camera is arranged on the rotating frame for surveying the damaged part; a controller and an alarm are arranged in the display, the camera is connected to the display by wires for alarming the detected damaged part, the controller is connected to the rangefinder by wires, and the alarm is connected to the controller by wires; a transmission mechanism is arranged on the limit plate for driving the sliding rod to move downward and driving the rotating frame to rotate so that the detection plate contacts the bottom of the bridge. The device can solve the problem in the prior art that after the damaged position of the bridge is known, it is necessary to manually go down to the bottom of the bridge for surveying, but its detection depends on the accuracy of the sensor, and it is easy to be damaged during use, and it is inconvenient for daily maintenance. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for monitoring feature point displacement based on video and adaptive to environmental changes in view of the deficiencies of the prior art. The present invention dynamically adjusts feature point extraction, matching and optimization parameters to adapt to the movement and changes of feature points under different scenes and environmental conditions, thereby improving the accuracy and robustness of feature point detection and tracking.

[0006] The objective of the present invention is achieved through the following technical solutions: In a first aspect, an embodiment of the present invention provides a method for monitoring feature point displacement based on video and adaptive environmental changes, comprising the following steps:

[0007] (1) Set the target structure to be monitored, obtain a long-term detection video of the target structure, preprocess the video, and use the Shi-Tomasi corner detection method to detect feature points in the specified area of ​​interest of the initial video frame of the target structure;

[0008] (2) Based on the feature points initially detected in step (1), the Lucas-Kanade optical flow method is used to calculate the displacement size of the initially detected feature points, and the feature point detection parameters are dynamically adjusted according to the displacement size of the feature points;

[0009] (3) Based on the feature points obtained in step (2), the feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold are eliminated, and finally a set of target feature point displacements after adaptive adjustment and optimization is obtained, and the average value of the displacements of all target feature points is calculated as the pixel displacement of the target structure in the region of interest.

[0010] Furthermore, the target structures include node plates of truss beam bridges, corner points of roofs of high-rise buildings, corner points of floor windows of high-rise buildings, intersection points of bridge railings and bridges, anchorage points of cable-stayed bridges, anchorage points of suspension bridge cables, and intersection points of truss arch bridge members.

[0011] Furthermore, the preprocessing specifically includes: video capture, cropping and denoising.

[0012] Further, the Shi-Tomasi corner point detection method is used to detect feature points in a designated area of ​​interest of an initial video frame of the target structure, specifically including:

[0013] Read the first frame of the video;

[0014] Set the initial values ​​of the feature point detection parameters and use the Shi-Tomasi corner point detection method to detect feature points in the area of ​​interest of the first frame image;

[0015] Record the detected feature point coordinates.

[0016] Furthermore, the region of interest is obtained specifically by the following method:

[0017] Manual annotation: Use image processing tools to manually annotate rectangular bounding boxes to select regions of interest in the target structure;

[0018] Image segmentation: Automatically detect regions of interest using image segmentation algorithms;

[0019] Contour Detection: Use edge detection algorithms and contour detection algorithms to extract the contours of target structures to determine the region of interest.

[0020] Furthermore, based on the feature points initially detected in step (1), the Lucas-Kanade optical flow method is used to calculate the displacement size of the initially detected feature points, which specifically includes:

[0021] The Lucas-Kanade optical flow method is used to calculate the displacement of the initially detected feature points between consecutive video frames. Specifically, for the initially detected feature points, the calcOpticalFlowPyrLK function of the OpenCV library is first used to calculate the optical flow, and the new position and new state of the feature points are obtained by updating the video frame; then, the initially detected feature points are screened according to the new state of the feature points, and only the feature points with a state of 1 are retained as valid feature points, and their displacement is calculated according to the new and old positions of the valid feature points.

[0022] Furthermore, the dynamically adjusting the feature point detection parameters according to the displacement size of the feature point specifically includes:

[0023] When the displacement value of the feature point is greater than or equal to the preset displacement value threshold of the feature point, the quality threshold and the neighborhood size are increased, and the minimum distance is reduced;

[0024] When the displacement value of the feature point is less than the preset displacement value threshold of the feature point, the quality threshold and the neighborhood size are reduced, and the minimum distance is increased.

[0025] Furthermore, based on the feature points obtained in step (2), the feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold are eliminated, which specifically includes:

[0026] The Lucas-Kanade optical flow method is used to monitor the maximum and minimum displacements of the feature points of the target structure during the entire vibration process, and the difference between the maximum and minimum displacements of the feature points is calculated as the tracking displacement error of the feature point. Based on the feature points obtained in step (2), the feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold are eliminated.

[0027] A second aspect of an embodiment of the present invention provides a feature point displacement monitoring device based on video and adaptive environmental changes, comprising one or more processors and a memory, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned feature point displacement monitoring method based on video and adaptive environmental changes.

[0028] A third aspect of an embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the above-mentioned feature point displacement monitoring method based on video and adaptive environmental changes.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] (1) When selecting the region of interest of a structural target, the present invention directly marks the rectangular bounding box selected in the video through an algorithm, without the need for on-site manual marking, thus saving labor costs and expanding application scenarios.

[0031] (2) The present invention adopts an efficient calculation method in the process of feature point detection and parameter adjustment, which can complete the detection and tracking of feature points in a short time and meet the real-time requirements.

[0032] (3) By dynamically adjusting the feature point detection parameters, the present invention can better adapt to the changes in feature point displacement under different scenes and environmental conditions, improve the accuracy of feature point detection, and reduce cumulative errors; compared with the traditional fixed parameter method, the present invention can provide more accurate feature point displacement detection results in complex environments.

[0033] (4) The present invention eliminates feature points with low tracking quality or large errors through the feature point optimization step, thereby reducing errors in the feature point detection and tracking process and making it less likely for cumulative errors to occur; the present invention effectively reduces these errors and improves the accuracy of the detection results through adaptive adjustment and optimization.

[0034] (5) The present invention can handle complex environmental influences such as illumination changes, viewing angle changes, noise interference, and rain interference by adaptively adjusting feature point detection parameters. It has strong robustness and accuracy, and provides an effective non-contact measurement method for structural health monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of a method for monitoring feature point displacement based on video and adaptive environmental changes provided by an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of a truss beam model, a camera model and monitoring points provided by an embodiment of the present invention;

[0037] Figure 3 is a displacement monitoring result diagram of the region of interest in an embodiment of the present invention; wherein, Figure 3 (a) is the displacement monitoring result of the region of interest under illumination changes. Figure 3 (b) is the displacement monitoring result of the region of interest under the change of viewing angle. Figure 3 (c) is the displacement monitoring result of the region of interest under noise interference. Figure 3 (d) is the displacement monitoring result of the area of ​​interest under rain interference. Figure 3 (e) is a displacement monitoring result diagram of the region of interest of the original video without interference;

[0038] Figure 4 It is a structural schematic diagram of the feature point displacement monitoring device based on video and adaptive to environmental changes of the present invention. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. It should be understood that the above general description and the detailed description that follows are merely exemplary and explanatory and cannot limit the present application.

[0040] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0041] It should be understood that, although the term first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of..." or "when..." or "in response to determination". Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process and method including a series of elements include not only those elements, but also include other elements not clearly listed, or also include elements inherent to such process and method. In the absence of more restrictions, the elements limited by the statement "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0042] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0043] See also Figure 1 The method for monitoring the displacement of feature points based on video and adaptive environmental changes of the present invention specifically comprises the following steps:

[0044] (1) Set the target structure to be monitored, obtain a long-term detection video of the target structure, preprocess the video, and use the Shi-Tomasi corner detection method to detect feature points in the specified region of interest of the initial video frame of the target structure.

[0045] Furthermore, the target structures include but are not limited to: node plates of truss beam bridges, corner points of roofs of high-rise buildings, corner points of floor windows of high-rise buildings, intersections of bridge railings and bridges, anchorage points of cable-stayed bridges, anchorage points of suspension bridge cables, and intersections of truss arch bridge members, etc.

[0046] Depend on Figure 2 As shown, the target structure monitored in this embodiment is a truss beam bridge as a sample, and the vibration of the third node of the lower chord of the truss beam bridge is monitored. The long-term detection video of the target structure is obtained by camera detection.

[0047] Furthermore, the video is preprocessed, specifically including: video capture, cropping and denoising, which can reduce the error of the original video.

[0048] Furthermore, after video preprocessing, in order to simulate interference in a real environment, it is necessary to obtain videos under lighting changes (including bright light and dark light), viewing angle changes (rotation 180 degrees), noise interference (including strong noise and weak noise) and rain interference (including heavy rain and light rain). The specific steps are as follows:

[0049] (1.1.1) Import necessary libraries: cv2 for OpenCV functions.

[0050] (1.1.2) Open video file: Use cv2.VideoCapture to open the input video file.

[0051] (1.1.3) Get video information: Get the width, height and frame rate of the video.

[0052] (1.1.4) Define video writer: Use cv2.VideoWriter to write the adjusted frames to the output video file.

[0053] (1.1.5) Process video frames: Read each frame of video, process each frame according to the corresponding simulation environment requirements, and write it to the output video. For example, adjust the brightness and contrast under lighting changes, rotate the video when the viewing angle changes, add noise under noise interference, use cv2.line to generate random raindrops under rain interference, and adjust the brightness and contrast of the frame to simulate the dim effect of rainy days.

[0054] (1.1.6) Release resources: Release video capture and writing objects, and close all display windows.

[0055] Furthermore, the Shi-Tomasi corner point detection method is used to detect feature points in a designated region of interest of the initial video frame of the target structure, specifically including the following steps:

[0056] (1.2.1) Read the first frame of the video.

[0057] (1.2.2) Set the initial values ​​of the feature point detection parameters, and use the Shi-Tomasi corner point detection method to detect feature points in the region of interest of the first frame image. The feature point detection parameters include but are not limited to the quality threshold (qualityLevel), neighborhood size (blockSize), minimum distance (minDistance), input image (image), and the maximum number of corner points returned (maxCorners).

[0058] (1.2.3) Record the coordinates of the detected feature points.

[0059] It should be noted that in addition to using the Shi-Tomasi corner point detection method to detect feature points in the specified area of ​​interest of the initial video frame in the preprocessed original video, it is also necessary to use the Shi-Tomasi corner point detection method to detect feature points in the specified area of ​​interest of the initial video frame in the video affected by simulated environmental factors, so as to facilitate the subsequent verification of the method described in the present invention to verify that the method described in the present invention can adapt to environmental changes.

[0060] Among them, the acquisition of the region of interest (ROI) is one of the key steps in feature point detection and tracking. By obtaining the region of interest, the amount of calculation can be reduced, the algorithm efficiency can be improved, and the key parts of the target structure can be focused on for analysis. The region of interest is obtained by the following method:

[0061] (1.3.1) Manual annotation: Use image processing tools (such as OpenCV) to manually annotate rectangular bounding boxes to select regions of interest in the target structure. Users can select ROIs through the interactive interface of image processing tools, which can improve flexibility and accuracy.

[0062] (1.3.2) Image segmentation: Use image segmentation algorithms (such as K-means clustering, GrabCut, threshold segmentation, etc.) to automatically detect regions of interest. Image segmentation algorithms can separate structural regions from background regions.

[0063] (1.3.3) Contour detection: Use edge detection algorithms (such as Canny) and contour detection algorithms (such as findContours) to extract the contours of the target structure to determine the region of interest.

[0064] (2) Based on the feature points initially detected in step (1), the Lucas-Kanade (LK) optical flow method is used to calculate the displacement size of the initially detected feature points, and the feature point detection parameters are dynamically adjusted according to the displacement size of the feature points.

[0065] Among them, the displacement size of the feature point is the absolute displacement.

[0066] Furthermore, based on the feature points preliminarily detected in step (1), the Lucas-Kanade optical flow method is used to calculate the displacement of the preliminarily detected feature points between consecutive video frames, specifically including: using the Lucas-Kanade optical flow method to calculate the displacement of the preliminarily detected feature points between consecutive video frames, specifically: for the preliminarily detected feature points, first use the calcOpticalFlowPyrLK function of the OpenCV library to calculate the optical flow, and obtain the new position and new state of the feature points by updating the video frame; then filter the preliminarily detected feature points according to the new state of the feature points, retain only the feature points with a state of 1 as valid feature points, and calculate their displacement according to the new position and old position of the valid feature points.

[0067] It should be understood that the calculation of optical flow can be achieved through the calcOpticalFlowPyrLK function of the OpenCV library, which is implemented based on the Lucas-Kanade optical flow method. The Lucas-Kanade optical flow method is a classic optical flow algorithm for tracking the movement of feature points in an image sequence, which is mainly used for the calculation of sparse optical flow fields. The main advantages of the LK optical flow method are two-fold. In terms of scalability, it can be used in combination with other technologies. For example, the Pyramid Lucas-Kanade optical flow can handle a larger range of motion and improve the applicability of the algorithm. In terms of application, the LK optical flow method has a wide range of applications and mature implementations in the field of computer vision. For example, it can be applied to target tracking, motion estimation, and robot navigation.

[0068] Furthermore, the feature point detection parameters are dynamically adjusted according to the displacement size of the feature point, which specifically includes the following steps:

[0069] (2.1) When the value of the feature point displacement size is large, that is, when the value of the feature point displacement size is greater than or equal to the preset feature point displacement size threshold, increase the quality threshold and the neighborhood size and reduce the minimum distance.

[0070] (2.2) When the value of the feature point displacement size is small, that is, when the value of the feature point displacement size is less than the preset feature point displacement size threshold, the quality threshold and the neighborhood size are reduced, and the minimum distance is increased.

[0071] It should be understood that the vibration displacement of each structure is different, and the displacement threshold for monitoring each structure is different, so the displacement threshold of the feature point can be preset according to actual needs. In addition, the quality threshold in the feature point detection parameters corresponds to the optimization parameters of the feature points, and the neighborhood size and minimum distance in the feature point detection parameters correspond to the extraction parameters and matching parameters of the feature points. Therefore, it is only necessary to dynamically adjust the quality threshold, neighborhood size and minimum distance in the feature point detection parameters according to the displacement size of the feature points. By dynamically adjusting the feature point detection parameters, it is possible to adapt to environmental changes.

[0072] (3) Based on the feature points obtained in step (2), the feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold are eliminated, and finally a target feature point displacement set after adaptive adjustment and optimization is obtained, and the average value of the displacements of all target feature points is calculated as the pixel displacement of the target structure in the region of interest. The average value of the target feature point displacement is obtained by averaging the target feature point displacement set.

[0073] It should be understood that, based on the feature points obtained in step (2), the feature points whose tracking displacement errors are greater than the preset tracking displacement error threshold are eliminated, and the remaining feature points are the target feature points. The target feature point displacement set is determined according to the feature point displacement sizes calculated in step (2).

[0074] It should be noted that the parameter adjustment described in step (2) is an adaptive adjustment, and the elimination of feature points with large tracking displacement errors described in step (3) is an adaptive optimization. In the adaptive optimization process, when it is detected that the displacement of a feature point is too large due to environmental factors and exceeds the tracking displacement error threshold set by the feature point displacement test in advance, the feature points with large tracking displacement errors are eliminated to achieve adaptive optimization of the feature points. The adaptive adjustment and optimization process can be adjusted dynamically in real time according to environmental changes, thereby improving the robustness and accuracy of the method described in the present invention.

[0075] Furthermore, based on the feature points obtained in step (2), feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold are eliminated, specifically including: using the Lucas-Kanade optical flow method to monitor the maximum and minimum displacement values ​​of the feature points of the target structure during the entire vibration process, and calculating the difference between the maximum and minimum displacement values ​​of the feature points as the tracking displacement error of the feature point, and then based on the feature points obtained in step (2), feature points whose tracking displacement errors are greater than the preset tracking displacement error threshold are eliminated.

[0076] In this embodiment, after the feature points are adaptively adjusted and optimized, the Lucas-Kanade optical flow method is used to track the feature points, calculate the displacement of the feature points in continuous frames, and continue to determine the tracking displacement error of the feature points based on the displacement, and finally obtain the feature point displacement set. Figure 3 FIG. 1 shows the displacement results of the feature points at the specified position of the method of the present invention under the conditions of illumination change, viewing angle change, noise interference and rain interference, and without any interference. The displacement monitoring results of the region of interest under illumination change are shown in FIG. Figure 3 As shown in (a) in the figure, the displacement monitoring results of the region of interest under the change of viewing angle are as follows Figure 3 As shown in (b), the displacement monitoring results of the region of interest under noise interference are as follows Figure 3 As shown in (c) in the figure, the displacement monitoring results of the area of ​​interest under rain interference are as follows: Figure 3 As shown in (d) in the figure, the displacement monitoring results of the area of ​​interest without interference are as follows: Figure 3 As shown in (e) in the figure. After comparison, it can be seen that the displacement trend is consistent under illumination change, perspective change, and noise interference, while the displacement trend is not obvious under rain interference. The number of feature points on the horizontal axis in the displacement results of the original video and the video under illumination change, perspective change, and noise interference is similar. The number of feature points on the horizontal axis is smaller under rain interference, indicating that more feature points with large errors have been eliminated.

[0077] The method of the present invention is used to monitor the displacement of feature points of the original video and the video under the influence of simulated environmental factors. Finally, the displacement average values ​​of the original video and the video under the influence of simulated environmental factors can be obtained, and the two can be obtained by comparison. Under the condition of illumination change (including bright light and dark light), the root mean square error of the algorithm is 0.015 and 0.002, and the error is very small. This shows that the adaptive feature point recognition algorithm can effectively cope with illumination changes and ensure the accuracy of feature point detection and tracking. At the same time, under the condition of viewing angle change, the root mean square error of the algorithm is 0.011, and the error is extremely small. This shows that the adaptive feature point recognition algorithm can adapt to changes in camera position and angle, and can still accurately detect and track feature points. Then under the condition of noise interference (including strong noise and weak noise), the root mean square error of the algorithm is 0.044 and 0.011. Although the error is slightly higher than the conditions of illumination and viewing angle change, it is still low, and in practical applications, the video will be subjected to noise reduction processing to further reduce the influence of noise, indicating that the adaptive feature point recognition algorithm has a certain robustness to noise changes. Finally, under the conditions of rain interference (including heavy rain and light rain), the adaptive feature point recognition algorithm showed different robustness. Under light rain conditions, the error was extremely low, indicating that the algorithm could cope well with slight rain interference. Under heavy rain conditions, the error increased, but still remained within an acceptable range. Therefore, when using this method for long-term monitoring, it can be used in light rain and heavy rain should be avoided as much as possible, but it can cope with short-term heavy rain.

[0078] Corresponding to the above-mentioned embodiment of the feature point displacement monitoring method based on video and adaptively adjusting to environmental changes, the present invention further provides an embodiment of the feature point displacement monitoring device based on video and adaptively adjusting to environmental changes.

[0079] See also Figure 4 A video-based adaptive environmental change feature point displacement monitoring device provided in an embodiment of the present invention includes one or more processors and a memory, and the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the video-based adaptive environmental change feature point displacement monitoring method in the above embodiment.

[0080] The embodiment of the feature point displacement monitoring device based on video adaptive environmental changes of the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capability in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 4As shown in the figure, it is a hardware structure diagram of any device with data processing capability where the feature point displacement monitoring device based on video adaptive environment change of the present invention is located. Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0081] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0082] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0083] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for monitoring feature point displacement based on video and adaptive environmental changes in the above embodiment is implemented.

[0084] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0085] In summary, the captured video of the designated detection target structure is input into the device, and the method in the above embodiment is executed to output the displacement of the target structure's region of interest.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring feature point displacement based on video and adaptive environmental changes, characterized in that: The following steps are involved: (1) Set the target structure to be monitored, obtain a long-term detection video of the target structure, pre-process the video, and use the Shi-Tomasi corner detection method to detect feature points in the specified area of ​​interest of the initial video frame of the target structure; (2) Based on the feature points preliminarily detected in step (1), the Lucas-Kanade optical flow method is used to calculate the displacement size of the preliminarily detected feature points, and the feature point detection parameters are dynamically adjusted according to the displacement size of the feature points; The Lucas-Kanade optical flow method is used to calculate the displacement of the feature points initially detected based on the feature points initially detected in step (1), specifically including: The Lucas-Kanade optical flow method is used to calculate the displacement of the initially detected feature points between consecutive video frames. Specifically, for the initially detected feature points, the calcOpticalFlowPyrLK function of the OpenCV library is first used to calculate the optical flow, and the new position and new state of the feature points are obtained by updating the video frame; then, the initially detected feature points are screened according to the new state of the feature points, and only the feature points with a state of 1 are retained as valid feature points, and their displacement is calculated according to the new and old positions of the valid feature points; The dynamically adjusting the feature point detection parameters according to the displacement size of the feature point specifically includes: When the displacement value of the feature point is greater than or equal to the preset displacement value threshold of the feature point, the quality threshold and the neighborhood size are increased, and the minimum distance is reduced; When the displacement value of the feature point is less than the preset displacement value threshold of the feature point, the quality threshold and the neighborhood size are reduced, and the minimum distance is increased; (3) Based on the feature points obtained in step (2), the feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold are eliminated, and finally a target feature point displacement set after adaptive adjustment and optimization is obtained, and the average value of the displacements of all target feature points is calculated as the pixel displacement of the target structure in the region of interest; The step of removing the feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold from the feature points obtained in step (2) specifically includes: The Lucas-Kanade optical flow method is used to monitor the maximum and minimum displacements of the feature points of the target structure during the entire vibration process, and the difference between the maximum and minimum displacements of the feature points is calculated as the tracking displacement error of the feature point. Based on the feature points obtained in step (2), the feature points whose tracking displacement errors are greater than a preset tracking displacement error threshold are eliminated.

2. The method for monitoring feature point displacement based on video and adaptive environmental changes according to claim 1, characterized in that: The target structures include node plates of truss beam bridges, corner points of roofs of high-rise buildings, corner points of floor windows of high-rise buildings, intersection points of bridge railings and bridges, anchorage points of cable-stayed bridges, anchorage points of suspension bridge cables, and intersection points of truss arch bridge members.

3. The method for monitoring feature point displacement based on video and adaptive to environmental changes according to claim 1, characterized in that: The preprocessing specifically includes: video capture, cropping and denoising.

4. The method for monitoring feature point displacement based on video and adaptive to environmental changes according to claim 1, characterized in that: The Shi-Tomasi corner point detection method is used to detect feature points in a designated area of ​​interest of an initial video frame of a target structure, specifically comprising: Read the first frame of the video; Set the initial values ​​of the feature point detection parameters and use the Shi-Tomasi corner point detection method to detect feature points in the area of ​​interest of the first frame image; Record the detected feature point coordinates.

5. The method for monitoring feature point displacement based on video and adaptive to environmental changes according to claim 1 or 4, characterized in that: The region of interest is obtained specifically by the following method: Manual annotation: Use image processing tools to manually annotate rectangular bounding boxes to select regions of interest in the target structure; Image segmentation: Automatically detect regions of interest using image segmentation algorithms; Contour Detection: Use edge detection algorithms and contour detection algorithms to extract the contours of target structures to determine the region of interest.

6. A feature point displacement monitoring device based on video and adaptive to environmental changes, comprising one or more processors and a memory, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the feature point displacement monitoring method based on video and adaptive environmental changes as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, it is used to implement the feature point displacement monitoring method based on video and adaptive environmental changes as described in any one of claims 1-5.

Images