Structural multi-point dynamic displacement identification method based on improved KCF algorithm
Through the improved KCF algorithm, combined with FHOG features and adaptive learning rate, the difficulty of generalized target recognition and target loss in the existing technology is solved, and high-precision, real-time structural multi-job displacement recognition is achieved, which is suitable for complex engineering applications. and mobile platforms.
Patent Information
- Application Number
- CN202410962401.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-08
AI Technical Summary
Existing computer vision-based displacement measurement technology is difficult to identify in generalized targets and complex contexts, especially traditional passive detection and discriminative target tracking algorithms cannot meet the requirements of real-time and high-precision. KCF algorithms are prone to targets when targets are disturbed. Lost phenomenon.
Using the improved KCF algorithm, by obtaining video frames in real time, extracting FHOG features, combining adaptive learning rate and target response intensity judgment mechanisms, more stable tracking and more accurate displacement recognition are achieved, and the calculation of pixel displacement and real displacement is supported. Convert between physical coordinate system and pixel coordinate system.
It improves the accuracy and real-time performance of displacement recognition, enhances the tracking ability of generalized targets, reduces external interference, and avoids target loss. It is suitable for complex engineering applications and mobile platforms.
Smart Images

Figure CN118918142A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of displacement recognition, and in particular relates to a method, system, device and storage medium for structural multi-point dynamic displacement recognition based on an improved KCF algorithm. Background Art
[0002] Displacement measurement perception technology methods based on computer vision have been emerging in recent years. These methods are based on digital image processing technology and can realize deformation measurement and determine the multi-point deformation information of the structure by using image transmission equipment. This type of technology has the advantages of non-contact, non-destructive and low cost, and is more suitable for actual engineering application scenarios. At present, optical measurement technologies based on computer vision, such as digital image correlation method (DIC) based on grayscale information, particle image velocimetry method (PIV) based on traditional correlation matching, SIFT and SURF based on feature point matching, LK optical flow method based on optical flow, and center detection method based on special geometric features (the Gradient-based Hough Transform, GHT), have been widely used in the engineering field, but these algorithms are suitable for scenes with specific targets or relatively simple backgrounds, and are easily affected by factors such as illumination changes or target deformation, and it is difficult to identify generalized targets.
[0003] The target tracking algorithms that use discriminant methods have significant adaptability and robustness for generalized targets and are more suitable for complex engineering applications. For example, Bolme et al. proposed a tracking algorithm based on the minimum output squared error sum (MOSSE) correlation filter, which significantly improved the calculation speed, but the accuracy and confidence were poor. Henriques et al. proposed a cyclic structured tracking algorithm (CSK), which constructs a large number of training samples and test samples by cyclically shifting the benchmark training samples. The algorithm is used for training classifiers and target detection processes, and achieves fast calculations through the properties of Fourier diagonalization. Based on the further development of CSK, Henriques et al. proposed Kernelized Correlation Filters (KCF), which uses the Histogram of Oriented Gradient (HOG) feature and expands the correlation filter from single channel to multi-channel, significantly improving the target tracking capability. However, the KCF algorithm still loses the target when the target is disturbed. Currently, many scholars have taken improvement measures to address this problem. Researchers are improving the KCF algorithm in the areas of multi-target tracking and accurate recognition to further improve the recognition effect, so that the KCF algorithm can perform efficient tracking tasks in real-time scenarios and mobile devices.
[0004] Based on this, the present invention extracts a structural multi-point dynamic displacement identification method based on an improved KCF algorithm. Summary of the invention
[0005] In order to solve the above problems in the prior art, that is, to solve the problem that the existing traditional passive detection target algorithm cannot perform generalized target tracking and the discriminant-based target tracking algorithm cannot meet the requirements of real-time and high precision, the first aspect of the present invention proposes a structural multi-point dynamic displacement recognition method based on an improved KCF algorithm, the method comprising:
[0006] S100, acquiring an image frame at a current moment in an input video in real time as an input image; the input image contains a target to be structurally multi-point dynamic displacement recognized, i.e., a displacement recognition target;
[0007] S200, obtaining a region of interest of a displacement recognition target in the input image and extracting features;
[0008] S300, the first frame of the input image is an initial frame, a tracker based on an improved KCF algorithm is trained to obtain a distribution diagram of peak response intensity, and then the initial position of the displacement recognition target is determined; subsequently, the features extracted from the input image are input into the trained tracker based on the improved KCF algorithm to obtain a distribution diagram of peak response intensity, and then the position of the displacement recognition target at the current moment is determined;
[0009] S400, calculating the pixel displacement of the displacement recognition target by combining the current position of the displacement recognition target and the initial position;
[0010] S500, determining a calibration factor through a conversion relationship between a physical coordinate system and a pixel coordinate system; and calculating a real displacement of the displacement recognition target by combining the calibration factor and the pixel position.
[0011] In some preferred embodiments, the extracted features are FHOG features.
[0012] In some preferred embodiments, the learning rate update method of the tracker based on the improved KCF algorithm is:
[0013]
[0014] Where η represents the learning rate and f(z) represents the target response strength.
[0015] In some preferred implementations, the tracker template update method of the tracker based on the improved KCF algorithm is:
[0016] M t+1 =(1-η)M t +ηm t
[0017] Among them, M t 、M t+1 Represents the tracker template of frame t and frame t+1, where frame t is the current frame and m t Represents the current frame sequence template.
[0018] In some preferred embodiments, the calibration factor is calculated as follows:
[0019]
[0020] Among them, Cf represents the calibration factor, w represents the pixel size of the object, and x represents the actual size of the object.
[0021] In some preferred implementations, before calculating the pixel displacement, the method further includes:
[0022] If the maximum peak response intensity of the peak response intensity distribution diagram is greater than the first peak response intensity threshold, the maximum peak response intensity is used as a high confidence template, and the tracker based on the improved KCF algorithm is updated in combination with the confidence template update strategy;
[0023] If the maximum peak response intensity is less than or equal to a second peak response intensity threshold, stop updating the tracker based on the improved KCF algorithm, and replace the current template with the maximum peak response intensity with the high confidence template; after replacement, reposition the displacement identification target until the displacement identification target reappears in the input image.
[0024] In some preferred implementations, after calculating the pixel displacement and before calculating the real displacement, the method further includes:
[0025] Perform abnormal value detection on the pixel displacement of the current frame. If it is abnormal, the pixel displacement of the current frame is removed, and the average value of the pixel displacement corresponding to the set frames before and after the current frame is taken as the pixel displacement of the current frame;
[0026] The missing value of pixel displacement of the current frame is detected. If missing, the pixel displacement of the current frame is supplemented with the average value of the pixel displacement corresponding to the set frames before and after the current frame.
[0027] In a second aspect of the present invention, a structural multi-point dynamic displacement identification system based on an improved KCF algorithm is proposed, the system comprising:
[0028] An image acquisition module is configured to acquire an image frame at a current moment in an input video in real time as an input image; the input image contains a target to be structurally multi-point dynamic displacement identified, i.e., a displacement identification target;
[0029] A feature extraction module, configured to obtain a region of interest of a displacement recognition target in the input image and extract features;
[0030] The position determination module is configured to use the first frame of the input image as the initial frame, train a tracker based on the improved KCF algorithm, obtain a distribution map of peak response intensity, and then determine the initial position of the displacement identification target; subsequently, the features extracted from the input image are input into the trained tracker based on the improved KCF algorithm to obtain a distribution map of peak response intensity, and then determine the position of the displacement identification target at the current moment;
[0031] A pixel displacement calculation module, configured to calculate the pixel displacement of the displacement identification target in combination with the current position of the displacement identification target and the initial position;
[0032] The real displacement calculation module is configured to determine the calibration factor through the conversion relationship between the physical coordinate system and the pixel coordinate system; and calculate the real displacement of the displacement identification target in combination with the calibration factor and the pixel position.
[0033] The third aspect of the present invention proposes a structural multi-point dynamic displacement identification device based on an improved KCF algorithm, comprising: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned structural multi-point dynamic displacement identification method based on the improved KCF algorithm.
[0034] In a fourth aspect of the present invention, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned structural multi-point dynamic displacement identification method based on the improved KCF algorithm.
[0035] Beneficial effects of the present invention:
[0036] The present invention improves the accuracy and real-time performance of displacement identification.
[0037] 1) The present invention adopts FHOG feature extraction to replace the traditional HOG feature extraction, thereby improving the ability to extract structural generalization measurement point features, thereby more stably tracking structural measurement points;
[0038] 2) The output layer of the KCF algorithm is improved by using the bilinear interpolation method to enable it to completely calculate the center position of each point in the image sequence, so as to calculate a more accurate target position and realize displacement recognition;
[0039] 3) The present invention adopts an adaptive learning rate method to update the tracker and model, which solves the problem that the original KCF algorithm is sensitive to external interference and is prone to target loss;
[0040] 4) The present invention introduces a target response strength judgment mechanism to solve the problem in the original KCF algorithm that when the target is blocked or suddenly moves out of the search area, the response strength drops to the minimum, the tracker will learn serious wrong information, and even if the target reappears, the tracking task cannot be completed;
[0041] 5) The improvement of the present invention enables the KCF algorithm to realize real-time displacement calculation of generalized targets with high precision. The structural dynamic displacement identification technology of the improved KCF algorithm is lightweight in design and can be used on mobile platforms such as mobile phones and drones. It may play an important role in engineering, high-rise buildings, large-span bridges and other fields; it provides an efficient and accurate solution for engineering applications and improves the displacement acquisition efficiency to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings.
[0043] Figure 1 It is a flow chart of a method for identifying multi-point dynamic displacement of a structure based on an improved KCF algorithm according to an embodiment of the present invention;
[0044] Figure 2 is a schematic diagram of a small lamp centroid extracted by using a grayscale centroid method according to an embodiment of the present invention;
[0045] Figure 3 is a schematic diagram of circle center detection according to an embodiment of the present invention;
[0046] Figure 4 It is a detailed flow chart of a method for identifying multi-point dynamic displacement of a structure based on an improved KCF algorithm according to an embodiment of the present invention;
[0047] Figure 5 It is a schematic diagram of the conversion relationship between physical coordinates and pixel coordinates according to an embodiment of the present invention;
[0048] Figure 6 It is a schematic diagram of a structural multi-point displacement identification system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0049] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0051] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.
[0052] The structural multi-point dynamic displacement identification method based on the improved KCF algorithm of the present invention is as follows: Figure 1 As shown, the following steps are included:
[0053] S100, acquiring an image frame at a current moment in an input video in real time as an input image; the input image contains a target to be structurally multi-point dynamic displacement recognized, i.e., a displacement recognition target;
[0054] S200, obtaining a region of interest of a displacement recognition target in the input image and extracting features;
[0055] S300, the first frame of the input image is an initial frame, a tracker based on an improved KCF algorithm is trained to obtain a distribution diagram of peak response intensity, and then the initial position of the displacement recognition target is determined; subsequently, the features extracted from the input image are input into the trained tracker based on the improved KCF algorithm to obtain a distribution diagram of peak response intensity, and then the position of the displacement recognition target at the current moment is determined;
[0056] S400, calculating the pixel displacement of the displacement recognition target by combining the current position of the displacement recognition target and the initial position;
[0057] S500, determining a calibration factor through a conversion relationship between a physical coordinate system and a pixel coordinate system; and calculating a real displacement of the displacement recognition target by combining the calibration factor and the pixel position.
[0058] In order to more clearly illustrate the structural multi-point dynamic displacement identification method based on the improved KCF algorithm of the present invention, each step in an embodiment of the method of the present invention is described in detail below with reference to the accompanying drawings.
[0059] The existing algorithms for identifying structural dynamic displacements include the algorithms for identifying structural dynamic displacements based on the grayscale centroid method and the algorithms for identifying structural dynamic displacements based on the Hough transform (GHT);
[0060] Among them, Figure 2As shown in the figure, the algorithm for structural dynamic displacement recognition based on the grayscale centroid method has the following disadvantages: ① The algorithm needs to perform binarization calculation on the input image. The selection of the binarization threshold will directly affect the difference in the binarization results. Therefore, the brightness of the tracking area directly affects the accuracy of the algorithm. Therefore, in the use scenario, the brightness of the tracking area should be ensured to be stable or significantly stronger than the surrounding environment before a stable calculation can be performed; ② If the brightness of the tracked area is the same as the surrounding environment, the tracking will fail. Therefore, in the daily tracking process, it is necessary to install an active light source on the target to be measured, which increases the complexity of the tracking process; ③ In the calculation of the binary adaptive threshold, the data of the calculated image grayscale is used. This will cause the same target to have different grayscale centroids extracted under different lighting conditions, resulting in different displacement measurement results and deviations in the results; ④ The target cannot be identified and tracked. The algorithm only relies on the binarization threshold to distinguish the tracked target from the surrounding environment (that is, the image of the tracked target must have obvious differences in grayscale from the surrounding environment), and then uses the center of the grayscale centroid to determine the position of the current target. And in the process of identifying displacement, it is impossible to determine whether the target is lost.
[0061] like Figure 3 As shown in the figure, the algorithm for structural dynamic displacement recognition based on Hough transform (GHT) has the following disadvantages: ① The circle center detection method based on gradient Hough transform requires pasting a circular target on the structure to be measured or the structure to be measured has obvious circular features. The dynamic displacement of the structure can only be calculated after calculating the center coordinates of the circular marker points before and after deformation. Operations such as pasting targets are difficult to a certain extent and cannot meet the rapid detection needs of engineering; ② The GHT method has the problems of long time consumption and poor calculation accuracy. It cannot meet the requirements of rapid displacement recognition under complex backgrounds and cannot track in real time; ③ The GHT method cannot determine whether there is a target loss during the displacement recognition process.
[0062] As can be seen from the above, the existing displacement detection methods based on computer vision are usually passive detection, that is, detection is performed based on the special geometric features and gray value information of the target, such as the center target, the light source in the project, etc. These algorithms are suitable for scenes with specific targets or relatively simple backgrounds, and are easily affected by factors such as illumination changes or target deformation, and it is difficult to identify generalized targets. However, the discriminant target tracking algorithm (such as the KCF algorithm) has significant adaptability and robustness for generalized targets, and can track any object with shape features. This type of algorithm usually first performs sample training on the selected area, generates a tracker, and then tracks. Compared with the above algorithms, it is more proactive and therefore more suitable for complex engineering applications. Taking the KCF algorithm as an example, although the KCF algorithm has certain advantages as a target tracking algorithm, it has problems of insufficient stability and poor accuracy when applied to the multi-point dynamic displacement acquisition of engineering structures. The KCF algorithm has the following shortcomings in engineering applications: ① The original KCF algorithm has a small number of feature extraction modules, poor generalization point feature extraction capabilities, and cannot stably track structural points; ② The original KCF algorithm cannot identify displacement and can only track targets, and the accuracy cannot meet engineering needs; ③ The original KCF algorithm cannot achieve the conversion between the pixel coordinate system and the physical coordinate system; ④ The original KCF algorithm is sensitive to external interference and is prone to target loss. For example, when the target is affected by factors such as lighting changes or occlusion, the algorithm will learn the disturbance factors, causing the tracking results to drift, reducing the robustness of the tracker, resulting in inconsistent tracking results and a certain deviation; ⑤ In the original KCF algorithm, when the target is occluded or suddenly moves out of the search area, the response intensity drops to the minimum, and the tracker will learn seriously wrong information. At this time, even if the target reappears, the tracking task cannot be completed.
[0063] Based on this, the present invention makes certain improvements to the existing discriminative target tracking algorithm KCF, so that it has the ability to stably extract structural target features under complex backgrounds, can be used to calculate the multi-point dynamic displacement response of the structure in real time and with high precision, improves the adaptability and robustness of the algorithm, and is more suitable for complex engineering practical applications.
[0064] The principle of the original KCF method is as follows: the algorithm first extracts the HOG features of the target sample, processes them through the Hanning window, and generates a large number of sample sets x in a circular shift manner. i To train the tracker. Then, the kernel function is used to calculate the maximum similarity response between the area to be tested and the target to be tracked, so as to obtain the target position of the area to be tested, and the area to be tested with the largest similarity is selected as the new target area to be tracked, and the above process is repeated until the end. Specifically:
[0065] In order to enhance the sample set and improve the ability to distinguish the target area from other areas, the algorithm trains the filter in a ridge regression manner.i To construct the target classifier f(x i )=w T x i , thus forming the minimum cost function of the trainer:
[0066] min w ∑ i (f(x i )-y i )+λ‖w‖ 2 (1)
[0067] The optimal solution of the classifier weight coefficient w is w=(X T X+λI) -1 X T y, and then according to the properties of circulant matrix and Fourier transform, the frequency domain solution of equation (1) is obtained:
[0068]
[0069] Among them, X is the sample data matrix; I is the unit matrix; y represents the expected regression matrix corresponding to the sample; ⊙ is the dot product; is the discrete Fourier transform of x; for The complex conjugate of .
[0070] Introducing kernel functions to solve nonlinear problems, The weight coefficient w can be expressed in high-dimensional space as The optimal value of w can be converted into the optimal value of α, and the calculation formula of β is:
[0071] β=(K+λI) -1 (3)
[0072] in, is the sample x i The high-dimensional space representation of i,j = k(x i , x j ) The kernel matrix assembled by α is α. Perform Fourier transform operation on α, and α can be expressed as The output response calculation formula can be expressed as: The peak area of f(z) is the target position to be tracked.
[0073]
[0074] in, is the discrete Fourier transform of α; k xx is the first row vector of the kernel matrix k; F -1 is DFT.
[0075] The present invention improves the KCF algorithm as follows:
[0076] S100, acquiring an image frame at a current moment in an input video in real time as an input image; the input image contains a target to be structurally multi-point dynamic displacement recognized, i.e., a displacement recognition target;
[0077] In this embodiment, before collecting the input video, it is preferred to use Zhang Zhengyou calibration method to determine parameters such as the internal and external parameter matrix of the acquisition device, and then perform video image acquisition on the feature points, perform operations such as dedistortion on the collected image, reduce errors, that is, obtain the internal and external parameter matrix and distortion matrix of the camera, and calibrate the acquisition device; the acquisition device includes a camera, a video camera, etc. After correction, the acquisition device is fixed, and the target of interest (i.e., the displacement measurement target) is photographed (preferably 30 frames in real-time calculation in the present invention, and there is no upper limit on the frame rate in the post-processing calculation mode), and then the image frame at the current moment in the input video is obtained as the input image.
[0078] S200, obtaining a region of interest of a displacement recognition target in the input image and extracting features;
[0079] The original KCF algorithm feature extraction module has a small dimension, poor generalized measurement point feature extraction capability, and cannot stably track structural measurement points. In this embodiment, in order to adapt to complex engineering application environments, the FHOG feature extraction module is used to replace the traditional HOG feature extraction method, and the feature extraction dimension is increased from 9 dimensions to 31 dimensions, thereby improving the ability to extract structural generalized measurement point features, thereby more stably tracking structural measurement points.
[0080] S300, the first frame of the input image is an initial frame, a tracker based on an improved KCF algorithm is trained to obtain a distribution diagram of peak response intensity, and then the initial position of the displacement recognition target is determined; subsequently, the features extracted from the input image are input into the trained tracker based on the improved KCF algorithm to obtain a distribution diagram of peak response intensity, and then the position of the displacement recognition target at the current moment is determined;
[0081] The original KCF algorithm is sensitive to external interference and is prone to target loss. For example, when the target is affected by factors such as lighting changes or occlusion, the algorithm will learn the disturbance factors, causing the tracking results to drift, reducing the robustness of the tracker, and resulting in inconsistent tracking results with certain deviations.
[0082] In this embodiment, the target response strength is proportional to the target confidence. When the response strength is above 0.6, the tracker learns less noise information and can be regarded as a stable tracking state. However, when the target is affected by disturbance factors, the response strength will drop sharply, roughly between 0.3 and 0.6. At this time, the traditional KCF algorithm will still update the tracker and model with a linear interpolation method with a fixed learning rate. The algorithm will learn the disturbance factors, causing the tracking results to drift and reduce the robustness of the tracker. If the target is occluded, that is, the response strength is between 0-0.3, the tracker update will destroy the model and the tracking will fail.
[0083] The present invention adopts an adaptive learning rate method to update the tracker and model. As shown in formulas (5) and (6), when the response strength is greater than 0.6, the learning rate is the preset learning rate; when the response strength is between 0.3 and 0.6, the learning rate is adjusted by linear interpolation; when the response strength is less than 0.3, the learning rate is set to 0. This method increases the tracker's resistance to measurement point deformation and can significantly reduce the impact of noise information on the results, thereby enhancing the tracker's robustness.
[0084] M t+1 =(1-η)M t +ηm t (5)
[0085]
[0086] Where η represents the learning rate, f(z) represents the target response strength, and M t 、M t+1 Represents the tracker template of frame t and frame t+1, where frame t is the current frame or the image frame at the current moment, and m t Represents the current frame sequence template.
[0087] In addition, in the original KCF algorithm, when the target is obscured or suddenly moves out of the search area, the response intensity drops to the minimum, and the tracker will learn seriously erroneous information. At this time, even if the target reappears, the tracking task cannot be completed. In the present invention, a target response intensity judgment mechanism is introduced on the tracking framework. When the response intensity is higher than 0.8, the algorithm will save the template (i.e., if the maximum peak response intensity of the peak response intensity distribution diagram is greater than the first peak response intensity threshold, the maximum peak response intensity is used as a high confidence template, and the tracker based on the improved KCF algorithm is updated in combination with the confidence template update strategy); when the response intensity is between 0 and 0.3, the algorithm sets the learning rate to 0 and expands the search area until the target appears. If the target appears again and the tracker cannot track it, the algorithm will use the previously saved template to perform a repositioning operation to ensure that the tracking continues (i.e., if the maximum peak response intensity is less than or equal to the second peak response intensity threshold, the tracker based on the improved KCF algorithm is stopped from being updated, and the high confidence template replaces the current maximum peak response intensity template; after the replacement, the displacement identification target is repositioned until the displacement identification target reappears in the input image). This improvement enhances the stable tracking capability of the algorithm and can be applied to long-term displacement acquisition tasks.
[0088] S400, calculating the pixel displacement of the displacement recognition target by combining the current position of the displacement recognition target and the initial position;
[0089] The original KCF algorithm cannot identify displacement and can only perform target tracking, and its accuracy cannot meet engineering needs. In this embodiment, the output layer of the algorithm is improved, and the bilinear interpolation method is used to enable it to completely calculate the center position of each point in the image sequence to calculate a more accurate target position.
[0090] Identify the displacement part: Figure 4 As shown, let the time series frame of the video stream be t i , the target position corresponding to each frame is Pos i. After selecting the target in the image of the initial frame, FHOG extracts the target features for tracker training and calculates the initial position of the target Pos 0. Then, in the time series frame t i By detecting the target position Pos i and subtracting it from the reference position Pos0 of the initial frame, the pixel displacement of the target in each time series frame can be calculated, and then the pixel displacement is integrated according to the time series to obtain the pixel displacement time course curve. The dynamic learning rate and outlier detection part will be mentioned in the following questions.
[0091] S500, determining a calibration factor through a conversion relationship between a physical coordinate system and a pixel coordinate system; and calculating a real displacement of the displacement recognition target by combining the calibration factor and the pixel position.
[0092] The original KCF algorithm cannot realize the conversion between the pixel coordinate system and the physical coordinate system. In this embodiment, Figure 5 As shown, the calibration factor is determined by the conversion relationship between the physical coordinate system and the pixel coordinate system to realize the conversion from pixel displacement to physical displacement. The calibration factor is calculated by the ratio of the pixel size of the corresponding object and the actual size of the known object.
[0093]
[0094] Among them, Cf represents the calibration factor, w represents the pixel size of the object, and x represents the actual size of the object.
[0095] The calibration factor and pixel position are combined to calculate the real displacement of the displacement recognition target.
[0096] In addition, before calculating the real displacement, data outlier detection and processing are performed first, specifically: the calculated pixel displacement is subjected to a secondary check, and the check items include missing value detection, outlier detection, etc. When the algorithm detects problems such as frame loss, it will automatically prompt the problematic time series, and supplement the missing items with the average value of the set data before and after (preferably 4 in the present invention); the algorithm will detect whether there are outliers in the displacement sequence that are obviously unrelated to the displacement value (that is, the details exceed the maximum amplitude of the structure's own vibration... not in line with the laws of mechanics, etc.), remove and correct it to the average value of the set data before and after (preferably 4 here), that is, the average value of the pixel displacement corresponding to the set frames before and after the current frame is used as the pixel displacement of the current frame. If it is real-time, the confidence level is between 0-0.3 after an abnormal situation occurs. At this time, the template is replaced, and its displacement is the previous displacement with a higher confidence level (that is, the displacement higher than the set confidence threshold is selected according to actual needs). Although this displacement is not accurate, it is better than anomalies and will not cause the algorithm to crash. If it is not real-time, after the algorithm calculation is completed, it will automatically alarm which frame has a problem and average it before and after.
[0097] In summary, the improved algorithm of the present invention can track and identify the target through the image sequence collected by the image transmission device, calculate the pixel displacement of the target in each time series frame, and convert the pixel displacement into the actual physical displacement through the scaling factor to obtain the actual displacement time history curve of the target. At the same time, the present invention also develops a corresponding software system for structural multi-point dynamic displacement identification, such as Figure 6 Through this system, the changes in structural displacement can be detected in real time, and relevant data can be collected and recorded, so that users can efficiently use the system to perform multi-point displacement identification work and improve the accuracy and work efficiency of displacement identification.
[0098] In addition, in order to more clearly illustrate the present invention, an application example is given, which is as follows:
[0099] A high-precision wire-type displacement meter (KD9050LX, measuring range 0-50mm, external power supply 24VDC, sensitivity 69.800mV / mm) was fixed to the bottom of the structure, and the displacement meter data was collected and stored by the NI PXIe-1082 data acquisition instrument with a collection frequency of 1000Hz. The video recording device was a mobile phone camera (IPhone 13Pro Max), which was horizontally and vertically to the surface to be measured, with a straight-line distance of 1m, and recorded at 1080p / 160fps;
[0100] The selected measuring point is the hook of the cable displacement meter to verify the tracking ability of the algorithm under the generalization target. A hammer is used to apply an impact to the structure fixed at both ends, so that the structure only vibrates in the vertical direction. During the video data collection process, the lens remains stationary, and the cable displacement meter synchronously collects the displacement of the measuring point position.
[0101] In the image, select the ROI area and enter the calibration factor to calculate the displacement time history curve of the structure.
[0102] A structural multi-point dynamic displacement identification system based on an improved KCF algorithm according to a second embodiment of the present invention includes:
[0103] An image acquisition module is configured to acquire an image frame at a current moment in an input video in real time as an input image; the input image contains a target to be structurally multi-point dynamic displacement identified, i.e., a displacement identification target;
[0104] A feature extraction module, configured to obtain a region of interest of a displacement recognition target in the input image and extract features;
[0105] The position determination module is configured to use the first frame of the input image as the initial frame, train a tracker based on the improved KCF algorithm, obtain a distribution map of peak response intensity, and then determine the initial position of the displacement identification target; subsequently, the features extracted from the input image are input into the trained tracker based on the improved KCF algorithm to obtain a distribution map of peak response intensity, and then determine the position of the displacement identification target at the current moment;
[0106] A pixel displacement calculation module, configured to calculate the pixel displacement of the displacement identification target in combination with the current position of the displacement identification target and the initial position;
[0107] The real displacement calculation module is configured to determine the calibration factor through the conversion relationship between the physical coordinate system and the pixel coordinate system; and calculate the real displacement of the displacement identification target in combination with the calibration factor and the pixel position.
[0108] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0109] It should be noted that the structural multi-point dynamic displacement identification system based on the improved KCF algorithm provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.
[0110] A third embodiment of the present invention is a structural multi-point dynamic displacement identification device based on an improved KCF algorithm, comprising: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned structural multi-point dynamic displacement identification method based on the improved KCF algorithm.
[0111] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned structural multi-point dynamic displacement identification method based on the improved KCF algorithm.
[0112] Technicians in the technical field can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the structural multi-point dynamic displacement identification device and computer-readable storage medium based on the improved KCF algorithm described above can refer to the corresponding process in the aforementioned method example and will not be repeated here.
[0113] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.
[0114] The terms "first", "second", "third", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.
[0115] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.
Claims
1. A method for identifying multi-point dynamic displacement of a structure based on an improved KCF algorithm, characterized in that: The method includes: S100, acquiring an image frame at a current moment in an input video in real time as an input image; the input image contains a target to be structurally multi-point dynamic displacement recognized, i.e., a displacement recognition target; S200, obtaining a region of interest of a displacement recognition target in the input image and extracting features; S300, the first frame of the input image is an initial frame, a tracker based on an improved KCF algorithm is trained to obtain a distribution diagram of peak response intensity, and then the initial position of the displacement recognition target is determined; subsequently, the features extracted from the input image are input into the trained tracker based on the improved KCF algorithm to obtain a distribution diagram of peak response intensity, and then the position of the displacement recognition target at the current moment is determined; S400, calculating the pixel displacement of the displacement recognition target by combining the current position of the displacement recognition target and the initial position; S500, determining a calibration factor through a conversion relationship between a physical coordinate system and a pixel coordinate system; and calculating a real displacement of the displacement recognition target by combining the calibration factor and the pixel position.
2. The structural multi-point dynamic displacement identification method based on the improved KCF algorithm according to claim 1 is characterized in that: The extracted features are FHOG features.
3. The structural multi-point dynamic displacement identification method based on the improved KCF algorithm according to claim 1 is characterized in that: The learning rate update method of the tracker based on the improved KCF algorithm is: Where η represents the learning rate and f(z) represents the target response strength.
4. The structural multi-point dynamic displacement identification method based on the improved KCF algorithm according to claim 3 is characterized in that: The tracker template updating method of the tracker based on the improved KCF algorithm is: M t+1 =(1-n)M t +ηm t Among them, M t 、M t+1 Represents the tracker template of frame t and frame t+1, where frame t is the current frame and m t Represents the current frame sequence template.
5. The structural multi-point dynamic displacement identification method based on the improved KCF algorithm according to claim 4 is characterized in that: The calibration factor is calculated as follows: Among them, Cf represents the calibration factor, w represents the pixel size of the object, and x represents the actual size of the object.
6. The structural multi-point dynamic displacement identification method based on the improved KCF algorithm according to claim 1 is characterized in that: Before calculating the pixel displacement, the method further includes: If the maximum peak response intensity of the peak response intensity distribution diagram is greater than the first peak response intensity threshold, the maximum peak response intensity is used as a high confidence template, and the tracker based on the improved KCF algorithm is updated in combination with the confidence template update strategy; If the maximum peak response intensity is less than or equal to a second peak response intensity threshold, stop updating the tracker based on the improved KCF algorithm, and replace the current template with the maximum peak response intensity with the high confidence template; after replacement, reposition the displacement identification target until the displacement identification target reappears in the input image.
7. The structural multi-point dynamic displacement identification method based on the improved KCF algorithm according to claim 1 is characterized in that: After calculating the pixel displacement and before calculating the real displacement, the method further includes: Perform abnormal value detection on the pixel displacement of the current frame. If it is abnormal, the pixel displacement of the current frame is removed, and the average value of the pixel displacement corresponding to the set frames before and after the current frame is taken as the pixel displacement of the current frame; The missing value of pixel displacement of the current frame is detected. If missing, the pixel displacement of the current frame is supplemented with the average value of the pixel displacement corresponding to the set frames before and after the current frame.
8. A structural multi-point dynamic displacement identification system based on an improved KCF algorithm, characterized in that: The system includes: An image acquisition module is configured to acquire an image frame at a current moment in an input video in real time as an input image; the input image contains a target to be structurally multi-point dynamic displacement identified, i.e., a displacement identification target; A feature extraction module, configured to obtain a region of interest of a displacement recognition target in the input image and extract features; The position determination module is configured to use the first frame of the input image as the initial frame, train a tracker based on the improved KCF algorithm, obtain a distribution map of peak response intensity, and then determine the initial position of the displacement identification target; subsequently, the features extracted from the input image are input into the trained tracker based on the improved KCF algorithm to obtain a distribution map of peak response intensity, and then determine the position of the displacement identification target at the current moment; A pixel displacement calculation module, configured to calculate the pixel displacement of the displacement identification target in combination with the current position of the displacement identification target and the initial position; The real displacement calculation module is configured to determine the calibration factor through the conversion relationship between the physical coordinate system and the pixel coordinate system; and calculate the real displacement of the displacement identification target in combination with the calibration factor and the pixel position.
9. A structural multi-point dynamic displacement identification device based on an improved KCF algorithm, characterized in that: include: at least one processor; and a memory communicatively coupled to at least one of the processors; Wherein, the memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the structural multi-point dynamic displacement identification method based on the improved KCF algorithm as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the structural multi-point dynamic displacement identification method based on the improved KCF algorithm as described in any one of claims 1 to 7.
Citation Information
Cited By
Intelligent camera system for three-dimensional modeling, displacement detection and defect identification of bridge structure
CN121691640A
