Target displacement monitoring and tracking method and system
By preprocessing the target image data and initializing the target tracking model, combined with correlation filtering tracking and the CSRT algorithm, the problem of unstable target recognition in the long-distance displacement monitoring system was solved, and high-precision target displacement monitoring and tracking were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTH SURVEYING & MAPPING INSTR
- Filing Date
- 2025-12-10
- Publication Date
- 2026-05-19
Smart Images
Figure CN122066979A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing and target tracking technology, and in particular to a target displacement monitoring and tracking method and system. Background Technology
[0002] In long-distance non-contact displacement monitoring systems, near-infrared cameras are typically used to acquire image sequences containing targets (such as reflective films or circular markers), and the displacement is calculated by observing changes in the target's position within the images. This method offers advantages such as being non-contact and highly accurate, making it suitable for scenarios such as structural health monitoring.
[0003] However, in practical applications, the image acquisition process is easily affected by factors such as changes in lighting, camera shake, and blurred target shapes, leading to unstable recognition results for single-frame images and jumps or drifts in the measurement curve. Traditional methods often employ inter-frame image difference or static recognition, lacking an effective tracking mechanism. Especially when the target moves slowly or the background is complex, problems such as tracking loss and recognition failure can easily occur, affecting the continuity and reliability of the system measurement. Summary of the Invention
[0004] In order to overcome the shortcomings of the existing technology, the purpose of this invention is to propose a target displacement monitoring and tracking method and system, which can realize continuous monitoring and displacement calculation of the target in the image, and is applicable to various near-infrared visual measurement scenarios.
[0005] To achieve the objectives of this invention, the following technical solution is adopted: A target displacement monitoring and tracking method, the method comprising the following steps: Collect target image data of the target area and preprocess the target image data; Initial target region features are extracted from the preprocessed target image data, and a pre-trained target tracking model is initialized based on the initial target region features. Based on the preprocessed target image data, the target tracking model is used to track and locate the target in the target image data in real time to obtain real-time target region features. Based on the real-time target region features, spatial features of the target are extracted, and a displacement sequence of the target over time is constructed based on the spatial features, so as to perform real-time displacement monitoring and tracking of the target based on the target displacement sequence.
[0006] In the above technical solution, by preprocessing the collected target image data, the reliability and practicality of the data can be effectively improved; by extracting the initial target region features based on the preprocessed target image data, the efficiency in the target tracking model initialization process can be effectively improved, so that the initialized target tracking model can efficiently track the target in the target region in real time to obtain the real-time target region features, and then use the real-time target region features to perform real-time displacement monitoring and tracking of the target, effectively improving the accuracy of target displacement monitoring and tracking.
[0007] Furthermore, the process of acquiring target image data of the target area and preprocessing the target image data includes: A near-infrared camera is used to acquire continuous frame target image data of the target area, and the frame rate of the near-infrared camera is set to a fixed frame rate during the acquisition of continuous frame target image data; Each captured image frame is cropped and formatted.
[0008] Furthermore, the process of extracting initial target region features from the preprocessed target image data includes: The preprocessed target image data is initialized by drawing target rectangles to mark the initial target region to be tracked and obtain the initial target region features.
[0009] In the above technical solution, by drawing the target rectangle after initializing the preprocessed target image data, an initial target template with high flexibility and adaptability to diverse target features can be provided during the initialization of the target tracking model, thereby effectively initializing the parameters of the target tracking model.
[0010] Furthermore, the process of using the target tracking model to perform real-time tracking and localization of the target in the target image data includes: Based on the target image data, the target tracking model is used to perform correlation filtering and tracking calculations on the target. Based on the correlation filter tracking calculation results, the CSRT target tracker algorithm is used to calculate the channel reliability and spatial reliability of the real-time target tracking and positioning. Based on the relevant filtering tracking calculation results, channel reliability, and spatial reliability, the target in the target area is tracked and its position updated in real time to obtain the real-time target area features.
[0011] Furthermore, the process of using the target tracking model to perform correlation filtering tracking calculations on the target includes: Obtain the initial frame target image patch from the target image data. And based on the target image patch of the initial frame A correlation filter is used to perform correlation filtering tracking learning on the target. The goal is to minimize the error of the correlation filter in tracking the target. A regularization objective function is constructed, expressed as:
[0012] The regularization objective function is solved using the Fast Fourier Transform, and its expression is:
[0013] in, Represents the correlation filter. This indicates the number of channels in the correlation filter. Represents the target image patch in the initial frame. In the Characteristics of each channel This represents the expected Gaussian response during the learning process of the correlation filter. This represents the weight vector of the correlation filter for the corresponding channel. Indicates cyclic dependency. Represents the regularization coefficient. express conjugate, express conjugate, express conjugate, The asterisk (*) indicates element-wise multiplication, and the asterisk (*) indicates complex conjugation.
[0014] Furthermore, the process of calculating the channel reliability and spatial reliability of real-time target tracking and positioning using the CSRT target tracker algorithm includes: Channel reliability: Calculate the weight of each channel in the correlation filter to obtain the contribution of each channel to tracking. The expression is:
[0015] in, The response graph of channel c; Spatial reliability: The expression for calculating the spatial reliability of real-time target tracking and positioning is:
[0016] Where m represents the binary space mask.
[0017] In the above technical solution, by introducing two mechanisms, channel reliability and spatial reliability, into the target tracking model, the target tracking model can adapt to changes in target shape, partial occlusion, and background interference during the tracking of the target area, and maintain good tracking accuracy.
[0018] Furthermore, the process of extracting spatial features of the target based on the real-time target region features, and constructing a displacement sequence of the target over time based on the spatial features, includes: Based on the real-time target region features output by the target tracking model, the center coordinates of the target region at time t are calculated, as expressed by:
[0019] in, This indicates the pixel coordinates of the top-left corner of the rectangle; This indicates the width and height of the rectangle. Based on the center coordinates of the target region at time t The center coordinates of the target are calculated using the following expression:
[0020]
[0021] Based on the center coordinates of the target at time t , The spatial features of the target are obtained, expressed as:
[0022] Based on the spatial features The expression for constructing the displacement sequence of the target over time is:
[0023] in, Indicates in A time series of moments.
[0024] Furthermore, during the real-time displacement monitoring and tracking of the target based on the target displacement sequence, the target displacement is calculated based on the target center coordinates at previous and subsequent times, expressed as:
[0025] in, This represents the physical displacement of the target along the x-axis. This represents the physical displacement of the target on the y-axis.
[0026] Furthermore, the method also includes: Based on the collected target image data, a confidence-aware mechanism and a position drift tolerance mechanism are used to pre-train the target tracking model, wherein: When the target tracking model encounters matching offset, occlusion interference, or a confidence level lower than a preset threshold during target tracking, the target tracking model is reinitialized based on the spatial characteristics of the target in the current frame, and the reinitialized target tracking model is used to track and locate the target in the target image data in real time.
[0027] A target displacement monitoring and tracking system, the system comprising: The data acquisition module is used to acquire target image data of the target area and preprocess the target image data; The feature extraction module is used to extract initial target region features based on the preprocessed target image data, and initialize the pre-trained target tracking model based on the initial target region features. The tracking and positioning module is used to track and locate the target in the target image data in real time based on the preprocessed target image data and the target tracking model, so as to obtain the real-time target region features. The monitoring and tracking module is used to extract the spatial features of the target based on the real-time target area features, and construct the displacement sequence of the target over time based on the spatial features, so as to perform real-time displacement monitoring and tracking of the target based on the target displacement sequence.
[0028] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a target displacement monitoring and tracking method and system. By preprocessing the acquired target image data, the reliability and practicality of the data can be effectively improved. Based on the preprocessed target image data, the initial target region features are extracted, which can effectively improve the efficiency in the target tracking model initialization process. This enables the initialized target tracking model to efficiently track the target in the target region in real time, obtain real-time target region features, and then perform real-time displacement monitoring and tracking of the target based on the real-time target region features, effectively improving the accuracy of target displacement monitoring and tracking. Attached Figure Description
[0029] Figure 1 A flowchart illustrating the steps of a target displacement monitoring and tracking method provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of a target displacement monitoring and tracking system provided in an embodiment of this application; Figure 3 This is a schematic diagram of target displacement monitoring and tracking in an experiment provided for an embodiment of this application. Detailed Implementation
[0030] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.
[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0032] Example 1: This embodiment provides a target displacement monitoring and tracking method. See [link to relevant documentation] Figure 1 The method includes the following steps: Step S1: Collect target image data of the target area and preprocess the target image data; Step S2: Extract initial target region features from the preprocessed target image data, and initialize the pre-trained target tracking model based on the initial target region features; Step S3: Based on the preprocessed target image data, the target tracking model is used to track and locate the target in the target image data in real time to obtain real-time target region features; Step S4: Extract the spatial features of the target based on the real-time target region features, and construct the displacement sequence of the target over time based on the spatial features, so as to perform real-time displacement monitoring and tracking of the target based on the target displacement sequence.
[0033] In a preferred embodiment, step S1, which involves acquiring target image data of the target area and preprocessing the target image data, includes: A near-infrared camera is used to acquire continuous frame target image data of the target area, and the frame rate of the near-infrared camera is set to a fixed frame rate during the acquisition of continuous frame target image data; Each captured image frame is cropped and formatted.
[0034] Specifically, a near-infrared camera equipped with an 850nm narrowband filter is used to acquire continuous frame images, which contain one or more artificially placed visual targets. Image acquisition can be performed at a fixed frame rate to ensure the continuity and synchronization of the image time series, and each acquired frame image is cropped and formatted uniformly. This step provides the raw data source for subsequent image analysis.
[0035] In a preferred embodiment, step S2, the process of extracting the initial target region features based on the preprocessed target image data, includes: The preprocessed target image data is initialized by drawing target rectangles to mark the initial target region to be tracked and obtain the initial target region features.
[0036] Specifically, users can initialize the target rectangle drawing by drawing the preprocessed target image data. In the system software interface, users can manually draw a rectangle on the current video frame image using the mouse to mark the initial target area to be tracked. This initial target area can provide a flexible and adaptable initial target template during the initialization of the target tracking model, thereby effectively initializing the parameters of the target tracking model. During the initialization of the pre-trained target tracking model based on the initial target region features, after receiving the user's selected region, a pre-trained target tracking model is initialized based on the target template. In this embodiment, the target tracking model uses the CSRT target tracker, but it can also be other neural network models, deep learning models, or other trackers. This invention does not limit the specific tracker.
[0037] In a preferred embodiment, step S3, which involves using the target tracking model to perform real-time tracking and localization of the target in the target image data, includes: Step S31: Based on the target image data, perform correlation filtering and tracking calculations on the target using the target tracking model; Step S32: Based on the correlation filter tracking calculation results, the CSRT target tracker algorithm is used to calculate the channel reliability and spatial reliability of the real-time target tracking and positioning; Step S33: Based on the correlation filter tracking calculation results, channel reliability, and spatial reliability, the target in the target area is tracked and its position updated in real time to obtain the real-time target area features.
[0038] In step S31, the process of performing correlation filtering tracking calculations on the target using the target tracking model includes: Obtain the initial frame target image patch from the target image data. And based on the target image patch of the initial frame A correlation filter is used to perform correlation filtering tracking learning on the target. The goal is to minimize the error of the correlation filter in tracking the target. A regularization objective function is constructed, expressed as:
[0039] In the frequency domain, the regularization objective function is solved using the Fast Fourier Transform, and its expression is:
[0040] in, Represents the correlation filter. This indicates the number of channels in the correlation filter. Represents the target image patch in the initial frame. In the Characteristics of each channel This represents the expected Gaussian response during the learning process of the correlation filter. This represents the weight vector of the correlation filter for the corresponding channel. Indicates cyclic dependency. Represents the regularization coefficient. express conjugate, express conjugate, express conjugate, The asterisk (*) indicates element-wise multiplication, and the asterisk (*) indicates complex conjugation.
[0041] In step S32, the process of calculating the channel reliability and spatial reliability of real-time target tracking and positioning using the CSRT target tracker algorithm includes: Channel reliability: Calculate the weight of each channel in the correlation filter to obtain the contribution of each channel to tracking. The channel reliability for real-time target tracking and positioning is then determined based on this contribution. The expression is:
[0042] in, The response graph of channel c; Spatial reliability: The expression for calculating the spatial reliability of real-time target tracking and positioning is:
[0043] Where m represents a binary space mask, the spatial reliability graph modeling mechanism can adapt to changes in target shape, partial occlusion and background interference, and maintain good tracking accuracy.
[0044] Specifically, the initialization of the CSRT target tracker is achieved by calling the pre-defined `cv2.TrackerCSRT_create()` function in the OpenCV library. This function loads the default parameters of the CSRT algorithm, including modules for HOG / color feature extraction, channel weight calculation, and spatial mask calculation. The system automatically binds the current image frame to the user-selected region, generates a target tracking model, and starts the inter-frame tracking process. This target tracking model uses the initial target region image block selected by the user and extracts multi-channel features (HOG / color features, etc.) from that image block. The target tracking model uses the correlation filter weight vector calculated above. Space reliability mask Channel weight coefficient This enables rapid calculation of the relevant response map in the candidate region in each frame, and the location of the response peak position to determine the new position of the target, thereby achieving inter-frame tracking.
[0045] Understandably, by introducing two mechanisms, channel reliability and spatial reliability, into the target tracking model, the target tracking model can adapt to changes in target shape, partial occlusion, and background interference during the tracking of the target area, and maintain good tracking accuracy.
[0046] In a preferred embodiment, step S4, which involves extracting the spatial features of the target based on the real-time target region features and constructing a displacement sequence of the target over time based on the spatial features, includes: Based on the real-time target region features output by the target tracking model, the center coordinates of the target region at time t are calculated, as expressed by:
[0047] in, This indicates the pixel coordinates of the top-left corner of the rectangle; This indicates the width and height of the rectangle (in pixels). Based on the center coordinates of the target region at time t The center coordinates of the target are calculated using the following expression:
[0048]
[0049] Based on the center coordinates of the target at time t , The spatial features of the target are obtained, expressed as:
[0050] Based on the spatial features The expression for constructing the displacement sequence of the target over time is:
[0051] in, Indicates in A time series of moments.
[0052] Furthermore, during the real-time displacement monitoring and tracking of the target based on the target displacement sequence, the target displacement is calculated based on the target center coordinates at previous and subsequent times, expressed as:
[0053] in, This represents the physical displacement of the target along the x-axis. This represents the physical displacement of the target on the y-axis.
[0054] Specifically, displacement calculation: To obtain the target's displacement during the monitoring process, a reference frame needs to be selected. center point As the initial position, the pixel displacement at each moment is:
[0055]
[0056] If the conversion factor from pixel to physical distance is known , (Unit: mm / px), then the physical displacement is:
[0057]
[0058] The total displacement amplitude can be calculated using Euclidean distance:
[0059] Understandably, binding the center point position with a timestamp to construct a high-time-precision displacement sequence is used for remote structural response monitoring. This high-time-precision displacement sequence is constructed by recording the corresponding timestamp using a clock when acquiring the target center point position for each frame. The constructed displacement sequence is as follows: This sequence ensures a strict correspondence between displacement data and the time axis; the aforementioned remote structural response monitoring, when the system is deployed in a remote monitoring environment, can... The data is transmitted to the monitoring center via wireless network for real-time display.
[0060] In a preferred embodiment, the method further includes: Step S5: Based on the collected target image data, the target tracking model is pre-trained using a confidence-aware mechanism and a position drift tolerance mechanism, wherein: When the target tracking model encounters matching offset, occlusion interference, or a confidence level lower than a preset threshold during target tracking, the target tracking model is reinitialized based on the spatial characteristics of the target in the current frame, and the reinitialized target tracking model is used to track and locate the target in the target image data in real time.
[0061] Specifically, during continuous acquisition of image frame sequences, the target tracking model predicts the position of the current frame based on the target information of the previous frame, thereby continuously iteratively updating the parameters of the target tracking model. If matching offset, occlusion interference, or confidence level below a preset threshold occurs during tracking, the tracker will be reinitialized based on the current center coordinates to restore the accuracy of the target tracking model and the stability of the system. This update strategy, based on a confidence perception mechanism and position drift tolerance judgment, can effectively enhance the system's fault tolerance and automatic recovery capability, avoiding tracking interruption or target loss.
[0062] In this embodiment, by preprocessing the acquired target image data, the reliability and practicality of the data can be effectively improved. By extracting the initial target region features from the preprocessed target image data, the efficiency in the target tracking model initialization process can be effectively improved. This enables the initialized target tracking model to efficiently track the target in the target region in real time, obtain the real-time target region features, and then use the real-time target region features to perform real-time displacement monitoring and tracking of the target, effectively improving the accuracy of target displacement monitoring and tracking.
[0063] Example 2: This embodiment provides a target displacement monitoring and tracking system. (See also...) Figure 2 The system includes: The data acquisition module is used to acquire target image data of the target area and preprocess the target image data; The feature extraction module is used to extract initial target region features based on the preprocessed target image data, and initialize the pre-trained target tracking model based on the initial target region features. The tracking and positioning module is used to track and locate the target in the target image data in real time based on the preprocessed target image data and the target tracking model, so as to obtain the real-time target region features. The monitoring and tracking module is used to extract the spatial features of the target based on the real-time target area features, and construct the displacement sequence of the target over time based on the spatial features, so as to perform real-time displacement monitoring and tracking of the target based on the target displacement sequence.
[0064] Example 3: This embodiment provides corresponding instance data based on the method described in Embodiment 1 and the system described in Embodiment 2, as follows: See Figure 3 In the selected target area, a near-infrared industrial camera equipped with an 850 nm narrowband filter was installed on site to monitor the displacement of the reflective target / near-infrared target lamp.
[0065] The camera is fixed and connected to the monitoring computer via Ethernet to acquire real-time near-infrared target image streams. After the monitoring interface is started, the operator selects the target area in the real-time image using the mouse, which serves as the initial target template. The system calls the cv2.TrackerCSRT_create() interface to initialize the tracking model using the user-selected template, starts the CSRT tracker, and begins updating the target position frame by frame. In each frame, the tracker returns the target rectangle coordinates (x, y, w, h), the system calculates the center point (x + w / 2, y + h / 2), and combines it with the current timestamp to generate displacement monitoring data points. The tracker maintains high-precision tracking even when the target has slight occlusion, changes in background light, or shape deformation. The system continuously records the center coordinate time series, providing input data for subsequent displacement calculation and structural response analysis. The displacement calculation uses the Euclidean distance of the total displacement amplitude. calculate.
[0066] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.
Claims
1. A target displacement monitoring and tracking method, characterized in that, The method includes the following steps: Collect target image data of the target area and preprocess the target image data; Initial target region features are extracted from the preprocessed target image data, and a pre-trained target tracking model is initialized based on the initial target region features. Based on the preprocessed target image data, the target tracking model is used to track and locate the target in the target image data in real time to obtain real-time target region features. Based on the real-time target region features, spatial features of the target are extracted, and a displacement sequence of the target over time is constructed based on the spatial features, so as to perform real-time displacement monitoring and tracking of the target based on the target displacement sequence.
2. The target displacement monitoring and tracking method according to claim 1, characterized in that, The process of acquiring target image data of the target area and preprocessing the target image data includes: A near-infrared camera is used to acquire continuous frame target image data of the target area, and the frame rate of the near-infrared camera is set to a fixed frame rate during the acquisition of continuous frame target image data; Each captured image frame is cropped and formatted.
3. The target displacement monitoring and tracking method according to claim 1, characterized in that, The process of extracting initial target region features from preprocessed target image data includes: The preprocessed target image data is initialized by drawing target rectangles to mark the initial target region to be tracked and obtain the initial target region features.
4. The target displacement monitoring and tracking method according to claim 3, characterized in that, The process of using the target tracking model to perform real-time tracking and localization of the target in the target image data includes: Based on the target image data, the target tracking model is used to perform correlation filtering and tracking calculations on the target. Based on the correlation filter tracking calculation results, the CSRT target tracker algorithm is used to calculate the channel reliability and spatial reliability of the real-time target tracking and positioning. Based on the relevant filtering tracking calculation results, channel reliability, and spatial reliability, the target in the target area is tracked and its position updated in real time to obtain the real-time target area features.
5. The target displacement monitoring and tracking method according to claim 4, characterized in that, The process of using the target tracking model to perform correlation filtering tracking calculations on the target includes: Obtain the initial frame target image patch from the target image data. And based on the target image patch of the initial frame A correlation filter is used to perform correlation filtering tracking learning on the target. The goal is to minimize the error of the correlation filter in tracking the target. A regularization objective function is constructed, expressed as: The regularization objective function is solved using the Fast Fourier Transform, and its expression is: in, Represents the correlation filter. This indicates the number of channels in the correlation filter. Represents the target image patch in the initial frame. In the Characteristics of each channel This represents the expected Gaussian response during the learning process of the correlation filter. This represents the weight vector of the correlation filter for the corresponding channel. Indicates cyclic dependency. Represents the regularization coefficient. express conjugate, express conjugate, express conjugate, The asterisk (*) indicates element-wise multiplication, and the asterisk (*) indicates complex conjugation.
6. The target displacement monitoring and tracking method according to claim 5, characterized in that, The process of calculating the channel reliability and spatial reliability of real-time target tracking and positioning using the CSRT target tracker algorithm includes: Channel reliability: Calculate the weight of each channel in the correlation filter to obtain the contribution of each channel to tracking. The expression is: in, The response graph of channel c; Spatial reliability: The expression for calculating the spatial reliability of real-time target tracking and positioning is: Where m represents the binary space mask.
7. The target displacement monitoring and tracking method according to claim 6, characterized in that, The process of extracting spatial features of the target based on the real-time target region features, and constructing a displacement sequence of the target over time based on the spatial features, includes: Based on the real-time target region features output by the target tracking model, the center coordinates of the target region at time t are calculated, as expressed by: in, This indicates the pixel coordinates of the top-left corner of the rectangle; This indicates the width and height of the rectangle. Based on the center coordinates of the target region at time t The center coordinates of the target are calculated using the following expression: Based on the center coordinates of the target at time t , The spatial features of the target are obtained, expressed as: Based on the spatial features The expression for constructing the displacement sequence of the target over time is: in, Indicates in A time series of moments.
8. The target displacement monitoring and tracking method according to claim 7, characterized in that, During the real-time displacement monitoring and tracking of the target based on the target displacement sequence, the target displacement is calculated based on the target center coordinates at previous and subsequent times, expressed as: in, This represents the physical displacement of the target along the x-axis. This represents the physical displacement of the target on the y-axis.
9. The target displacement monitoring and tracking method according to claim 2, characterized in that, The method further includes: Based on the collected target image data, a confidence-aware mechanism and a position drift tolerance mechanism are used to pre-train the target tracking model, wherein: When the target tracking model encounters matching offset, occlusion interference, or a confidence level lower than a preset threshold during target tracking, the target tracking model is reinitialized based on the spatial characteristics of the target in the current frame, and the reinitialized target tracking model is used to track and locate the target in the target image data in real time.
10. A target displacement monitoring and tracking system, characterized in that, The system includes: The data acquisition module is used to acquire target image data of the target area and preprocess the target image data; The feature extraction module is used to extract initial target region features based on the preprocessed target image data, and initialize the pre-trained target tracking model based on the initial target region features. The tracking and positioning module is used to track and locate the target in the target image data in real time based on the preprocessed target image data and the target tracking model, so as to obtain the real-time target region features. The monitoring and tracking module is used to extract the spatial features of the target based on the real-time target area features, and construct the displacement sequence of the target over time based on the spatial features, so as to perform real-time displacement monitoring and tracking of the target based on the target displacement sequence.