Structural displacement visual tracking method, medium and system

Abstract

The application discloses a kind of structural displacement visual tracking method, medium and system, wherein method includes the following steps: obtaining first image, and dividing, to obtain target image block, and establish short-term reference template;Current monitoring image frame is obtained, and is divided, to obtain multiple current monitoring image block, and calculate short-term branch displacement;Confidence index is calculated based on short-term branch displacement, and whether it meets update condition is judged based on confidence index;If yes, then update short-term reference template;Current monitoring overall measurement quality and retention ratio are calculated;Judge whether current monitoring image frame needs to carry out rough recovery;If yes, then trigger whole ROI rough matching, to obtain the absolute displacement of modified;Incremental displacement is calculated;In the process of structural displacement visual tracking, the image obtained in real time can be effectively analyzed to ensure the accuracy of short-term template, and then avoid the error of final measurement result caused by inaccurate short-term template.

Description

Technical Field

[0001] This application relates to the field of visual displacement measurement technology, and in particular to a method, medium and system for visual tracking of structural displacement. Background Technology

[0002] Visual displacement measurement technology is a computer vision-based structural health monitoring system used for non-contact measurement of displacement changes in structures. The system uses a camera to photograph a target mounted on the monitoring object, and employs a specific algorithm to convert the pixel displacement of the target into actual physical displacement, thereby achieving long-term, real-time, and synchronous observation of multi-dimensional displacements such as horizontal and settlement displacements.

[0003] In related technologies, an online template update mechanism is typically used during visual displacement measurement to adapt to changes in the local appearance of the target. Specifically, the online template update method involves directly replacing the template based on the current frame. However, this update method can lead to inaccurate measurement results if the current frame is occluded or mismatched, as the occluded texture or incorrect pose is written into the template. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, one objective of this invention is to propose a structural displacement visual tracking method that can effectively analyze real-time acquired images during the structural displacement visual tracking process to ensure the accuracy of short-term templates, thereby avoiding errors in the final measurement results caused by inaccurate short-term templates.

[0005] In a first aspect, the present invention proposes a visual tracking method for structural displacement, comprising the following steps: acquiring a first frame image, dividing the target region in the first frame image into multiple target image blocks, and establishing a short-term reference template corresponding to each target image block; acquiring a current monitoring image frame, dividing the current monitoring image frame into multiple corresponding current monitoring image blocks, and calculating the short-term branch displacement between the current monitoring image block and the corresponding short-term reference template; calculating the confidence index corresponding to the current monitoring image frame based on the short-term branch displacement corresponding to each current monitoring image block, and judging based on the confidence index. Does the current monitoring image frame meet the update conditions? If so, update the short-term reference template based on the current monitoring image frame. Calculate the overall measurement quality and the retention ratio relative to the historical quality benchmark corresponding to the current monitoring image frame based on the confidence index. Determine whether the current monitoring image frame needs coarse recovery based on the overall measurement quality and the retention ratio. If so, trigger coarse matching of the entire ROI to obtain the corrected absolute displacement. Calculate the difference between the absolute displacement at the current moment and the absolute displacement at the previous moment, and use the difference as the incremental displacement. Use the absolute displacement at the current moment and the incremental displacement as the current monitoring result.

[0006] In some embodiments, calculating the short-term branch displacement between the current monitored image block and the corresponding short-term reference template includes: matching the current monitored image block with the corresponding short-term reference template to obtain the original displacement of the current monitored image block relative to the corresponding short-term reference template; and converting the original displacement to a unified reference coordinate system based on the anchor displacement to obtain the short-term branch displacement.

[0007] In some embodiments, the confidence index includes global confidence, local reliability, visibility, phase position confidence, matching quality, and temporal consistency. Determining whether the current monitored image frame meets the update conditions based on the confidence index includes: Determine whether the following conditions are met simultaneously: global confidence greater than or equal to global confidence threshold, local reliability greater than or equal to local reliability threshold, visibility greater than or equal to visibility threshold, phase position confidence greater than or equal to phase position confidence threshold, matching quality greater than or equal to matching quality threshold, and time consistency less than or equal to time consistency threshold; if so, determine that the current monitoring image frame meets the update conditions.

[0008] In some embodiments, after updating the short-term reference template based on the current monitored image frame, the method further includes: Update the corresponding anchor position displacement based on the current monitored image frame; The short-term reference template is updated according to the following formula: in, This indicates the updated short-term reference template. Indicates the template update coefficient. Indicates the current monitored image block. This indicates a short-term reference template before the update; The anchor displacement is updated according to the following formula: in, This indicates the updated anchor displacement. This indicates the displacement of the current monitored image patch in a unified reference coordinate system. This indicates the anchor position shift before the update.

[0009] In some embodiments, the method further includes: For any short-term reference template, calculate the frame difference between the current time and the template's most recent update time; Determine whether the frame difference is greater than a preset frame difference threshold; If so, calculate the corresponding obsolescence factor and reduce the reliability of the short-term reference template based on the obsolescence factor; The obsolescence factor is calculated using the following formula: in, Indicates the first The obsolescence factor of a short-term reference template Indicates the attenuation coefficient. This represents the frame difference between the current time and the template's most recent update time. This indicates the maximum number of fresh frames allowed.

[0010] In some embodiments, the overall measurement quality is calculated using the following formula: in, Indicates the first The effective quality of the current monitored image block Indicates local reliability. Indicates visibility, Indicates the confidence level of the phase position. Indicates the quality of the match; in, This indicates the overall measurement quality of the current monitored image frame. This represents the summation of the effective quality of each currently monitored image block in the current monitored image frame. This represents the dispersion of the absolute displacement of each currently monitored image block in the current monitored image frame around the median displacement. Indicates the number of currently monitored image blocks. The Gaussian kernel variance parameter represents the displacement dispersion.

[0011] In some embodiments, the retention ratio is calculated using the following formula: in, Indicates the ratio to be maintained. This indicates the overall measurement quality of the current monitored image frame. This represents the historical quality benchmark formed by the exponential moving average of preceding normal frames. This represents a regularization constant to prevent the denominator from being zero. This indicates the updated historical quality benchmark. The exponential moving average decay coefficient represents the historical quality benchmark.

[0012] In some embodiments, determining whether the current monitoring image frame needs coarse restoration based on the overall measurement quality and the hold ratio includes: determining whether the overall measurement quality is less than a preset overall measurement quality threshold or the hold ratio is less than a preset hold ratio threshold; if so, determining that the current monitoring image frame needs coarse restoration.

[0013] Secondly, the present invention proposes a computer-readable storage medium storing a structural displacement visual tracking program thereon, which, when executed by a processor, implements the structural displacement visual tracking method as described above.

[0014] Thirdly, this invention proposes a structural displacement visual tracking system, comprising: an initialization module, which acquires a first frame image, divides the target region in the first frame image to obtain multiple target image blocks, and establishes a short-term reference template corresponding to each target image block; a monitoring module, which acquires a current monitoring image frame, divides the current monitoring image frame to obtain multiple corresponding current monitoring image blocks, and calculates the short-term branch displacement between the current monitoring image block and the corresponding short-term reference template; a first calculation module, which calculates the confidence index corresponding to the current monitoring image frame based on the short-term branch displacement corresponding to each current monitoring image block, and determines whether the current monitoring image frame meets the update conditions based on the confidence index; and an update module, which uses... When the current monitoring image frame meets the update conditions, the short-term reference template is updated based on the current monitoring image frame; a second calculation module is used to calculate the overall measurement quality and the retention ratio relative to the historical quality benchmark corresponding to the current monitoring image frame based on the confidence index; a judgment module is used to determine whether the current monitoring image frame needs to be coarsely restored based on the overall measurement quality and the retention ratio; a coarse matching module is used to trigger coarse matching of the entire ROI when the current monitoring image frame needs to be coarsely restored, so as to obtain the corrected absolute displacement; an output module is used to calculate the difference between the absolute displacement at the current time and the absolute displacement at the previous time, and use the difference as the incremental displacement, and use the absolute displacement at the current time and the incremental displacement as the current monitoring result.

[0015] The beneficial effects of this invention are as follows: According to the structural displacement visual tracking method of this invention, during real-time monitoring, the currently acquired monitoring image frame is divided into multiple current monitoring image blocks; a confidence index of the current monitoring image frame is calculated based on the current monitoring image blocks to effectively evaluate the current monitoring image frame; and the confidence index is used to determine whether to update the short-term reference template based on the current monitoring image frame, thereby ensuring the accuracy of the short-term reference template. Simultaneously, the overall measurement quality and hold ratio of the current monitoring image frame are judged, and when it is determined that the current monitoring image frame needs coarse recovery, coarse matching of the entire ROI is triggered to correct the displacement. This avoids instantaneous deviation of the absolute displacement when a local image block fails entirely, ensuring the accuracy of the measurement results.

[0016] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a visual tracking method for structural displacement according to an embodiment of the present invention. Figure 2 This is a block diagram of a structural displacement visual tracking system according to an embodiment of the present invention. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0019] The structural displacement visual tracking method of the present invention is described below with reference to the accompanying drawings.

[0020] Please see Figure 1 , Figure 1 This is a flowchart illustrating the structural displacement visual tracking method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the visual tracking method for structural displacement includes the following steps: S101, acquire the first frame image, divide the target region in the first frame image to obtain multiple target image blocks, and establish a short-term reference template corresponding to each target image block.

[0021] As an example, firstly, during the initialization phase, the first frame image is acquired; then, the target region in the first frame image is divided into multiple target image blocks; next, each target image block is established as a corresponding short-term reference template. Simultaneously, the short-term reference anchor displacement, template update timestamp, absolute displacement, and velocity state are initialized.

[0022] The initialization process can be recorded as follows: in, Indicates the first Anchor position shift of a short-term reference template. Indicates the first A template update timestamp for a short-term reference template. Indicates absolute displacement. Indicates the speed status.

[0023] S102, acquire the current monitoring image frame, divide the current monitoring image frame into multiple corresponding current monitoring image blocks, and calculate the short-term branch displacement between the current monitoring image block and the corresponding short-term reference template.

[0024] In some embodiments, calculating the short-term branch displacement between the current monitored image block and the corresponding short-term reference template includes: matching the current monitored image block with the corresponding short-term reference template to obtain the original displacement of the current monitored image block relative to the corresponding short-term reference template; and converting the original displacement to a unified reference coordinate system based on the anchor displacement to obtain the short-term branch displacement.

[0025] As an example, firstly, the current monitoring image frame is divided into multiple current monitoring image blocks, just like the first image frame; then, for any current monitoring image block, the current monitoring image block is matched with the corresponding short-term reference template to obtain the original displacement of the current monitoring image block relative to the short-term reference template; then, the original displacement is transformed to a unified reference coordinate system through anchor displacement.

[0026] Specifically, the anchor displacement compensation relationship can be expressed by the following formula: in, Indicates the first The short-term branch displacement of each currently monitored image block in the same reference coordinate system Indicates the original displacement. This indicates the anchor position displacement.

[0027] S103, calculate the confidence index corresponding to the current monitoring image frame based on the short-term branch displacement corresponding to each current monitoring image block, and determine whether the current monitoring image frame meets the update conditions based on the confidence index.

[0028] S104, if so, update the short-term reference template based on the current monitoring image frame.

[0029] In some embodiments, the confidence metrics include global confidence, local reliability, visibility, phase position confidence, matching quality, and time consistency. Determining whether the current monitored image frame meets the update conditions based on the confidence metrics includes: determining whether the following conditions are simultaneously met: global confidence is greater than or equal to a global confidence threshold, local reliability is greater than or equal to a local reliability threshold, visibility is greater than or equal to a visibility threshold, phase position confidence is greater than or equal to a phase position confidence threshold, matching quality is greater than or equal to a matching quality threshold, and time consistency is less than or equal to a time consistency threshold. If so, the current monitored image frame is determined to meet the update conditions.

[0030] In some embodiments, after updating the short-term reference template based on the current monitored image frame, the method further includes: Update the corresponding anchor position displacement based on the current monitoring image frame; The short-term reference template is updated according to the following formula: in, This indicates the updated short-term reference template. Indicates the template update coefficient. Indicates the current monitored image block. This indicates a short-term reference template before the update; The anchor displacement is updated according to the following formula: in, This indicates the updated anchor displacement. This indicates the displacement of the current monitored image patch in a unified reference coordinate system. This indicates the anchor position shift before the update.

[0031] As an example, after acquiring the current monitoring image frame, based on the matching results between the current monitoring image frame and each short-term reference template, the inter-block displacement statistics, and the predicted displacement corresponding to the previous monitoring image frame, the global confidence, local reliability, visibility, phase position confidence, matching quality, and temporal consistency of the current monitoring image frame are calculated. Only when all the above confidence indicators simultaneously meet the conditions are the short-term reference templates updated to avoid erroneous observations contaminating the templates.

[0032] Specifically, the update conditions are whether the following conditions are met simultaneously: global confidence greater than or equal to the global confidence threshold, local reliability greater than or equal to the local reliability threshold, visibility greater than or equal to the visibility threshold, phase position confidence greater than or equal to the phase position confidence threshold, matching quality greater than or equal to the matching quality threshold, and time consistency less than or equal to the time consistency threshold. If so, the current monitored image frame is determined to meet the update conditions, and the short-term reference template is updated.

[0033] Preferably, the above confidence indices are all calculated for the current monitored image frame, assuming the first frame... The normalized matching response map in each current monitored image block is as follows: Its main peak value is The number of pixels within the block is ,but: in, Indicates the quality of the match. Indicates the main peak value. Indicates the confidence level of the phase position. Indicates the second peak value. This represents a regularization constant to prevent the denominator from being zero. Indicates visibility, Indicates the first frame in the current monitored image frame The current monitoring image block satisfies The number of pixels, Indicates local reliability. Indicates short-term branch displacement. This represents the median displacement of the short-term branch displacements of all currently monitored image blocks in the current monitored image frame. Indicates the global confidence level. Indicates time consistency error. This represents the absolute displacement at the current time predicted by the time-series filter based on the state at the previous time step.

[0034] When the update conditions are met, the short-term reference template is updated using an exponential moving average method: in, This indicates the updated short-term reference template. Indicates the template update coefficient. Indicates the current monitored image block. This indicates a short-term reference template before the update; Simultaneously, the anchor displacement is updated according to the following formula: in, This indicates the updated anchor displacement. This indicates the displacement of the current monitored image patch in a unified reference coordinate system. This indicates the anchor position shift before the update.

[0035] In some embodiments, the method further includes, for any short-term reference template, calculating the frame difference between the current time and the template's most recent update time; Determine if the frame difference is greater than a preset frame difference threshold; If so, calculate the corresponding obsolescence factor and reduce the reliability of the short-term reference template based on the obsolescence factor; The obsolescence factor is calculated using the following formula: in, Indicates the first The obsolescence factor of a short-term reference template Indicates the attenuation coefficient. This represents the frame difference between the current time and the template's most recent update time. This indicates the maximum number of fresh frames allowed.

[0036] In other words, if a short-term reference template has not been updated for a long period of time, its reliability will be degraded to reduce the impact of the outdated template on the final result.

[0037] S105, calculates the overall measurement quality and the retention ratio relative to the historical quality benchmark for the current monitoring image frame based on the confidence index.

[0038] In some embodiments, the overall measurement quality is calculated using the following formula: in, Indicates the first The effective quality of the current monitored image block Indicates local reliability. Indicates visibility, Indicates the confidence level of the phase position. Indicates the quality of the match; in, This indicates the overall measurement quality of the current monitored image frame. This represents the summation of the effective quality of each currently monitored image block in the current monitored image frame. This represents the dispersion of the absolute displacement of each currently monitored image block in the current monitored image frame around the median displacement. Indicates the number of currently monitored image blocks. The Gaussian kernel variance parameter represents the displacement dispersion and is used to control the sensitivity of the dispersion to the overall mass decay.

[0039] In some embodiments, the retention ratio is calculated using the following formula: in, Indicates the ratio to be maintained. This indicates the overall measurement quality of the current monitored image frame. This represents the historical quality benchmark formed by the exponential moving average of preceding normal frames. This represents a regularization constant to prevent the denominator from being zero. This indicates the updated historical quality benchmark. The exponential moving average decay coefficient represents the historical quality benchmark.

[0040] As an example, after obtaining the measurement results of all current monitoring image blocks corresponding to the current monitoring image frame, the overall measurement quality corresponding to the current monitoring image frame is first calculated. This overall measurement quality is not simply a comparison of the difference between the current monitoring image frame and the previous frame, but is calculated based on the block-level effective quality and inter-block displacement dispersion of the current monitoring image frame, and is jointly judged with the historical quality benchmark formed by the preceding normal frames. If the overall measurement quality corresponding to the current monitoring image frame is lower than the absolute threshold, or significantly lower than the historical quality benchmark, then coarse restoration is performed on the entire ROI, and the current absolute displacement is pulled back and corrected using the entire block matching results.

[0041] Specifically, the overall measured mass is calculated using the following formula: in, Indicates the first The effective quality of the current monitored image block Indicates local reliability. Indicates visibility, Indicates the confidence level of the phase position. Indicates the quality of the match; in, This indicates the overall measurement quality of the current monitored image frame. This represents the summation of the effective quality of each currently monitored image block in the current monitored image frame. This represents the dispersion of the absolute displacement of each currently monitored image block in the current monitored image frame around the median displacement. Indicates the number of currently monitored image blocks. The Gaussian kernel variance parameter represents the displacement dispersion and is used to control the sensitivity of the dispersion to the overall mass decay.

[0042] The retention ratio is calculated using the following formula: in, This represents the dispersion of the absolute displacement of each currently monitored image block in the current monitored image frame around the median displacement. This represents the median displacement of the short-term branch displacements of all currently monitored image blocks. Indicates the ratio to be maintained. This indicates the overall measurement quality of the current monitored image frame. This represents the historical quality benchmark formed by the exponential moving average of preceding normal frames. This represents a regularization constant to prevent the denominator from being zero. This indicates the updated historical quality benchmark. The exponential moving average decay coefficient represents the historical quality benchmark.

[0043] S106, determine whether the current monitoring image frame needs coarse restoration based on the overall measurement quality and hold ratio.

[0044] S107, if so, trigger coarse matching of the entire ROI to obtain the corrected absolute displacement.

[0045] S108, calculate the difference between the absolute displacement at the current moment and the absolute displacement at the previous moment, and use the difference as the incremental displacement, and use the absolute displacement and incremental displacement at the current moment as the current monitoring result.

[0046] In some embodiments, determining whether the current monitoring image frame needs coarse restoration based on the overall measurement quality and the hold ratio includes: determining whether the overall measurement quality is less than a preset overall measurement quality threshold or the hold ratio is less than a preset hold ratio threshold; if so, determining that the current monitoring image frame needs coarse restoration.

[0047] As an example, if it is determined that the current monitored image frame requires coarse restoration, then coarse matching of the entire ROI is triggered to obtain the coarse restoration displacement of the entire ROI. and its matching score And calculate: in, This represents the confidence level term after the coarse matching score has been normalized. Indicates the lowest matching score. Indicates the highest matching score. This represents the measured displacement calculated by local image block-level fusion at the current moment.

[0048] Specifically, the computer processing procedure for this step is as follows: First, complete the matching of all local image blocks in the current monitored image frame, and then calculate... , and Then based on the threshold and Determine if coarse recovery of the entire ROI is triggered; if triggered, then calculate... , and And perform a pullback correction on the current absolute displacement.

[0049] The coarse recovery fusion coefficient is defined as: in, Indicates time The coarse recovery fusion coefficient, This represents the credibility term after the coarse matching score has been normalized. This represents the normalized bifurcation term of the difference between the coarsely recovered displacement and the currently measured displacement. , , Indicates the fusion parameters, This indicates the maximum fusion limit.

[0050] The absolute displacement after coarse recovery correction is: in, This represents the absolute displacement after coarse recovery correction. This represents the absolute displacement measured from a local image patch. This represents the absolute displacement obtained by coarse recovery of the entire ROI.

[0051] In some embodiments, the method further performs adaptive temporal filtering on the coarsely recovered corrected absolute displacement. This filter adjusts the position and velocity gains in real time based on the predicted residuals and velocity state, ensuring the system maintains good responsiveness under abnormal disturbances and rapid motion scenarios.

[0052] Among them, state prediction and residual are defined as follows: in, Indicates time The predicted absolute displacement, This represents the absolute displacement at the previous moment. This indicates the velocity state at the previous moment. This represents the predicted residual.

[0053] The basic position gain is defined as: in, Indicates the base position gain. Indicates process noise parameters, Indicates effective measurement noise. Indicates the lower limit of the measured noise. This indicates the measured noise parameter.

[0054] The actual position gain is defined as: in, Indicates the actual position gain. This represents the gain amplification term calculated based on the residual amplitude and velocity amplitude. , These represent the lower and upper limits of the position gain, respectively.

[0055] The velocity gain and state update equations are as follows: in, Indicates speed gain. , These represent the lower limit and upper limit of the speed gain, respectively; This represents the filtered absolute displacement output. This indicates that the time-series filter predicts the absolute displacement at the current time based on the state at the previous time step. This represents the product of the position gain and the residual, i.e., the position correction at the current moment. This indicates the updated speed status. This indicates the velocity state at the previous moment. This represents the product of the velocity gain and the residual, i.e., the velocity correction amount at the current moment.

[0056] Then, the incremental displacement, consisting of the current absolute displacement and the difference between the current absolute displacement and the previous absolute displacement, is output for continuous long-term structural displacement tracking: in, This indicates incremental displacement.

[0057] In summary, the structural displacement visual tracking method according to embodiments of the present invention includes the following steps: acquiring a first frame image, dividing the target region in the first frame image to obtain multiple target image blocks, and establishing a short-term reference template corresponding to each target image block; acquiring a current monitoring image frame, dividing the current monitoring image frame to obtain multiple corresponding current monitoring image blocks, and calculating the short-term branch displacement between the current monitoring image block and the corresponding short-term reference template; calculating the confidence index corresponding to the current monitoring image frame based on the short-term branch displacement corresponding to each current monitoring image block, and judging based on the confidence index. Does the current monitoring image frame meet the update conditions? If so, update the short-term reference template based on the current monitoring image frame. Calculate the overall measurement quality and the retention ratio relative to the historical quality benchmark corresponding to the current monitoring image frame based on the confidence index. Determine whether the current monitoring image frame needs coarse recovery based on the overall measurement quality and the retention ratio. If so, trigger coarse matching of the entire ROI to obtain the corrected absolute displacement. Calculate the difference between the absolute displacement at the current moment and the absolute displacement at the previous moment, and use the difference as the incremental displacement. Use the absolute displacement at the current moment and the incremental displacement as the current monitoring result.

[0058] Secondly, the present invention proposes a computer-readable storage medium storing a structural displacement visual tracking program thereon, which, when executed by a processor, implements the structural displacement visual tracking method as described above.

[0059] Thirdly, embodiments of the present invention propose a structural displacement visual tracking system, such as... Figure 2 As shown, the structural displacement visual tracking system includes: an initialization module 10, a monitoring module 20, a first calculation module 30, an update module 40, a second calculation module 50, a judgment module 60, a coarse matching module 70, and an output module 80.

[0060] The initialization module 10 is used to acquire the first frame image, divide the target region in the first frame image to obtain multiple target image blocks, and establish a short-term reference template corresponding to each target image block. The monitoring module 20 is used to acquire the current monitoring image frame, divide the current monitoring image frame to obtain multiple corresponding current monitoring image blocks, and calculate the short-term branch displacement between the current monitoring image block and the corresponding short-term reference template. The first calculation module 30 is used to calculate the confidence index corresponding to the current monitoring image frame based on the short-term branch displacement corresponding to each current monitoring image block, and to determine whether the current monitoring image frame meets the update conditions based on the confidence index. The update module 40 is used to update the short-term reference template based on the current monitoring image frame when the current monitoring image frame meets the update conditions; The second calculation module 50 is used to calculate the overall measurement quality and the retention ratio relative to the historical quality benchmark corresponding to the current monitoring image frame based on the confidence index; The judgment module 60 is used to determine whether the current monitoring image frame needs to be coarsely restored based on the overall measurement quality and the hold ratio; The coarse matching module 70 is used to trigger coarse matching of the entire ROI when the current monitored image frame needs to be coarsely restored, so as to obtain the corrected absolute displacement; The output module 80 is used to calculate the difference between the absolute displacement at the current moment and the absolute displacement at the previous moment, and uses the difference as the incremental displacement, and uses the absolute displacement at the current moment and the incremental displacement as the current monitoring result.

[0061] It should be noted that the above description of the structural displacement visual tracking method also applies to this structural displacement visual tracking system, and will not be repeated here.

[0062] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0063] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0064] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0065] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0066] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0067] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0068] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that they are in indirect contact through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0069] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for visual tracking of structural displacement, characterized in that, Includes the following steps: The first frame image is acquired, and the target region in the first frame image is divided to obtain multiple target image blocks, and a short-term reference template corresponding to each target image block is established. The current monitoring image frame is acquired and divided into multiple corresponding current monitoring image blocks, and the short-term branch displacement between the current monitoring image block and the corresponding short-term reference template is calculated. The confidence index corresponding to the current monitoring image frame is calculated based on the short-term branch displacement corresponding to each current monitoring image block, and the current monitoring image frame is judged to meet the update conditions based on the confidence index. If so, the short-term reference template is updated based on the current monitored image frame; The overall measurement quality and the retention ratio relative to the historical quality benchmark corresponding to the current monitoring image frame are calculated based on the confidence index. Based on the overall measurement quality and the retention ratio, determine whether the current monitored image frame needs coarse restoration; If so, a coarse match of the entire ROI is triggered to obtain the corrected absolute displacement; Calculate the difference between the absolute displacement at the current moment and the absolute displacement at the previous moment, and use the difference as the incremental displacement. Use the absolute displacement at the current moment and the incremental displacement as the current monitoring result.

2. The structural displacement visual tracking method as described in claim 1, characterized in that, Calculating the short-term branch displacement between the current monitored image block and the corresponding short-term reference template includes: The current monitored image block is matched with the corresponding short-term reference template to obtain the original displacement of the current monitored image block relative to the corresponding short-term reference template; The original displacement is transformed to a unified reference coordinate system based on the anchor displacement to obtain the short-term branch displacement.

3. The structural displacement visual tracking method as described in claim 1, characterized in that, The confidence metrics include global confidence, local reliability, visibility, phase position confidence, matching quality, and temporal consistency. Determining whether the current monitored image frame meets the update conditions based on these confidence metrics includes: Determine whether the following conditions are met simultaneously: global confidence is greater than or equal to the global confidence threshold, local reliability is greater than or equal to the local reliability threshold, visibility is greater than or equal to the visibility threshold, phase position confidence is greater than or equal to the phase position confidence threshold, matching quality is greater than or equal to the matching quality threshold, and time consistency is less than or equal to the time consistency threshold. If so, determine that the current monitored image frame meets the update conditions.

4. The structural displacement visual tracking method as described in claim 2, characterized in that, After updating the short-term reference template based on the current monitored image frame, the method further includes: Update the corresponding anchor position displacement based on the current monitored image frame; The short-term reference template is updated according to the following formula: in, This indicates the updated short-term reference template. Indicates the template update coefficient. Indicates the current monitored image block. This indicates a short-term reference template before the update; The anchor displacement is updated according to the following formula: in, This indicates the updated anchor displacement. This indicates the displacement of the current monitored image patch in a unified reference coordinate system. This indicates the anchor position shift before the update.

5. The structural displacement visual tracking method as described in claim 1, characterized in that, Also includes: For any short-term reference template, calculate the frame difference between the current time and the template's most recent update time; Determine whether the frame difference is greater than a preset frame difference threshold; If so, calculate the corresponding obsolescence factor and reduce the reliability of the short-term reference template based on the obsolescence factor; The obsolescence factor is calculated using the following formula: in, Indicates the first The obsolescence factor of a short-term reference template Indicates the attenuation coefficient. This represents the frame difference between the current time and the template's most recent update time. This indicates the maximum number of fresh frames allowed.

6. The structural displacement visual tracking method as described in claim 1, characterized in that, The overall measured quality is calculated using the following formula: in, Indicates the first The effective quality of the current monitored image block Indicates local reliability. Indicates visibility, Indicates the confidence level of the phase position. Indicates the quality of the match; in, This indicates the overall measurement quality of the current monitored image frame. This represents the summation of the effective quality of each currently monitored image block in the current monitored image frame. This represents the dispersion of the absolute displacement of each currently monitored image block in the current monitored image frame around the median displacement. Indicates the number of currently monitored image blocks. The Gaussian kernel variance parameter represents the displacement dispersion.

7. The structural displacement visual tracking method as described in claim 1, characterized in that, The retention ratio is calculated using the following formula: in, Indicates the ratio to be maintained. This indicates the overall measurement quality of the current monitored image frame. This represents the historical quality benchmark formed by the exponential moving average of preceding normal frames. This represents a regularization constant to prevent the denominator from being zero. This indicates the updated historical quality benchmark. The exponential moving average decay coefficient represents the historical quality benchmark.

8. The structural displacement visual tracking method as described in claim 1, characterized in that, Determining whether the current monitored image frame needs coarse restoration based on the overall measurement quality and the hold ratio includes: Determine whether the overall measurement quality is less than a preset overall measurement quality threshold or the retention ratio is less than a preset retention ratio threshold; If so, it is determined that the current monitored image frame needs to be coarsely restored.

9. A computer-readable storage medium, characterized in that, It stores a structural displacement visual tracking program, which, when executed by a processor, implements the structural displacement visual tracking method as described in any one of claims 1-8.

10. A structural displacement visual tracking system, characterized in that, include: An initialization module is used to acquire the first frame image, divide the target region in the first frame image to obtain multiple target image blocks, and establish a short-term reference template corresponding to each target image block. The monitoring module is used to acquire the current monitoring image frame, divide the current monitoring image frame to obtain multiple corresponding current monitoring image blocks, and calculate the short-term branch displacement between the current monitoring image block and the corresponding short-term reference template. The first calculation module is used to calculate the confidence index corresponding to the current monitoring image frame based on the short-term branch displacement corresponding to each current monitoring image block, and to determine whether the current monitoring image frame meets the update conditions based on the confidence index. The update module is used to update the short-term reference template based on the current monitoring image frame when the current monitoring image frame meets the update conditions; The second calculation module is used to calculate the overall measurement quality and the retention ratio relative to the historical quality benchmark corresponding to the current monitoring image frame based on the confidence index. The judgment module is used to determine whether the current monitoring image frame needs to be coarsely restored based on the overall measurement quality and the hold ratio; A coarse matching module is used to trigger coarse matching of the entire ROI when the current monitored image frame needs to be coarsely restored, so as to obtain the corrected absolute displacement. The output module is used to calculate the difference between the absolute displacement at the current moment and the absolute displacement at the previous moment, and to use the difference as the incremental displacement, and to use the absolute displacement at the current moment and the incremental displacement as the current monitoring result.

Images