Bidirectional video extensometer measurement method and system based on multi-algorithm fusion image recognition
By employing a multi-algorithm fusion image recognition method, utilizing both ordinary cameras and innovative algorithms, the stability and accuracy issues of traditional equipment in bidirectional strain measurement under complex environments have been resolved. This has enabled low-cost, highly robust bidirectional strain measurement, applicable to mechanical testing of various materials.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHENGZHOU UNIV
- Filing Date
- 2026-03-02
- Publication Date
- 2026-07-14
AI Technical Summary
Existing technologies struggle to achieve stable and accurate bidirectional strain synchronous measurement in complex and variable testing environments with varying lighting, deformation, and potential obstruction. Furthermore, traditional equipment is costly, complex to operate, and lacks robustness.
A multi-algorithm fusion image recognition method is adopted, including sample marking, feature deep learning, multi-algorithm parallel detection and weighted clustering fusion, multi-hypothesis tracking and adaptive optimization, to achieve bidirectional strain measurement using ordinary cameras and innovative algorithms.
It achieves low-cost, high-precision, and robust bidirectional strain measurement, enabling long-term stable tracking in complex environments, reducing reliance on operators, and supporting high-speed testing requirements.
Smart Images
Figure CN122385301A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of material mechanical property testing technology, specifically relating to a bidirectional video extensometer measurement method and system based on multi-algorithm fusion image recognition. Background Technology
[0002] Accurate strain measurement is crucial in the testing of materials' mechanical properties. Traditional contact extensometers suffer from problems such as damaging specimens, uniaxial measurement only, high cost, complex installation, and poor adaptability to harsh environments such as high temperatures. In recent years, non-contact optical measurement technologies based on digital image correlation (DIC) have been developed, but they typically require expensive specialized equipment and complex calibration, have complex algorithms, poor real-time performance, and are difficult to popularize.
[0003] In the field of computer vision, traditional methods for marker detection and tracking (such as single contour detection and template matching) face problems such as insufficient robustness, poor adaptability, and easy tracking loss in complex industrial scenarios. Although there have been attempts to combine multiple algorithms, most of them are simple superpositions, lacking effective collaborative fusion and adaptive optimization mechanisms, and thus failing to fully leverage the advantages of each algorithm.
[0004] Chinese patent CN201810123456.7 can only achieve unidirectional strain measurement; CN202010987654.3 uses a dual-camera system, which is complex and requires cumbersome calibration. Existing technologies have not yet effectively solved how to use low-cost, general-purpose equipment to achieve stable, accurate, and adaptive bidirectional strain synchronous measurement in complex and variable testing environments with lighting, deformation, and potential obstruction. Summary of the Invention
[0005] To address the aforementioned issues, embodiments of this invention propose a bidirectional video extensometer measurement method and system based on multi-algorithm fusion image recognition. The aim is to achieve high-precision and robust bidirectional strain non-contact measurement of materials at extremely low cost through an innovative multi-algorithm fusion architecture and intelligent tracking strategy.
[0006] The bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition of the present invention includes the following steps:
[0007] S1: Sample marking and initialization: Prepare a high-contrast background on the sample surface, and set at least two black round dots and at least two black square dots as strain measurement marks. Round dots are used to measure longitudinal strain, and square dots are used to measure transverse strain.
[0008] S2: First Frame Feature Deep Learning and Benchmark Library Establishment: Before the experiment begins, capture the first frame image, perform multi-dimensional feature extraction and data augmentation on the marked points, and establish a feature benchmark library;
[0009] S3: Synchronous video acquisition and image preprocessing during the test: Real-time acquisition of video sequences of the specimen deformation process, and preprocessing of each frame of image;
[0010] S4: Parallel detection of multiple algorithms and weighted clustering fusion: After preprocessing each frame of image, at least three independent detection algorithms are run in parallel, and then a two-layer fusion strategy is adopted to obtain the fused coordinates of the marker points;
[0011] S5: Multi-hypothesis tracking and feature matching: For subsequent frames, a multi-hypothesis tracking framework is adopted, which combines Kalman filter prediction and feature benchmark library matching to achieve robust tracking of marker points and recover them based on historical trajectories when tracking is lost;
[0012] S6: Bidirectional strain calculation and Poisson's ratio calculation: Calculate the longitudinal strain based on the change in the spacing between the round dots, calculate the transverse strain based on the change in the spacing between the square dots, and calculate the Poisson's ratio of the material in real time based on the longitudinal and transverse strains;
[0013] S7: Adaptive Optimization and Result Output: Based on real-time performance evaluation, the weights of each detection algorithm and system parameters are dynamically adjusted to achieve closed-loop optimization, synchronously display measurement results and output complete test data.
[0014] The two-layer fusion strategy in S4 includes:
[0015] a. Density clustering layer: The density clustering algorithm is used to cluster the candidate points detected by each algorithm according to their spatial location. Points that are less than the threshold ε are grouped into the same cluster and correspond to a physical marker point.
[0016] b. Weighted decision layer: For each candidate point within a cluster, the decision layer is determined based on the dynamic weights of its source algorithm. The final fused coordinates of the physical markers are calculated by weighting the detection confidence level c.
[0017] The final fusion coordinates are:
[0018]
[0019] In the formula, This indicates the final merged coordinates, which are the final marker point coordinates. Indicates the first Clusters, Let c be the coordinates of the candidate point, and c be the confidence level. For the algorithm The dynamic weight at time t.
[0020] The method for updating the dynamic weights is as follows:
[0021] According to the algorithm Performance rating within the most recent W frames The performance score is updated, taking into account both the utilization and accuracy of the detection results.
[0022]
[0023] in, It represents the number of times the algorithm's result is adopted in the fusion step. This is the total number of tests. It is the average confidence level of the adopted results. As a balance factor, The algorithm is represented by t, which represents the current frame number, i.e., the time step. W represents the sliding window size, which means that the algorithm performance is calculated by considering the data of the most recent W frames. This is used to balance short-term fluctuations and long-term trends and to prevent single-frame anomalies from affecting the weights. For example, W=10 means that the performance score will be calculated using the data of the most recent 10 frames.
[0024] The dynamic weights are updated using the softmax function:
[0025]
[0026] In the formula, For the algorithm The fusion weight at time t+1 represents the importance of the algorithm in weighted clustering fusion. K is the smoothing factor, and K is the total number of algorithms. To score performance, This represents the performance score of the j-th detection algorithm at time t.
[0027] The multiple hypothesis tracking and feature matching in S5 include:
[0028] S501: Establish a motion state vector for each marker point and use a uniform velocity model for Kalman filter prediction;
[0029] S502: Perform multi-dimensional similarity matching between the fused detection points of the current frame and the features in the feature benchmark library;
[0030] S503: Uses the Hungarian algorithm for globally optimal data association;
[0031] S504: Update the multi-hypothesis tracking state based on motion prediction and feature matching results, and select the hypothesis with the highest confidence as the final tracking result;
[0032] S505: When markers are occluded or lost, prediction and recovery are performed based on motion models and historical trajectories.
[0033] The similarity calculation combines spatial distance similarity and feature similarity, and the calculation formula is as follows:
[0034]
[0035] In the formula, To assess overall similarity, Let i be the detection point, and i be the trajectory. , These are the weighting coefficients. For distance tolerance parameters, Let i be the multi-dimensional feature set of trajectory i in the feature benchmark library. For testing points Features It is the predicted position of the existing trajectory i at time t.
[0036] The bidirectional strain calculation and Poisson's ratio solution in S6 include:
[0037] Based on the real-time precise coordinates of the round and square markers, and combined with the pixel-to-physical-dimension conversion relationship obtained in advance through the calibration plate, the longitudinal strain is calculated respectively. and transverse strain :
[0038] Calculate longitudinal strain: ,in The initial dot spacing, This refers to the real-time dot spacing.
[0039] Calculate the transverse strain: ,in The initial spacing between square points. This refers to the real-time square point spacing;
[0040] Calculate Poisson's ratio: .
[0041] The sample marking and initialization in S1 includes:
[0042] Two solid black dots are set on the center line of the vertical direction of the sample. The diameter of the dots is 2-5 mm and the spacing is 20-60 mm.
[0043] Two solid black square dots are set symmetrically in the width direction of the sample. The side length of the square dots is 2-5mm and the spacing is 10-30mm.
[0044] The two square dots are located in the middle of the two round dots, forming a diamond-shaped spatial layout that facilitates automatic identification and verification.
[0045] The system of the present invention, applicable to a bidirectional video extensometer measurement method for multi-algorithm fusion image recognition, comprises:
[0046] Sample preparation and interaction module: used to guide the preparation of sample markings, providing a graphical interface for users to initially view the area of interest and system parameters;
[0047] Synchronous image acquisition module: includes a standard industrial camera, a uniform light source system, and a synchronous controller with the universal testing machine to achieve high-quality synchronous image acquisition;
[0048] The core module of intelligent image processing includes an image preprocessing unit, a multi-algorithm parallel detection unit, a weighted clustering and fusion unit, a feature learning and benchmark library management unit, and a multi-hypothesis tracking unit.
[0049] Strain Calculation and Adaptive Optimization Module: Used to calculate bidirectional strain and Poisson's ratio, and to achieve dynamic closed-loop optimization of algorithm weights and system parameters;
[0050] Data visualization and output module: Enables real-time multi-window display of measurement results, data storage, and test report generation.
[0051] The beneficial effects of this invention are:
[0052] 1. This invention deeply integrates the demand for "two-way video extensometers" in materials testing with cutting-edge "multi-algorithm fusion intelligent detection" technology in the field of computer vision, creating a completely new solution. Utilizing ordinary cameras and innovative algorithms, it achieves two-way strain measurement with an accuracy of 0.1%-0.5%, at a cost that is only a tiny fraction of that of professional non-contact systems.
[0053] 2. The innovative architecture of "multi-algorithm parallel detection + weighted clustering fusion + multi-hypothesis tracking" enables the system to strongly resist complex interferences such as illumination changes, marker deformation, partial occlusion, and image noise. It has been specifically optimized for the marker detection and tracking problem used for strain measurement, ensuring long-term stable tracking and achieving a highly robust and high-precision conversion from image to strain data.
[0054] 3. By establishing an enhanced feature library through deep learning on the first frame and combining continuous performance evaluation and dynamic parameter adjustment, the system possesses adaptive optimization capabilities, reducing reliance on operators. It achieves end-to-end online dynamic optimization from algorithm weights to processing parameters, enabling the system to "intelligently" cope with complex experimental environment changes.
[0055] 4. Simultaneous measurement of longitudinal and transverse strain, direct and real-time calculation of Poisson's ratio, providing more comprehensive data support for materials mechanics analysis. Completely avoids contact with the specimen, simplifying specimen preparation (spraying paint and applying spots), and highly simplifying system calibration and operation procedures. The algorithm has been optimized and designed in parallel, supporting high frame rates (e.g., above 30fps) for real-time processing, meeting the requirements of high-speed testing.
[0056] 5. Applicable to strain measurement of various materials such as metals, plastics, rubber, and composite materials in various mechanical tests (tension, compression, bending), especially suitable for environmental chambers and other scenarios with observation windows. Attached Figure Description
[0057] Figure 1 This is a flowchart of the bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition according to the present invention.
[0058] Figure 2 This is a structural diagram of the bidirectional video extensometer measurement method applicable to multi-algorithm fusion image recognition according to the present invention.
[0059] Figure 3 This is a schematic diagram of sample and system preparation according to an embodiment of the present invention.
[0060] Figure 4 This is a schematic diagram of the initialization and feature learning steps in an embodiment of the present invention.
[0061] Figure 5 This is a schematic diagram of the recognition results according to an embodiment of the present invention.
[0062] Figure 6 This is a distance-frame count curve diagram according to an embodiment of the present invention.
[0063] Figure 7 This is a test report of an embodiment of the present invention.
[0064] Figure 8 This is a comparison chart showing the test results identified using an extensometer and the method of the present invention, respectively, in embodiments of the present invention. Detailed Implementation
[0065] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0066] like Figure 1 As shown, the bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition of the present invention includes the following steps:
[0067] S1: Specimen Marking and Initialization: A high-contrast background is prepared on the specimen surface, such as by spraying white matte paint. At least two black dots and at least two black squares are set as strain measurement markers. The black dots are used to measure longitudinal strain, and the black squares are used to measure transverse strain, forming a specific spatial layout. The user selects the initial region of interest (ROI) for each marker point through the interactive interface, and the system enters learning mode.
[0068] Two solid black dots are set on the center line of the vertical direction of the sample. The diameter of the dots is 2-5 mm and the spacing is 20-60 mm.
[0069] Two solid black square dots are set symmetrically in the width direction of the sample. The side length of the square dots is 2-5mm and the spacing is 10-30mm.
[0070] The two square dots are located in the middle of the two round dots, forming a diamond-shaped spatial layout that facilitates automatic identification and verification.
[0071] S2: Deep learning of first-frame features and establishment of benchmark library;
[0072] Includes the following sub-steps:
[0073] S201: Extract multi-dimensional features of each marker point (user-selected ROI region) from the first frame image before the system captures the experiment, including geometric features (area, roundness, etc.), intensity features, Hu moment (invariant moment) features, and local image template features;
[0074] S202: Perform data augmentation on the extracted features, including rotation transformation, scaling transformation, brightness adjustment and noise addition, to generate variant features;
[0075] S203: Store the original features and enhanced features together in a multi-dimensional feature benchmark library, assign initial weights to each feature, and lay the foundation for subsequent highly robust matching;
[0076] S204: Establish the initial tracking state based on multiple hypothesis tracking.
[0077] S3: Synchronous video acquisition and image preprocessing during the test: Real-time acquisition of video sequences of the specimen deformation process, and preprocessing of each frame of image;
[0078] A standard industrial camera was used to synchronously trigger the universal testing machine, recording the test process video. Each frame of the image underwent preprocessing, including grayscale conversion, CLAHE-based adaptive contrast enhancement, bilateral filtering for noise reduction, and morphological feature enhancement, to optimize image quality.
[0079] Assume that during the experiment, at discrete time steps... (i.e., acquiring the corresponding video frame sequence) Acquire image sequence Pre-set on the sample surface The actual physical location of each marker (including circles and squares) at time [time missing] constituting a state set ,in Let be the coordinates of the i-th point in the image coordinate system (unit: pixels).
[0080] The system objective is: to analyze the observed image sequence In this process, the positions of all markers at each time step are robustly and accurately estimated. Then, strain is calculated based on the change in distance.
[0081] The true state set is the set of the true locations of all marked points at time t. The estimated state set is the set of all marker positions estimated by the system at time t. T represents the total number of image frames acquired during the experiment, and M is the total number of markers (including circles and squares).
[0082] Feature learning and benchmark library management unit in Execute in real time, and build an enhanced feature library for each marker point i. .
[0083] Step 1: Multi-dimensional feature extraction
[0084] Extract geometric feature vectors from the user-specified ROI. Area, perimeter, circularity / rectangularity, Hu (cube) Intensity eigenvector Normalized grayscale histogram, local contrast; template features Normalized, point-centered sub-image patches.
[0085] Step 2: Data Augmentation
[0086] Apply a set of transformations to the original features (such as rotation) Scaling Brightness adjustment Add Gaussian noise ), generate variant feature sets .
[0087] Enhanced / Variant Feature Set:
[0088] Then the complete feature benchmark library:
[0089] in, This concatenates the original features into a vector, which significantly improves the robustness of subsequent matching to deformation and illumination changes. j This represents the j-th data augmentation transformation function. It transforms the input features into augmented features, where j is the index number of the transformation function, j=1,2,...,N (N is the total number of transformation functions). Each T... j Implement a specific enhancement operation.
[0090] S4: Parallel detection of multiple algorithms and weighted clustering fusion: After preprocessing each frame of image, at least three independent detection algorithms are run in parallel to obtain the preliminary candidate positions and confidence scores of each algorithm. Then, a two-layer fusion strategy is adopted to obtain the fused coordinates of the markers.
[0091] Image preprocessing includes grayscale conversion, CLAHE contrast enhancement, bilateral filtering for noise reduction, and morphological feature enhancement.
[0092] At least three detection algorithms are run in parallel, including traditional contour detection, blob detection, and learning-based feature matching detection.
[0093] Two-layer fusion strategies include:
[0094] a. Density clustering layer: Cluster all candidate points detected by the algorithm based on spatial density, grouping points that are geographically close together into one class, corresponding to a physical marker point.
[0095] b. Weighted decision layer: For each point within a cluster, the decision layer is determined based on the dynamic weights of the algorithm from which the decision was made. The final fused coordinates of the marker point in this frame are calculated by weighting the coordinates of the marker point with the confidence level of the current detection (adjusted in real time based on historical performance) and the current detection confidence level. This mechanism intelligently integrates the advantages of multiple algorithms, significantly improving detection accuracy and stability in complex situations.
[0096] Parallel detection using multiple algorithms:
[0097] The system runs K independent detection algorithms in parallel. (For example: For contour detection, For Blob detection, (For learning-based feature matching detection). For the t-th frame image ,algorithm Output a set of candidate point detection results: ,in, Let n be the coordinates of the nth candidate point. This represents the confidence level of the test result.
[0098] In the formula, t represents the time, k is the detection algorithm index, identifying the k-th detection algorithm; n is the candidate point index, representing the n-th candidate point detected by algorithm Ak at the current time; N k Let $k$ be the number of candidate points for algorithm $Ak$, and $k$ be the total number of candidate points detected by algorithm $k$ at the current time. Let be the set of detection results of algorithm Ak at time t.
[0099] Weighted clustering fusion:
[0100] This step aims to select K sets of candidate points. The locations are then merged into M final marker points.
[0101] Step 1: Density clustering to form candidate clusters
[0102] All candidate points Input a density-based clustering algorithm (such as DBSCAN). Set a distance threshold. The distance between them is less than Points grouped into the same cluster indicate that they likely originated from the same physical marker. Assume we obtain L candidate clusters. Each cluster It contains several candidate points from different algorithms.
[0103] Let represent the coordinates of the nth candidate point detected by the k-th detection algorithm at time t; This indicates the confidence level of the corresponding candidate point; and After clustering, the first... Cluster The coordinates and confidence level of the i-th point in the dataset.
[0104] Step 2: Weighted centroid calculation
[0105] For each candidate cluster The corresponding final marker coordinates Calculated by weighted average:
[0106]
[0107] In the formula, This indicates the final merged coordinates, which are the final marker point coordinates. Indicates the first 1 candidate cluster Let c be the coordinates of the candidate point, and c be the confidence level. For the algorithm The dynamic weights at time t (satisfying) This dynamic weight reflects the algorithm For recent performance, the initial value can be set to 1 / K and updated online according to the adaptive mechanism.
[0108] If L=M, then This is the fusion result for this frame. If L≠M (e.g., some markers are not detected by any algorithm), then the multi-hypothesis tracker will perform prediction and completion.
[0109] S5: Multi-hypothesis tracking and feature matching: For subsequent frames, a multi-hypothesis tracking framework is adopted, which combines Kalman filter prediction and feature benchmark library matching to achieve robust tracking of marker points and recover them based on historical trajectories when tracking is lost;
[0110] S501: Establish a motion state vector for each marker point and use a uniform velocity model for Kalman filter prediction;
[0111] Establish a motion state vector for each physical marker point i Includes position and velocity.
[0112] The horizontal and vertical coordinates of the i-th physical marker at time t; The horizontal and vertical velocities of the i-th physical marker at time t.
[0113] A uniform velocity (CV) model is used as the state prediction model:
[0114]
[0115] in, Here is the state transition matrix. This is the inter-frame time interval. The process noise covariance matrix is... The prior state estimate of the i-th marker at time t, which is the predicted value. This is the optimal posterior state estimate for the i-th marker at time t−1. Process noise, also known as system noise, describes the difference between the motion model and the actual motion, including changes in acceleration, external disturbances, and errors caused by model simplification. express It follows a multidimensional Gaussian distribution with mean 0 and covariance Q. It follows a multidimensional Gaussian normal distribution, and Q is the process noise covariance matrix.
[0116] The observation model is:
[0117]
[0118] in, For the observation matrix, To observe the noise covariance matrix, the observed values That is, from the fusion detection results (When there is a match) The observation noise vector is used to describe the random errors introduced during the observation process. express It follows a multidimensional Gaussian distribution with zero mean and covariance matrix R.
[0119] Observations This refers to the detection point after data association. When a match is successfully made with trajectory i, the coordinates of the detection point are taken as the observation value of trajectory i at time t. If trajectory i has no matching detection point at time t, it is considered a missing observation (missed detection), and no observation update is performed in this case. Observation value It does not come directly from the fusion detection result, but from the fusion detection point that matches trajectory i.
[0120] S502: Perform multi-dimensional similarity matching between the fused detection points of the current frame and the features in the feature benchmark library. The similarity calculation combines spatial distance similarity and feature similarity.
[0121] In the MHT framework, each frame maintains multiple hypotheses about data associations and trajectories. A key step is to update the current detection... Predicted location relative to existing trajectory Establish a connection.
[0122] Multi-dimensional feature similarity calculation:
[0123] For testing points Calculate the overall similarity between trajectory i and trajectory i. It not only includes spatial distance, but also integrates the similarity of apparent features from the feature benchmark library:
[0124]
[0125] The first term is spatial location similarity based on the Gaussian function, and the second term... For feature similarity, , Weighting coefficients ( This emphasizes that when spatial predictions are unreliable (such as during high-speed motion), they rely more on apparent features. It is the predicted position of the existing trajectory i at time t. For distance tolerance parameters, For trajectory i, there is a multi-dimensional feature set (including enhanced variants) in the feature benchmark library. From the current testing point Corresponding features extracted from the surrounding area.
[0126] Feature similarity can be specified as:
[0127] ,here, It can be calculated based on the difference in Hu moments. It can be calculated based on normalized cross-correlation (NCC) and cosine similarity. For weights. To maximize the operation, take the maximum value of the expression within the parentheses. For feature benchmark library The m-th eigenvector in the data. Apparent similarity function.
[0128] S503: Uses the Hungarian algorithm for globally optimal data association;
[0129] Construct the cost matrix , of which elements (The negative value is used because the Hungarian algorithm solves for the minimum weight matching). The optimal allocation is solved by the Hungarian algorithm to determine the correspondence between the current detection and the existing trajectory, or hypotheses such as new trajectory or trajectory termination.
[0130] C is the cost matrix, which describes the matching cost between all trajectories and all detection points; M is the number of existing trajectories, representing the total number of marker point trajectories that the system is tracking up to time t-1; L is the number of detection points in the current frame, representing the number of effective detection points obtained by weighted clustering fusion at time t; i is the trajectory index, i=1,2,...,M; For the detection point index, =1,2,...,L; To achieve a comprehensive similarity score, the similarity between trajectory i and the detection point is represented. The overall matching degree at time t, Represents the space of real number matrices.
[0131] S504: Update the multi-hypothesis tracking state based on motion prediction and feature matching results, select the hypothesis with the highest confidence as the final tracking result, and slightly update the feature benchmark library accordingly (e.g., using a low learning rate) so that the system can adapt to the slow deformation of the marker points.
[0132] Hypothesis generation: For each hypothesis from the previous time step, multiple new sub-hypotheses are generated based on the association results of the current frame (including the possibility of successful association, missed detection, new target, etc.).
[0133] Kalman filter update: For successfully associated trajectories, use the corresponding observations. Perform Kalman filtering to update the posterior state estimate. and its covariance :
[0134]
[0135] K t H is the Kalman gain matrix, used to weigh the relative confidence of predicted and observed values. H is the observation matrix, representing the mapping matrix from state to observation. T P is the transpose of the observation matrix, and P represents the covariance matrix of the state estimate, which represents the magnitude of the uncertainty in the state estimate. Let I be the prior estimate covariance matrix, representing the uncertainty measure of the predicted state, and let I be the identity matrix.
[0136] Hypothesis Probability Update and Pruning: Based on the observed likelihood (positively correlated with the quality of association matching) and the prior of the motion model, the probability of each hypothesis is updated using Bayesian rules. The hypothesis with the highest probability is retained. Based on this assumption, pruning is performed to control computational complexity.
[0137] S505: When markers are occluded or lost, prediction and recovery are performed based on motion models and historical trajectories.
[0138] S6: Bidirectional strain calculation and Poisson's ratio calculation: Calculate the longitudinal strain based on the change in the distance between the round dots, calculate the transverse strain based on the change in the distance between the square dots, and calculate the Poisson's ratio of the material in real time based on the longitudinal and transverse strains.
[0139] Based on the real-time precise coordinates of the round and square markers, and combined with the pixel-to-physical-dimension conversion relationship obtained in advance through the calibration plate, the longitudinal strain is calculated respectively. and transverse strain :
[0140] Calculate longitudinal strain: ,in The initial dot spacing, This refers to the real-time dot spacing.
[0141] Calculate the transverse strain: ,in The initial spacing between square points. This refers to the real-time square point spacing;
[0142] Calculate Poisson's ratio: .
[0143] S7: Adaptive Optimization and Result Output: Based on real-time performance evaluation, the weights of each detection algorithm and system parameters are dynamically adjusted to achieve closed-loop optimization, synchronously display measurement results and output complete test data.
[0144] The adaptive optimization module evaluates the performance (detection accuracy, stability) of each detection algorithm in real time within the current frame and dynamically adjusts its weight in the fusion of the next frame. Simultaneously, it adaptively fine-tunes preprocessing parameters based on the overall image quality assessment. Finally, it displays the original video, tracking status, strain-time curve, and Poisson's ratio-time curve in real time, and simultaneously stores and outputs all test data.
[0145] The adaptive optimization module dynamically adjusts key parameters through a closed-loop feedback mechanism.
[0146] Step 1: Algorithm Weights Online updates:
[0147] According to the algorithm Performance rating within the most recent W frames The performance score is updated, taking into account both the utilization and accuracy of the detection results.
[0148]
[0149] in, It represents the number of times the algorithm's result is adopted in the fusion step. This is the total number of tests. It is the average confidence level of the adopted results. As a balance factor, The algorithm is represented by t, which is the current frame number, i.e., the time step, and W is the sliding window size. It means that the algorithm performance is calculated by considering the data of the most recent W frames. This is used to balance short-term fluctuations and long-term trends and to prevent single-frame anomalies from affecting the weights. For example, W=10 means that the performance score will be calculated using the data of the most recent 10 frames.
[0150] The dynamic weights are updated using the softmax function, which introduces a smoothing factor. :
[0151]
[0152] In the formula, K is the smoothing factor, and K is the total number of algorithms. To score performance, For the algorithm The fusion weight at time t+1 (i.e., the t+1th frame) represents the importance of the algorithm in weighted cluster fusion. This represents the performance score of the j-th detection algorithm at time t.
[0153] Step 2: Preprocessing and adaptive detection parameters:
[0154] Surveillance image quality indicators (Average gradient, local contrast). If Below the threshold This automatically enhances preprocessing intensity (e.g., increasing CLAHE's clipLimit and decreasing the spatial standard deviation of the bilateral filter). Simultaneously, based on tracking confidence and feature matching scores, the threshold parameters of the detection algorithm are dynamically adjusted to achieve optimal detection sensitivity that adapts to the environment.
[0155] This invention is capable of self-learning, self-optimization, and collaborative operation, thereby improving the reliability, accuracy, and practicality of measurements in real and complex experimental environments.
[0156] like Figure 2 As shown, the system of the bidirectional video extensometer measurement method for multi-algorithm fusion image recognition of the present invention includes:
[0157] Sample preparation and interaction module: used to guide the preparation of sample markings, providing a graphical interface for users to initialize the region of interest (ROI) and system parameters.
[0158] Synchronous image acquisition module: includes a standard industrial camera, a uniform light source system, and a synchronous controller with the universal testing machine to achieve synchronous acquisition of high-quality images.
[0159] The core module of intelligent image processing includes an image preprocessing unit, a multi-algorithm parallel detection unit, a weighted clustering fusion unit, a feature learning and benchmark library management unit, and a multi-hypothesis tracking unit. The multi-algorithm parallel detection unit integrates at least three detection algorithms, including traditional contour detection, blob detection, and learning-based feature matching detection. The weighted clustering fusion unit employs a two-layer fusion strategy based on density clustering and dynamic weighted decision-making; the multi-hypothesis tracking unit uses a combination of Kalman filter prediction and feature matching, supporting multi-hypothesis generation, data association, and hypothesis pruning.
[0160] The strain calculation and adaptive optimization module is used to calculate bidirectional strain and Poisson's ratio, and to implement dynamic closed-loop optimization of algorithm weights and system parameters. This module enables real-time calculation of bidirectional strain and dynamic parameter adjustment based on performance evaluation.
[0161] Data visualization and output module: Enables real-time multi-window display of measurement results, data storage, and test report generation.
[0162] Example
[0163] Take the tensile test of materials as an example.
[0164] Step 1: Sample and System Preparation
[0165] The test was conducted using white PVC boards. Black dots (vertical) with a diameter of 3mm and black squares (horizontal) with a side length of 3mm were affixed to the gauge length of the sample. The actual distance between the dots was measured to be 51mm, and the actual distance between the squares was 21mm. A Ugreen2K high-definition camera and LED light source were installed. Figure 3 As shown.
[0166] Step 2: Initialization and Feature Learning
[0167] The system software is started, the camera is turned on, and the sample is installed on the universal testing machine to begin the test. To compare the test results, a contact extensometer is installed on the sample, and video is recorded simultaneously with the start of the test. After the test and video recording are completed, frames are extracted from the video at a frequency of 1Hz and stored as images. The user selects four ROIs (Regions of Interest) in the first frame image. The system automatically executes S2 to extract the geometric, strength, Hu moment, and template features of each point, and performs data augmentation to build a feature benchmark library. Figure 4As shown.
[0168] Step 3: Image Processing
[0169] Each frame of the image is processed as follows: Figure 5 As shown:
[0170] 1. Run contour detection, blob detection, and feature matching algorithms in parallel to obtain a set of candidate points.
[0171] 2. Perform density clustering on the candidate points. Assume that three clusters are identified (corresponding to three labeled points, one point may not be detected temporarily).
[0172] 3. Based on the current weights (initial average) and detection confidence of each algorithm, perform a weighted average of the point coordinates within each cluster, and output the fused coordinates of the three labeled points.
[0173] 4. Perform multi-dimensional feature matching between these three points and the feature library, and substitute them into the multi-hypothesis tracker to update the optimal tracking trajectory and predict the position of the fourth point.
[0174] 5. Based on the tracking results, calculate the longitudinal strain (based on round points), transverse strain (based on square points), and Poisson's ratio.
[0175] 6. Evaluate the performance of each algorithm in this frame: If the feature matching algorithm successfully correlates data even under occlusion, its weight is increased; if the contour algorithm produces false detections due to noise, its weight is decreased. The adjusted weights are used for weighted fusion in the next frame.
[0176] Step 4: Output Results
[0177] After the experiment, the data will be output, and a distance-frame count curve will be plotted, as shown below. Figure 6 As shown, a test report containing complete data is automatically generated, such as... Figure 7 As shown in the results, the overall recognition and test results are good when the deformation is small in the early stages.
[0178] Step 5: Result Comparison:
[0179] The results of the extensometer test and the identification test of the present invention were compared, and the results are as follows: Figure 8 As shown, the two match well.
[0180] Although the above embodiments have been shown and described, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Any changes, modifications, substitutions and variations made to the above embodiments by those skilled in the art are within the protection scope of the present invention.
Claims
1. A bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition, characterized in that, Includes the following steps: S1. Specimen Marking and Initialization: Prepare a high-contrast background on the specimen surface, and set at least two black round dots and at least two black square dots as strain measurement marks. Round dots are used to measure longitudinal strain, and square dots are used to measure transverse strain. S2. First Frame Feature Deep Learning and Benchmark Library Establishment: Before the experiment begins, the first frame image is captured, multi-dimensional feature extraction and data augmentation are performed on the marked points, and a feature benchmark library is established. S3. Synchronous video acquisition and image preprocessing during the test: Real-time acquisition of video sequences of the specimen deformation process, and preprocessing of each frame of image; S4. Parallel detection of multiple algorithms and weighted clustering fusion: After preprocessing each frame of image, at least three independent detection algorithms are run in parallel, and then a two-layer fusion strategy is adopted to obtain the fused coordinates of the marker points; S5. Multi-hypothesis tracking and feature matching: For subsequent frames, a multi-hypothesis tracking framework is adopted, which combines Kalman filter prediction and feature benchmark library matching to achieve robust tracking of marker points and recover them based on historical trajectories when tracking is lost; S6. Bidirectional strain calculation and Poisson's ratio calculation: Calculate the longitudinal strain based on the change in the distance between the round dots, calculate the transverse strain based on the change in the distance between the square dots, and calculate the Poisson's ratio of the material in real time based on the longitudinal strain and the transverse strain; S7. Adaptive Optimization and Result Output: Based on real-time performance evaluation, the weights of each detection algorithm and system parameters are dynamically adjusted to achieve closed-loop optimization, synchronously display measurement results and output complete test data.
2. The bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition according to claim 1, characterized in that, The two-layer fusion strategy in S4 includes: a. Density clustering layer: The density clustering algorithm is used to cluster the candidate points detected by each algorithm according to their spatial location. Points that are less than the threshold ε are grouped into the same cluster and correspond to a physical marker point. b. Weighted decision layer: For each candidate point within a cluster, the decision layer is determined based on the dynamic weights of its source algorithm. The final fused coordinates of the physical markers are calculated by weighting the detection confidence level c.
3. The bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition according to claim 2, characterized in that, The final fusion coordinates are: In the formula, This indicates the final merged coordinates, which are the final marker point coordinates. Indicates the first Clusters, Let c be the coordinates of the candidate point, and c be the confidence level. For the algorithm The dynamic weight at time t.
4. The bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition according to claim 3, characterized in that, The method for updating the dynamic weights is as follows: According to the algorithm Performance rating within the most recent W frames The performance score is updated, taking into account both the utilization and accuracy of the detection results. in, It represents the number of times the algorithm's result is adopted in the fusion step. This is the total number of tests. It is the average confidence level of the adopted results. As a balance factor, Let represent the algorithm, t represent time, and W represent the sliding window size.
5. The bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition according to claim 4, characterized in that, The dynamic weights are updated using the softmax function: In the formula, For the algorithm The fusion weight at time t+1 K is the smoothing factor, and K is the total number of algorithms. To score performance, Let represent the performance score of the j-th detection algorithm at time t.
6. The bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition according to claim 1, characterized in that, The multiple hypothesis tracking and feature matching in S5 include: S501. Establish a motion state vector for each marker point and use a uniform velocity model for Kalman filter prediction; S502. Perform multi-dimensional similarity matching between the fused detection points of the current frame and the features in the feature benchmark library; S503. Use the Hungarian algorithm for globally optimal data association; S504. Update the multi-hypothesis tracking state based on motion prediction and feature matching results, and select the hypothesis with the highest confidence as the final tracking result; S505. When markers are occluded or lost, prediction and recovery are performed based on motion models and historical trajectories.
7. The bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition according to claim 6, characterized in that, The similarity calculation combines spatial distance similarity and feature similarity, and the calculation formula is as follows: In the formula, To assess overall similarity, Let i be the detection point, and i be the trajectory. , These are the weighting coefficients. For distance tolerance parameters, Let i be the multi-dimensional feature set of trajectory i in the feature benchmark library. For testing points Features It is the predicted position of the existing trajectory i at time t.
8. The bidirectional video extensometer measurement method based on multi-algorithm fusion image recognition according to claim 1, characterized in that, The bidirectional strain calculation and Poisson's ratio solution in S6 include: Based on the real-time precise coordinates of the round and square markers, and combined with the pixel-to-physical-dimension conversion relationship obtained in advance through the calibration plate, the longitudinal strain is calculated respectively. and transverse strain : Calculate longitudinal strain: ,in The initial dot spacing, This refers to the real-time dot spacing. Calculate the transverse strain: ,in The initial spacing between square points. This refers to the real-time square point spacing; Calculate Poisson's ratio: .
9. The bidirectional video extensometer measurement method and system based on multi-algorithm fusion image recognition according to claim 1, characterized in that, The sample marking and initialization in S1 includes: Two solid black dots are set on the center line of the vertical direction of the sample. The diameter of the dots is 2-5 mm and the spacing is 20-60 mm. Two solid black square dots are set symmetrically in the width direction of the sample. The side length of the square dots is 2-5mm and the spacing is 10-30mm. The two square dots are located in the middle of the two round dots, forming a diamond-shaped spatial layout that facilitates automatic identification and verification.
10. A system applicable to the bidirectional video extensometer measurement method for multi-algorithm fusion image recognition as described in any one of claims 1-9, characterized in that, include: Sample preparation and interaction module: used to guide the preparation of sample markings, providing a graphical interface for users to initially view the area of interest and system parameters; Synchronous image acquisition module: includes a standard industrial camera, a uniform light source system, and a synchronous controller with the universal testing machine to achieve high-quality synchronous image acquisition; The core module of intelligent image processing includes an image preprocessing unit, a multi-algorithm parallel detection unit, a weighted clustering and fusion unit, a feature learning and benchmark library management unit, and a multi-hypothesis tracking unit. Strain Calculation and Adaptive Optimization Module: Used to calculate bidirectional strain and Poisson's ratio, and to achieve dynamic closed-loop optimization of algorithm weights and system parameters; Data visualization and output module: Enables real-time multi-window display of measurement results, data storage, and test report generation.