Impact response spectrum resolving method based on mixed monocular and binocular high-speed video measurement

Abstract

The invention relates to an impact response spectrum resolving method based on mixed monocular and binocular high-speed video measurement, and the method comprises the following steps: carrying out the camera calibration of a high-speed camera, so as to obtain a camera parameter and a scale factor; carrying out stereo matching on an initial static frame of a binocular image obtained by a high-speed camera under a static condition; using a shielding target detection algorithm combined with affine transformation to identify shielding targets in the left and right image sequences; on the basis of a shielding target detection result, image sequence matching is carried out through a shielding condition discrimination strategy, binocular stereo-sequence double matching is carried out on unshielded speckles, and monocular sequence matching is carried out on speckles which are shielded in only one camera visual angle; based on the image sequence matching result, coordinate calculation, coordinate system correction, information fusion, displacement and acceleration calculation and impact response spectrum calculation are sequentially carried out, and an impact response spectrum is obtained. Compared with the prior art, the method has the advantages of being accurate in impact response spectrum calculation result under the shielding condition and the like.

Description

Technical Field

[0001] The invention relates to the technical field of high-speed video measurement, and in particular to an impulse response spectrum solving method based on hybrid monocular and binocular high-speed video measurement. Background Art

[0002] Shock environments are widely used in military and civilian products, such as armor penetration during missile attacks, stage separation during rocket launches, bird strikes during aircraft flight, electronic product drop tests, and car crash tests. Shock response spectrum (SRS) testing is an important means of evaluating the performance and reliability of products under shock. Traditional measurement methods use contact sensors to collect shock signals and obtain shock response spectra to evaluate the degree of damage to products. However, shock environments have the characteristics of high frequency, transient state, and high amplitude, which can easily cause damage to contact sensors. In addition, contact sensors have the disadvantages of difficult installation and single-point measurement. At the same time, when responding to the needs of large-scale measurement, installing a large number of contact sensors will cause changes in the structural characteristics of the target being measured, introducing sensor mass effects. High-speed video measurement technology has the advantages of non-contact, high frame rate, three-dimensional measurement, and large-scale monitoring. It uses high-speed cameras to record the transient changes of the target being measured, and accurately measures the three-dimensional coordinates of the target feature points through computer vision and photogrammetry analysis methods, thereby obtaining the dynamic parameters of the object being measured, providing a new means for shock environment testing.

[0003] However, the application of high-speed video measurement in impact testing also faces certain challenges. Since high-level impact environments often produce obstructions, such as splashes in non-pyrotechnic impact tests and debris generated by explosions in pyrotechnic impact tests, when obstructions appear in the camera's field of view, it will greatly affect the performance and accuracy of the image processing algorithm, resulting in large deviations in the shock response spectrum measurement results. Summary of the invention

[0004] The purpose of the present invention is to provide a method for solving the impact response spectrum based on hybrid monocular and binocular high-speed video measurement in order to solve the problem of measuring the impact response spectrum of a target point under occlusion conditions, taking into account the characteristics of the morphological differences produced when observing the same target using multi-view imaging technology. Since the morphological information of the occluded target cannot be fully captured at a specific moment or when observed from a single perspective, the integration of observation information from different perspectives may obtain a complete dynamic change process. Therefore, the present invention uses a digital image correlation speckle matching method to match targets from different perspectives in time and space, and then combines the occluded target extracted by the target detection algorithm to propose a hybrid monocular and binocular measurement strategy, and adopts a scale correction method based on coordinate system transformation to achieve the reconstruction of the temporal displacement response of the measured target point under occlusion, thereby further obtaining a reliable impact response spectrum.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A method for calculating an impact response spectrum based on hybrid monocular and binocular high-speed video measurement comprises the following steps:

[0007] Camera calibration: Use Zhang Zhengyou's plane target calibration method to calibrate the high-speed camera to obtain camera parameters and scale factors;

[0008] Static image stereo matching: Under static conditions, the initial static frames of binocular images obtained by high-speed cameras are stereo matched using the digital image correlation speckle matching method;

[0009] Occluded object detection: Use an occluded object detection algorithm combined with affine transformation to identify occluded objects in the left and right image sequences;

[0010] Image sequence matching: Based on the occluded target detection results, image sequence matching is performed through the occlusion condition discrimination strategy. For unoccluded speckles, the digital image correlation speckle matching method is used in combination with camera parameters to perform binocular stereo-sequence dual matching in binocular sequence images; for speckles that are occluded in only one camera perspective, the digital image correlation speckle matching method is used in combination with the scale factor to perform monocular sequence matching;

[0011] Shock response spectrum solution: Based on the image sequence matching results, the physical coordinates of the measurement points are reconstructed, and after the coordinate system of the images taken from multiple perspectives is corrected, the corrected coordinates are fused; the displacement of the scattered speckles is determined according to the change of coordinates in the fusion results, and the acceleration is calculated; based on the calculated acceleration, the shock response spectrum is calculated using the shock response spectrum solution algorithm.

[0012] The digital image correlation speckle matching method is specifically:

[0013] The digital image correlation algorithm is used to match the scattered speckles in images collected by different high-speed cameras to obtain the correspondence between the same scattered speckles at different collection times and in different imaging spaces. During the matching process, the first image of a selected camera is always used as the reference image, the region of interest is defined in the reference image, and the size and step size of the matching window are set with a certain pixel coordinate as the center. The image sequence is matched using coarse matching and fine matching in turn to determine the points of the same name.

[0014] The coarse matching uses normalized cross-correlation to obtain integer pixel displacement, as shown in the following formula:

[0015]

[0016] Among them, f m and g mRepresent the average gray value of the reference subset and the target subset, f(x,y) and Represents the coordinates (x, y) and The gray value at , M represents the half step length of the matching window, and C represents the correlation coefficient; the reference subset and the target subset are determined according to the execution object of the digital image correlation speckle matching method;

[0017] Select the area with the highest correlation coefficient and use the corresponding pixel offset as the initial deformation parameter;

[0018] The precise matching uses a first-order deformation function to describe the deformation of the reference area relative to the target area, that is, sub-pixel deformation, according to a preset matching window size. The first-order deformation function is shown in the following formula:

[0019]

[0020] Among them, u and v are the center point P(x 0 ,y 0 ) in the x and y directions, Δx and Δy are the displacement components from the center point P to point P i The distance between (x,y), u x 、u y 、v x and v y represent the displacement gradient, is the two-dimensional coordinate corresponding to the center point P;

[0021] The optimal correlation coefficient is obtained by iterative optimization to solve the fine sub-pixel deformation parameter p = [u, u x ,u y ,v,v x ,v y ].

[0022] The optimal correlation coefficient is obtained by using iterative normalized least squares method, and the criteria of the iterative normalized least squares method are as follows:

[0023]

[0024] The occluded target detection is specifically as follows:

[0025] When extracting the pixel coordinates of the occluded object in the image sequence, the first frame in each camera is used as the reference image;

[0026] Select the unobstructed scattered speckles in the reference image, use the digital image correlation speckle matching method to obtain the corresponding points of the selected scattered speckles in each frame of the image sequence, and establish the transformation relationship between the image sequence and the reference image based on the following affine transformation model:

[0027] x'=Ax+By+tx

[0028] y'=Cx+Dy+t y

[0029] Where (x, y) is the coordinate of the scattered spot in the reference image, (x', y') is the coordinate of the corresponding point in the image sequence, A, B, C and D are the parameters that control rotation, scaling and shearing, t x and t y is the translation amount, and its affine transformation relationship is obtained through at least 3 pairs of coordinate points;

[0030] Performing affine transformation on the image sequence using the affine transformation model to generate an affine transformed image, and obtaining a differential image between the affine transformed image and the original image sequence;

[0031] The occluders in the differential image are extracted using the independent multimodal BGS algorithm in the BGS library to form an occluded pixel set.

[0032] The image sequence matching comprises the following steps:

[0033] Selecting a search window centered on the target point to be measured, wherein the size of the search window is the same as the size of the matching window;

[0034] Execute the occlusion condition discrimination strategy to discriminate the occlusion condition in the image at every moment. When all the discriminations are completed or the early termination condition is reached, perform the corresponding matching according to the discrimination results. Specifically:

[0035] If the point to be measured is blocked in the search window of the image sequence taken by different cameras at a certain moment, the early termination condition is met, the identifier of the point to be measured is set to 0, and the data of the point to be measured is discarded;

[0036] If the point to be measured is not blocked in the search window of the image sequences taken by different cameras at all times, the identifier of the point to be measured is set to 1, and binocular stereo-sequence double matching is performed for the binocular measurement result of the point to be measured. The binocular stereo-sequence double matching is: firstly, sequence matching is performed for the monocular image sequence of a single camera, and after the matching is completed, stereo matching is performed for the binocular image sequences of different cameras;

[0037] If the point to be measured is not blocked in the search window of the image sequence taken by a certain camera at all times, but is blocked in the search window of the image sequence taken by another camera, then monocular sequence matching is performed according to the monocular measurement results of the camera without any blockage, wherein if the camera without any blockage is the left camera, the identifier of the point to be measured is set to 2, and if it is the right camera, the identifier of the point to be measured is set to 3;

[0038] If none of the above conditions are met, and in the entire image sequence of the test point, at any moment, there is occlusion in the search window of the image sequence taken by at most one camera, then the identifier of the test point is set to 4. For the moment when there is no occlusion in the search window of the image sequences taken by different cameras, the binocular measurement results are used to perform binocular stereo-sequence dual matching. For the moment when there is occlusion in the search window of the image sequence taken by one camera, the unoccluded monocular measurement results are used to perform monocular sequence matching.

[0039] The physical coordinates of the reconstructed measurement points are specifically:

[0040] According to the image sequence matching results, a spatial mapping operator is used to reconstruct the actual physical coordinates of the measurement points in the world coordinate system. The spatial mapping operator includes three-dimensional reconstruction and a scale factor. The scale factor is expressed as the ratio of the three-dimensional world coordinates to the two-dimensional image coordinates in the measurement area of ​​the object surface.

[0041] For the scattered spots with marker 1, their three-dimensional coordinates are obtained by three-dimensional reconstruction technology. For the scattered spots with markers 2, 3 and 4, the coordinates in the main impact direction are obtained by calculating the scale factor in the reference image.

[0042] The coordinate system correction of the images shot from multiple perspectives is specifically to correct the world coordinate system w To make a correction:

[0043] The world coordinate system O w The X-axis and Y-axis are parallel to the plane target, the Z-axis is perpendicular to the XOY plane, and the X-axis is assumed to be approximately the main impact direction. The world coordinate system O w The calibration is divided into two steps:

[0044] The first step of calibration is to rotate the XOZ plane around the Y axis so that the X axis is parallel to the object surface. 1 ,Y 1 ,Z 1 ) and (X 2 ,Y 2 ,Z 2 ), calculate the vector (X 1 ,Z 1 ) and (X 2 ,Z 2 ) between which the angle θ is;

[0045] The second step of calibration: The XOY plane is rotated around the Z axis to align the X axis with the main impact direction. The horizontal or vertical straight line on the object structure is obtained using the straight line extraction algorithm, and two three-dimensional points (X 3 ,Y 3 ,Z 3) and (X 4 ,Y 4 ,Z 4 ), the straight line formed by these two points is parallel to the extracted straight line, thus obtaining the vector (X 3 ,Y 3 ) and (X 4 ,Y 4 ) on the XOY plane, the corrected coordinates (X', Y', Z') are expressed as follows:

[0046]

[0047] Among them, (X, Y, Z) is the coordinate before correction.

[0048] The shock response spectrum solution algorithm performs the following steps:

[0049] According to Newton's law, the motion equation of the mass block in the single degree of freedom system is obtained as follows:

[0050]

[0051] Among them, m, c and k are the mass, damping coefficient and stiffness of the SDOF respectively, and f n is its natural frequency, x and y are the response displacement of the mass block and the displacement of the base, respectively. are the velocity response of the mass block and the velocity of the base, is the acceleration response at time t under this impact signal;

[0052] The relative displacement of the mass block relative to the base is expressed as z = xy, and the following is derived:

[0053]

[0054] in, The impact excitation acceleration signal is input to the single degree of freedom system, and the damping coefficient c = 2mξω n , stiffness Then the shock response spectrum relationship equation is derived:

[0055]

[0056] Where ξ is the damping ratio, ω n Represents the natural circular frequency, denoted as ω n =2πf n , f n is the natural frequency;

[0057] The digital recursive filtering method is used to solve the shock response spectrum relationship equation to obtain the relationship between the natural frequency and the shock response peak of the single-degree-of-freedom system, thereby obtaining the shock response spectrum of the complex real physical system.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] In view of the problem that the measured impact response spectrum is unreliable when the target point to be measured is blocked by an occluding target generated by a high-level impact during an impact test, the present invention proposes a hybrid single / binocular high-speed video measurement method to solve the impact response spectrum. A digital image correlation speckle matching method is used to obtain the spatiotemporal correspondence of the measurement point in different camera imaging angles, and then the occluding target detection combined with affine transformation is used to obtain the occluding pixel set. Finally, through a hybrid single / binocular high-speed video measurement strategy and a scale correction method based on coordinate system transformation, the information of the target points of multi-view imaging is fused to restore the dynamic change process of the occluded target point and obtain accurate impact response spectrum results. The reliability of this method is verified by comparing it with the impact response spectrum results obtained based on an accelerometer. The present invention significantly improves the feasibility of applying high-speed video measurement technology when conducting high-level impact tests under occlusion conditions, and has important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flow chart of the method of the present invention;

[0061] Figure 2 It is a schematic diagram of the standard shock response spectrum;

[0062] Figure 3 is a schematic diagram of a left and right camera image sequence in an embodiment;

[0063] Figure 4 is a schematic diagram of occluded target detection results in an embodiment;

[0064] Figure 5 It is a comparison diagram of impulse response spectrum solution results of different measurement methods when scattered speckles are blocked in an embodiment. DETAILED DESCRIPTION

[0065] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0066] It is crucial to accurately analyze the impact response of structures in complex impact environments. The contact sensor method has disadvantages such as difficult installation, single-point measurement, and easy damage during impact. The non-contact high-speed video measurement method has the advantages of high frame rate, three-dimensional measurement and large-scale monitoring, and has a high application potential in impact testing. However, high-level impact tests are usually accompanied by occlusion targets such as splashes, which greatly affects the accuracy of video measurement. In order to solve the problem of missing target dynamic information caused by occlusions during the test, this embodiment is based on the idea of ​​multi-viewpoint sequence image information transmission, and proposes a method for solving the impact response spectrum based on hybrid single / binocular high-speed video measurement. It mainly uses a matching method based on digital image correlation to achieve the temporal and spatial information matching of the target, and then uses a target detection algorithm to distinguish the occlusion of the target point. Finally, the dynamic response process of the target point under occlusion conditions is reconstructed by combining the hybrid single / binocular high-speed video measurement strategy and the scale correction method. The results of the air cannon impact test show that compared with the reference value of the impact response spectrum obtained based on the accelerometer, the impact response spectrum obtained by this method under the condition that the target point is blocked is consistent with the reference result, and the average acceleration error is 12.32%, which verifies the feasibility and effectiveness of the proposed method.

[0067] Specifically, Figure 1 As shown, the method comprises the following steps:

[0068] Step 1) Camera calibration: The high-speed camera is calibrated using Zhang Zhengyou's plane target calibration method to obtain camera parameters (intrinsic parameters, external parameters and distortion parameters) and scale factors.

[0069] Step 2) Stereo matching of static images: Under static conditions, the initial static frames of the binocular images obtained by the high-speed camera are stereo matched using the digital image correlation speckle matching method.

[0070] In this embodiment, there are at least two high-speed cameras. When performing image matching, if there are only two high-speed cameras, the images taken by the two cameras are directly matched. If there are more than two high-speed cameras, image matching is performed between each of them separately, and then the matching results are fused.

[0071] The digital image correlation speckle matching method is specifically as follows: a digital image correlation algorithm is used to match the speckles in images acquired by different high-speed cameras to obtain the corresponding relationship between the same speckle in different acquisition times and different imaging spaces; during the matching process, the first image of a selected camera (in this embodiment, it is assumed to be the left camera) is always used as a reference image, a region of interest (ROI) is defined in the reference image, and the size and step size of the matching window are set with a certain pixel coordinate as the center, and the image sequence is matched by coarse matching and fine matching in turn to determine the same-name points.

[0072] In the coarse matching, the normalized cross-correlation (NCC) is used to obtain the integer pixel displacement, as shown in the following formula:

[0073]

[0074] Among them, f m and g m Represent the average gray value of the reference subset and the target subset, f(x,y) and Represents the coordinates (x, y) and The gray value at , M represents the half step length of the matching window, and C represents the correlation coefficient. The reference subset and the target subset are determined according to the execution object of the digital image correlation speckle matching method.

[0075] Then, the region with the highest correlation coefficient is selected and the corresponding pixel offset is used as the initial deformation parameter.

[0076] In the precise matching, according to the preset matching window size, a first-order deformation function is used to describe the deformation of the reference area relative to the target area, that is, sub-pixel deformation, where the first-order deformation function is shown as follows:

[0077]

[0078] Among them, u and v are the center point P(x 0 ,y 0 ) in the x and y directions, Δx and Δy are the displacement components from the center point P to point P i The distance between (x,y), u x 、u y 、v x and v y represent the displacement gradient, is the two-dimensional coordinate corresponding to the center point P.

[0079] The optimal correlation coefficient is obtained by iterative optimization to solve the fine sub-pixel deformation parameter p = [u, u x ,uy ,v,v x ,v y ].

[0080] In this embodiment, an iterative normalized least squares method is used to obtain the optimal correlation coefficient, wherein the criterion of the iterative normalized least squares method is as follows:

[0081]

[0082] The digital image correlation speckle matching method is the key content of the present invention, and the method is also used in the image sequence matching in step 4). The difference lies in that the reference subset and the target subset determined are different: in static image stereo matching and image sequence matching, the matching methods used are consistent, but the objects targeted are different. For example, static image stereo matching targets the initial static frame of the binocular image; in image sequence matching, if there is no occlusion, the execution object is the binocular image, and the sequence matching of the monocular image is performed first, and then the stereo matching of the binocular image is performed; if there is occlusion, only the sequence matching of the monocular image is performed; that is, according to the different execution objects, different reference subsets and target subsets can be determined, so as to apply the digital image correlation speckle matching method for matching.

[0083] Step 3) Occluded target detection: Use an occluded target detection algorithm combined with affine transformation to identify occluded targets in the left and right image sequences.

[0084] Step 31) When extracting the pixel coordinates of the occluded object in the image sequence, the first frame in each camera is used as a reference image.

[0085] Step 32) Select the unobstructed scattered speckles in the reference image, use the digital image correlation speckle matching method to obtain the corresponding points of the selected scattered speckles in each frame of the image sequence, and establish the transformation relationship between the image sequence and the reference image based on the affine transformation model of the following formula:

[0086] x'=Ax+By+t x

[0087] y'=Cx+Dy+t y

[0088] Where (x, y) is the coordinate of the scattered spot in the reference image, (x', y') is the coordinate of the corresponding point in the image sequence, A, B, C and D are the parameters that control rotation, scaling and shearing, t x and t y is the translation amount, and its affine transformation relationship is obtained through at least 3 pairs of coordinate points.

[0089] Step 33) Perform affine transformation on the image sequence using the affine transformation model to generate an affine transformed image, and obtain a differential image between the affine transformed image and the original image sequence.

[0090] Step 34) The occluders (artificial markers) in the differential image are extracted by an independent multimodal BGS algorithm in the BGS library to form an occluded pixel set.

[0091] Step 4) Image sequence matching: Based on the occluded target detection results, image sequence matching is performed through the occlusion condition discrimination strategy. For unoccluded speckles, the digital image correlation speckle matching method is used in combination with camera parameters to perform binocular stereo-sequence dual matching in binocular sequence images; for speckles that are occluded in only one camera perspective, the digital image correlation speckle matching method is used in combination with the scale factor to perform monocular sequence matching.

[0092] After obtaining the set of occluded pixels in the image sequence, it is first necessary to remove unreliable scattered spots in the ROI before image sequence matching, and then perform image sequence matching, which specifically includes the following steps:

[0093] Step 41) Select a search window centered on the target point to be measured, and the size of the search window is the same as the size of the matching window.

[0094] Step 42) executes the occlusion condition discrimination strategy, discriminates the occlusion condition in the image at each moment, and when all moments are judged or the early termination condition is reached, performs corresponding matching according to the discrimination result, specifically:

[0095] If the point to be measured is blocked in the search window of the image sequence taken by different cameras at a certain moment, that is, the search window (ROI) of the point to be measured in the left and right cameras at any moment L and ROI R ), if there is an occluded pixel in the image, the early termination condition is reached, the identifier of the point to be tested is set to 0, and the data of the point to be tested is discarded.

[0096] If the point to be measured is not blocked in the search window of the image sequences taken by different cameras at all times, the identifier of the point to be measured is set to 1, and binocular stereo-sequence double matching is performed on the binocular measurement result of the point to be measured, wherein the binocular stereo-sequence double matching is: firstly, sequence matching is performed on the monocular image sequence of a single camera, and then stereo matching of binocular image sequences of different cameras is performed after the matching is completed.

[0097] If the point to be measured is not blocked in the search window of the image sequence taken by a certain camera at all times, but is blocked in the search window of the image sequence taken by another camera, monocular sequence matching is performed according to the monocular measurement results of the camera without any blockage, wherein if the camera without any blockage is the left camera, the identifier of the point to be measured is set to 2, and if it is the right camera, the identifier of the point to be measured is set to 3.

[0098] If none of the above conditions are met, and in the entire image sequence of the test point, at any moment, there is occlusion in the search window of the image sequence taken by at most one camera, then the identifier of the test point is set to 4. For the moment when there is no occlusion in the search window of the image sequences taken by different cameras, the binocular measurement results are used to perform binocular stereo-sequence dual matching. For the moment when there is occlusion in the search window of the image sequence taken by one camera, the unoccluded monocular measurement results are used to perform monocular sequence matching.

[0099] The above occlusion condition judgment strategy can be expressed as:

[0100]

[0101] in, Indicates the point to be measured P n The identifier of and represents the search window of the entire image sequence of the left image and the right image, O is the set of occluded pixels, and It is represented as the search window of the i-th frame image of the point to be measured in the left image and the right image.

[0102] Step 5) Shock response spectrum solution: Based on the image sequence matching results, the physical coordinates of the measurement points are reconstructed, and after the coordinate system of the images taken from multiple perspectives is corrected, the corrected coordinates are fused; the displacement of the scattered speckles is determined according to the change of the coordinates in the fusion results, and the acceleration is calculated; based on the calculated acceleration, the shock response spectrum is calculated using the shock response spectrum solution algorithm.

[0103] Step 51) Reconstruct the physical coordinates of the measurement points

[0104] According to the image sequence matching results, a spatial mapping operator is used to reconstruct the actual physical coordinates of the measurement points in the world coordinate system. The spatial mapping operator includes three-dimensional reconstruction and a scale factor, wherein the scale factor is expressed as the ratio of the three-dimensional world coordinates to the two-dimensional image coordinates in the measurement area of ​​the object surface.

[0105] For the scattered spots with marker 1, their three-dimensional coordinates are obtained by three-dimensional reconstruction technology. For the scattered spots with markers 2, 3 and 4, the coordinates in the main impact direction are obtained by calculating the scale factor in the reference image.

[0106] Step 52) Coordinate system calibration

[0107] Since the world coordinate system O established according to the plane target w The coordinate axis of the world coordinate system O may not be consistent with the main impact direction, so it is necessary to w Make corrections.

[0108] In the usual calibration process, the world coordinate system O w The X-axis and Y-axis are parallel to the plane target, the Z-axis is perpendicular to the XOY plane, and the X-axis is assumed to be approximately the main impact direction. The world coordinate system O w The calibration is divided into two steps:

[0109] The first step of calibration is to rotate the XOZ plane around the Y axis so that the X axis is parallel to the object surface. 1 ,Y 1 ,Z 1 ) and (X 2 ,Y 2 ,Z 2 ), calculate the vector (X 1 ,Z 1 ) and (X 2 ,Z 2 ) between which the angle θ is;

[0110] The second step of calibration: The XOY plane is rotated around the Z axis to align the X axis with the main impact direction. The horizontal or vertical straight line on the object structure is obtained using the straight line extraction algorithm, and two three-dimensional points (X 3 ,Y 3 ,Z 3 ) and (X 4 ,Y 4 ,Z 4 ), the straight line formed by these two points is parallel to the extracted straight line, thus obtaining the vector (X 3 ,Y 3 ) and (X 4 ,Y 4 ) on the XOY plane, the corrected coordinates (X', Y', Z') are expressed as follows:

[0111]

[0112] Among them, (X, Y, Z) is the coordinate before correction.

[0113] Step 53) Information Fusion

[0114] The three-dimensional coordinates of each measuring point after coordinate system correction are fused to obtain the overall coordinate information of the surface of the object to be measured.

[0115] Step 54) Displacement and acceleration calculation

[0116] Based on the fused result in step 53), the displacement and acceleration are calculated using the existing displacement and acceleration calculation method, and this calculation process will not be described in detail in this embodiment.

[0117] Step 55) Impact response spectrum solution

[0118] The shock response spectrum is a calculation function based on the acceleration time history. Its basic theory is to apply the acceleration time-domain shock excitation to the real physical system to calculate the shock response caused by the shock input. By decomposing the complex real physical system into a series of single degree of freedom (SDOF) systems with different natural frequencies, each SDOF system has a unique time history response to a given basic shock input. The maximum value of each system response is obtained, and the results of each SDOF are plotted to obtain the shock response spectrum. Therefore, the shock response spectrum is the absolute peak acceleration response of each SDOF system to the time-domain load input. The horizontal axis is the natural frequency of each SDOF, and the vertical axis is the corresponding absolute peak acceleration response value. The shock response spectrum can be used to evaluate the damage boundary of a given component and the maximum dynamic load that the equipment can withstand. Standard shock response spectra such as Figure 2 As shown, it consists of three elements, namely the low-frequency slope, the inflection point frequency and the high-frequency amplitude.

[0119] Taking a certain SDOF as an example, according to Newton's law, the motion equation of the mass block in the single degree of freedom system is obtained as follows:

[0120]

[0121] Among them, m, c and k are the mass, damping coefficient and stiffness of the SDOF respectively, and f n is its natural frequency, x and y are the response displacement of the mass block and the displacement of the base, respectively. are the velocity response of the mass block and the velocity of the base, is the acceleration response at time t under this impact signal (the dots represent the derivative with respect to time).

[0122] The relative displacement of the mass block relative to the base is expressed as z = xy, and the following is derived:

[0123]

[0124] in, The impact excitation acceleration signal is input to the single degree of freedom system, and the damping coefficient c = 2mξω n , stiffness Then the shock response spectrum relationship equation is derived:

[0125]

[0126] Where ξ is the damping ratio (usually set to 0.05), ω n Represents the natural circular frequency, denoted as ω n =2πf n , f n is the natural frequency.

[0127] By solving the shock response spectrum relationship equation, the relationship between the natural frequency and the shock response peak of the single-degree-of-freedom system is obtained, thereby obtaining the shock response spectrum of the complex real physical system.

[0128] The impulse response spectrum relationship equation is usually solved using the digital recursive filtering method proposed by Smallwood. The Smallwood method uses a filter to simulate the SDOF system. It regards the output signal of the time domain signal after passing through the filter as the single degree of freedom impulse response result. It has the advantages of simplicity and high calculation accuracy. The final expression of the Smallwood method is as follows:

[0129]

[0130] p 0 =1-θsin(ω d Δt) / (ω d Δt)

[0131] p 1 =2θ(sin(ω d Δt)-cos(ω d Δt))

[0132] p 2 =θ(θ-sin(ω d Δt) / (ω d Δt))

[0133] q 1 =2θcos(ω d Δt)

[0134]

[0135] Where Δt represents the unit time interval, is the acceleration response at time t, is the impact excitation acceleration signal input to the single-degree-of-freedom system at time t, p 0、p 1 、p 2 ,q 1 and q 2 are coefficient terms, ω n represents the natural circular frequency and ξ is the damping ratio.

[0136] This embodiment verifies the feasibility of the impact response spectrum solution method based on hybrid single / binocular high-speed video measurement under occlusion conditions proposed in the present invention through an impact test conducted on an air cannon impact test bench. The measurement object is a T-shaped tooling plate with uniform material and a size of approximately 20 cm × 20 cm. Speckle patterns and circular marks are arranged on the surface of the T-shaped plate, and an accelerometer with an acquisition frequency of 100kHz is placed. The high-speed camera model used is Acuteye AE-1-M-3500, with a maximum resolution of 1280×860 pixels. In the experiment, the shooting distance of the two cameras was about 2 meters, and the resolution was set to 320×320 pixels. Each camera is equipped with a fixed-focus lens with a focal length of 35 mm, and the frame rate is set to 9000 Hz. In the experiment, due to the high-level impact force, the circular mark on the surface of the T-shaped plate fell off, resulting in occlusion of the target point to be measured.

[0137] The reliability of the present invention is verified by measuring scattered speckles under shielding conditions and comparing the shock response spectrum obtained by the present invention with the shock response spectrum obtained based on an accelerometer.

[0138] like Figure 3 As shown, (a), (b) and (d) are the 1st frame (static), 1120th frame and 1650th frame of the left image sequence, respectively, (c) and (e) are the occluded target detection results corresponding to (b) and (d), and (f)-(g) are the corresponding right image sequence diagrams. 3(a) shows the first frame (static) of the image sequence captured by the left camera, (b) and (d) show the 1120th and 1650th frames of the left image sequence, reflecting the spot occlusion caused by the falling of artificial markers. It can be seen that the target occluded in the left camera may not be occluded in the right camera. Figure 4 As shown, (a) is the set of occluded pixels in the left image sequence, (b) is the set of occluded pixels in the right image sequence, and (c) is the statistics of scattered speckle information in the ROI, where magenta indicates measurement using a binocular camera, green and blue indicate measurement using only the left or right camera, cyan indicates measurement using a monocular / binocular mixed method, and yellow indicates targets that do not need to be measured. Figure 4 (a) and (b) are the motion of the occluded target in the entire image sequence extracted by the occluded target detection algorithm, and a hybrid single / binocular high-speed video measurement strategy is used to identify each target point to be measured, and a corresponding identifier is assigned to each target point, such as Figure 4 (c) as shown.

[0139] Due to the occlusion of scattered speckles, the image processing algorithm produces large errors, which in turn affects the calculation results of SRS. Figure 5 Shown are the impact response spectrum results of a target point that is occluded in the left and right cameras at different times, and the occlusion time in the right camera is longer. It can be seen that when the left camera is used for measurement, a large error is generated in the high-frequency part. When the right camera and the binocular camera are used for measurement, the overall SRS curve has a large deviation, indicating the significant impact of the occluded target on the SRS measurement results. However, the hybrid measurement method integrates the information of the scattered speckles in different time and space dimensions through a hybrid strategy based on image matching, and accurately restores the dynamic change process of the scattered speckles under occlusion conditions. Compared with the SRS reference value obtained based on the accelerometer, the SRS measured by the present invention has good consistency with the reference result, and the average acceleration error is 12.32%, which proves the effectiveness of the present invention in impact testing under occlusion conditions.

[0140] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement, characterized in that: The following steps are involved: Camera calibration: Use Zhang Zhengyou's plane target calibration method to calibrate the high-speed camera to obtain camera parameters and scale factors; Static image stereo matching: Under static conditions, the initial static frames of binocular images obtained by high-speed cameras are stereo matched using the digital image correlation speckle matching method; Occluded object detection: Use an occluded object detection algorithm combined with affine transformation to identify occluded objects in the left and right image sequences; Image sequence matching: Based on the occluded target detection results, image sequence matching is performed through the occlusion condition discrimination strategy. For unoccluded speckles, the digital image correlation speckle matching method is used in combination with camera parameters to perform binocular stereo-sequence dual matching in binocular sequence images; for speckles that are occluded in only one camera perspective, the digital image correlation speckle matching method is used in combination with the scale factor to perform monocular sequence matching; Shock response spectrum solution: Based on the image sequence matching results, the physical coordinates of the measurement points are reconstructed, and after the coordinate system of the images taken from multiple perspectives is corrected, the corrected coordinates are fused; the displacement of the scattered speckles is determined according to the change of coordinates in the fusion results, and the acceleration is calculated; based on the calculated acceleration, the shock response spectrum is calculated using the shock response spectrum solution algorithm.

2. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 1 is characterized in that: The digital image correlation speckle matching method is specifically: The digital image correlation algorithm is used to match the scattered speckles in images collected by different high-speed cameras to obtain the correspondence between the same scattered speckles at different collection times and in different imaging spaces. During the matching process, the first image of a selected camera is always used as the reference image, the region of interest is defined in the reference image, and the size and step size of the matching window are set with a certain pixel coordinate as the center. The image sequence is matched using coarse matching and fine matching in turn to determine the points of the same name.

3. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 2 is characterized in that: The coarse matching uses normalized cross-correlation to obtain integer pixel displacement, as shown in the following formula: Among them, f m and g m Represent the average gray value of the reference subset and the target subset, f(x,y) and Represents the coordinates (x, y) and The gray value at , M represents the half step length of the matching window, and C represents the correlation coefficient; the reference subset and the target subset are determined according to the execution object of the digital image correlation speckle matching method; Select the area with the highest correlation coefficient and use the corresponding pixel offset as the initial deformation parameter; The precise matching uses a first-order deformation function to describe the deformation of the reference area relative to the target area, that is, sub-pixel deformation, according to a preset matching window size. The first-order deformation function is shown in the following formula: Among them, u and v are the displacement components of the center point P(x0,y0) in the x and y directions respectively, and Δx and Δy are the displacement components of the center point P to point P i The distance between (x,y), u x 、u y 、v x and v y represent the displacement gradient, is the two-dimensional coordinate corresponding to the center point P; The optimal correlation coefficient is obtained by iterative optimization to solve the fine sub-pixel deformation parameter p = [u, u x ,u y ,v,v x ,v y ].

4. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 3 is characterized in that: The optimal correlation coefficient is obtained by using iterative normalized least squares method, and the criteria of the iterative normalized least squares method are as follows:

5. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 1 is characterized in that: The occluded target detection is specifically as follows: When extracting the pixel coordinates of the occluded object in the image sequence, the first frame in each camera is used as the reference image; Select the unobstructed scattered speckles in the reference image, use the digital image correlation speckle matching method to obtain the corresponding points of the selected scattered speckles in each frame of the image sequence, and establish the transformation relationship between the image sequence and the reference image based on the following affine transformation model: x'=Ax+By+t x y'=Cx+Dy+t y Where (x, y) is the coordinate of the scattered spot in the reference image, (x', y') is the coordinate of the corresponding point in the image sequence, A, B, C and D are the parameters that control rotation, scaling and shearing, t x and t y is the translation amount, and its affine transformation relationship is obtained through at least 3 pairs of coordinate points; Performing affine transformation on the image sequence using the affine transformation model to generate an affine transformed image, and obtaining a differential image between the affine transformed image and the original image sequence; The occluders in the differential image are extracted using the independent multimodal BGS algorithm in the BGS library to form an occluded pixel set.

6. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 1 is characterized in that: The image sequence matching comprises the following steps: Selecting a search window centered on the target point to be measured, wherein the size of the search window is the same as the size of the matching window; Execute the occlusion condition discrimination strategy to discriminate the occlusion condition in the image at every moment. When all the discriminations are completed or the early termination condition is reached, perform the corresponding matching according to the discrimination results. Specifically: If the point to be measured is blocked in the search window of the image sequences taken by different cameras at a certain moment, the early termination condition is met, the identifier of the point to be measured is set to 0, and the data of the point to be measured is discarded; If the point to be measured is not blocked in the search window of the image sequences taken by different cameras at all times, the identifier of the point to be measured is set to 1, and binocular stereo-sequence double matching is performed for the binocular measurement result of the point to be measured. The binocular stereo-sequence double matching is: firstly, sequence matching is performed for the monocular image sequence of a single camera, and after the matching is completed, stereo matching is performed for the binocular image sequences of different cameras; If the point to be measured is not blocked in the search window of the image sequence taken by a certain camera at all times, but is blocked in the search window of the image sequence taken by another camera, then monocular sequence matching is performed according to the monocular measurement results of the camera without any blockage, wherein if the camera without any blockage is the left camera, the identifier of the point to be measured is set to 2, and if it is the right camera, the identifier of the point to be measured is set to 3; If none of the above conditions are met, and in the entire image sequence of the test point, at any moment, there is occlusion in the search window of the image sequence taken by at most one camera, then the identifier of the test point is set to 4. For the moment when there is no occlusion in the search window of the image sequences taken by different cameras, the binocular measurement results are used to perform binocular stereo-sequence dual matching. For the moment when there is occlusion in the search window of the image sequence taken by one camera, the unoccluded monocular measurement results are used to perform monocular sequence matching.

7. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 6 is characterized in that: The physical coordinates of the reconstructed measurement points are specifically: According to the image sequence matching results, a spatial mapping operator is used to reconstruct the actual physical coordinates of the measurement points in the world coordinate system. The spatial mapping operator includes three-dimensional reconstruction and a scale factor. The scale factor is expressed as the ratio of the three-dimensional world coordinates to the two-dimensional image coordinates in the measurement area of ​​the object surface.

8. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 7 is characterized in that: For the scattered spots with marker 1, their three-dimensional coordinates are obtained by three-dimensional reconstruction technology. For the scattered spots with markers 2, 3 and 4, the coordinates in the main impact direction are obtained by calculating the scale factor in the reference image.

9. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 1 is characterized in that: The coordinate system correction of the images shot from multiple perspectives is specifically to correct the world coordinate system w To make a correction: The world coordinate system O w The X-axis and Y-axis are parallel to the plane target, the Z-axis is perpendicular to the XOY plane, and the X-axis is assumed to be approximately the main impact direction. The world coordinate system O w The calibration is divided into two steps: The first step of calibration: the XOZ plane is rotated around the Y axis to make the X axis parallel to the object surface. By selecting two 3D points (X1, Y1, Z1) and (X2, Y2, Z2) in the measurement area, the angle θ between the vectors (X1, Z1) and (X2, Z2) on the XOZ plane is calculated. The second step of calibration: the XOY plane is rotated around the Z axis to align the X axis with the main impact direction. The horizontal or vertical straight line on the object structure is obtained using the straight line extraction algorithm, and two three-dimensional points (X3, Y3, Z3) and (X4, Y4, Z4) are selected in the measurement area. The straight line formed by these two points is parallel to the extracted straight line, thereby obtaining the angle β between the vectors (X3, Y3) and (X4, Y4) on the XOY plane. The corrected coordinates (X', Y', Z') are expressed as follows: Among them, (X, Y, Z) is the coordinate before correction.

10. The method for calculating the impulse response spectrum based on hybrid monocular and binocular high-speed video measurement according to claim 1, characterized in that: The shock response spectrum solution algorithm performs the following steps: According to Newton's law, the motion equation of the mass block in the single degree of freedom system is obtained as follows: Among them, m, c and k are the mass, damping coefficient and stiffness of the SDOF respectively, and f n is its natural frequency, x and y are the response displacement of the mass block and the displacement of the base, respectively. are the velocity response of the mass block and the velocity of the base, is the acceleration response at time t under this impact signal; The relative displacement of the mass block relative to the base is expressed as z = xy, and the following is derived: in, The impact excitation acceleration signal is input to the single degree of freedom system, and the damping coefficient c = 2mξω n , stiffness Then the shock response spectrum relationship equation is derived: Where ξ is the damping ratio, ω n Represents the natural circular frequency, denoted as ω n =2πf n , f n is the natural frequency; The digital recursive filtering method is used to solve the shock response spectrum relationship equation to obtain the relationship between the natural frequency and the shock response peak of the single-degree-of-freedom system, thereby obtaining the shock response spectrum of the complex real physical system.

Images