High-speed video measurement method and equipment for full-sequence three-dimensional motion parameters of fragment targets

Abstract

The present invention relates to a method and apparatus for high-speed video measurement of the three-dimensional motion parameters of a full sequence of fragment targets. The method comprises the following steps: acquiring a sequence of fragment images captured by a binocular high-speed camera and performing filtering processing; employing a Gaussian background modeling method based on an adaptive learning rate to obtain the two-dimensional pixel coordinates of the fragments in the fragment sequence images; generating fragment tracking trajectories by estimating the clutter rate and average detection probability, and utilizing the Hungarian algorithm to achieve cascade matching of fragments within the same high-speed camera viewing angle using the fragment tracking trajectories; achieving stereo matching of the fragments based on an adaptive matching window size and epipolar geometry constraints of the binocular high-speed camera; and constructing a set of collinearity condition equations using the bundle method to obtain the three-dimensional coordinates of the fragments in the sequence images through indirect adjustment calculation. Compared with existing technologies, the present invention has the advantages of strong adaptability to dynamic environments.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for high-speed video measurement of full-sequence three-dimensional motion parameters of a fragment target. Background Art

[0002] The three-dimensional dynamic parameters of the fragmentation field are key data for calculating the killing radius and damage area of ​​the fragmentation warhead. Conventional fragmentation field parameter testing methods, such as the target plate method, have low testing costs, but data acquisition is time-consuming and labor-intensive, making rapid monitoring impossible. The multispectral detection method can observe the explosion process through the fireball, but the resolution and acquisition frequency are low, making it difficult to meet the measurement requirements of high-speed fragments. Optical imaging methods, on the other hand, have higher resolution and acquisition frequency, which can compensate for the shortcomings of the above methods.

[0003] Chinese patent application publication number CN115272403A discloses a fragment dispersion characteristic testing method and system based on image processing technology. The method calculates the fragment distribution radius based on the warhead equivalent, arranges binocular cameras according to the fragment distribution radius safety zone, calibrates the cameras and sets parameters, and collects a sequence of images of the fragment flight trajectory; the collected trajectory sequence images are automatically identified and the coordinates are extracted; the extracted fragments are matched with each other in the fields of view of two cameras, and the two-dimensional coordinates of the extracted fragments in the image are used to calculate the three-dimensional coordinates of the fragments in space; the fragment motion equation is fitted in three-dimensional space, and the fragment motion parameters are solved based on the fitted equation and the known three-dimensional spatial coordinates of the fragments.

[0004] The above application achieves non-contact measurement of fragments, but the following problems still exist: the image quality of high-speed cameras deteriorates due to the short exposure time when shooting at high frame rates; the fragmentation experiment site environment is complex, and there are gradual and sudden changes in lighting, dynamic changes in natural or artificial backgrounds, etc. in the sequence images, resulting in unsatisfactory accuracy in fragment extraction; and there is a lack of adaptability to dynamic environments.

[0005] In summary, there is currently a lack of a high-speed video measurement method for the full sequence three-dimensional motion parameters of fragment targets to overcome or partially overcome the above problems. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method and equipment for high-speed video measurement of the full sequence three-dimensional motion parameters of fragment targets, so as to improve the accuracy of fragment parameter extraction.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] One aspect of the present invention provides a method for high-speed video measurement of three-dimensional motion parameters of a fragment target in a full sequence, comprising the following steps:

[0009] Collect fragment sequence images taken by binocular high-speed cameras and perform filtering processing;

[0010] A Gaussian background modeling method based on adaptive learning rate is used to obtain the two-dimensional pixel coordinates of fragments in the fragment sequence image;

[0011] By estimating the clutter rate and average detection probability, a fragment tracking trajectory is generated, and the Hungarian algorithm is used to achieve cascade matching of fragments under the same high-speed camera viewing angle.

[0012] Based on the adaptive matching window size and the epipolar geometry constraints of the binocular high-speed camera, the fragment stereo matching is achieved;

[0013] The bundle method is used to construct the collinearity condition equations, and the three-dimensional coordinates of the fragments in the sequence images are obtained through indirect adjustment calculation.

[0014] As a preferred technical solution, the filtering process includes:

[0015] The fragment sequence images are processed by filtering with a two-dimensional discrete zero-mean Gaussian function;

[0016] Top hat filtering is used to process fragment sequence images.

[0017] As a preferred technical solution, the Gaussian background modeling method specifically includes the following steps:

[0018] A background sample model based on the global illumination variation factor is used as the demarcation threshold to determine foreground and background pixels, and the two-dimensional pixel coordinates of the fragments are extracted through the three-frame difference method.

[0019] The learning rate of the sample model is adaptively determined based on the difference between the target pixel and the adjacent pixels in the current frame and the background model.

[0020] As a preferred technical solution, the learning rate is calculated using the following formula:

[0021]

[0022] Where k is the learning rate, a and b are constants, f represents the current frame, ΔD is the difference between the current frame pixel and its neighboring pixels and the background model, and F is the number of sequential image frames.

[0023] As a preferred technical solution, the process of generating the fragment tracking trajectory includes:

[0024] The initial tracking area of ​​the fragment motion is obtained by trajectory splicing, the clutter rate and average detection probability are calculated using the CPHD filter, and the fragment tracking trajectory is generated using the GLMB filter.

[0025] As a preferred technical solution, after the cascade matching, the following further comprises:

[0026] Trajectories whose diffusion angle exceeds a preset value or whose flight direction is opposite to / not consistent with the overall diffusion direction are screened out and / or trimmed.

[0027] As a preferred technical solution, the fragment stereo matching process includes the following steps:

[0028] Based on the epipolar geometry constraints of the binocular high-speed camera, the matching search area of ​​the fragments under different viewing angles is obtained;

[0029] In the image block matching process of a single frame of fragments, the result of cascade matching is used as the prior information for stereo matching. The matching window size is adaptively updated through the matching window information entropy to achieve stereo matching of fragments.

[0030] As a preferred technical solution, after the fragment stereo matching, a reliability verification process is also included, including the following steps:

[0031] The image block areas of the previous and next frames are matched. If the sum of the correlations between the image blocks of three consecutive frames under one foot of the high-speed camera and the image blocks of three consecutive frames under the right camera is greater than a preset threshold, the stereo matching of the current frame is considered reliable.

[0032] Another aspect of the present invention provides a high-speed video measurement device for full-sequence three-dimensional motion parameters of a fragment target, comprising:

[0033] A variable background curtain with a variable background light source on one side;

[0034] A binocular high-speed camera is arranged on the other side of the variable background curtain;

[0035] The synchronous control system comprises a memory and an actuator, wherein the memory stores instructions for executing the above-mentioned method for high-speed video measurement of full-sequence three-dimensional motion parameters of fragment targets.

[0036] Another aspect of the present invention provides a computer-readable storage medium comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the above-mentioned method for high-speed video measurement of full-sequence three-dimensional motion parameters of fragment targets.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] (1) Strong adaptability to dynamic environments: Gaussian background modeling can adapt to the dynamic changes of the scene, but when the background model is updated at a fixed learning rate, its adaptability to the dynamic environment is insufficient. When the learning rate is too small, the background model cannot be updated in time, and the target is prone to "ghosting". When the value is too large, the update frequency of the background model is increased, which can easily cause some slow-moving objects in the image to be judged as background, resulting in false detection. Therefore, in the early stage of model updating, in order to eliminate the interference information in the background model as quickly as possible, it is necessary to update the model at a faster rate; after the interference information in the background model is eliminated, a learning rate that is suitable for the actual scene should be used. This application adopts a Gaussian background modeling method based on an adaptive learning rate, takes the difference between the target pixel and the adjacent pixel in the current frame and the background model as a reference, and adaptively determines the learning rate to improve the target detection performance of the Gaussian background modeling method.

[0039] (2) Reducing false detections due to sudden changes in illumination: Adaptive Gaussian background modeling can solve or partially solve the problem of uneven illumination. However, it is difficult for the background model to respond in real time to sudden changes in illumination, especially in fragmentation test scenarios, which can easily cause a large number of false detections at the moment of sudden changes in illumination. This application adds a global illumination change factor to Gaussian background modeling to eliminate the impact of sudden changes in illumination on fragmentation target detection. The Gaussian background modeling based on adaptive learning rate improves the detection accuracy of the algorithm in dynamic environments by setting different learning rates and the number of models, making the acquired targets more complete.

[0040] (3) The target tracking results obtained by cascade matching are highly accurate: Since there is a certain error between the trajectory estimation obtained by filtering and the actual fragment target position (gravitational coordinates), this application uses the Hungarian algorithm for data association (matching). After the association process is completed, the trajectories with too large diffusion angles and whose flight directions are opposite to / not consistent with the overall diffusion direction are screened out, pruned, etc., to further improve and optimize the target tracking results.

[0041] (4) High stereo matching efficiency and accuracy: On the one hand, this application uses the epipolar geometry constraints of binocular high-speed cameras to narrow the search range. Since the distance from the photographic baseline to the fragment target can be estimated in advance, the search range on the epipolar line corresponding to the left camera image can be further narrowed, thereby improving matching efficiency and accuracy. On the other hand, an adaptive window stereo matching method based on tracking priors is adopted, using robust tracking results as prior information for stereo matching. The matching window size is adaptively updated by matching window information entropy, ultimately achieving stereo matching of the fragment target. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Flowchart of the method for high-speed video measurement of full-sequence three-dimensional motion parameters of fragment targets in an embodiment;

[0043] Figure 2 Schematic diagram of automatic detection of fragment targets under a dynamic background in an embodiment;

[0044] Figure 3 Schematic diagram of fragment target association based on the Hungarian algorithm in an embodiment;

[0045] Figure 4 Schematic diagram of the process of stereo matching of fragment targets under weak texture background in the embodiment;

[0046] Figure 5 Schematic diagram of the fragment target stereo matching process based on tracking priors in an embodiment;

[0047] Figure 6 Schematic diagram of a high-speed video measurement experiment scene of fragments in an embodiment;

[0048] Figure 7 Schematic diagram of automatic detection of fragment positions using four consecutive frames of images in an embodiment;

[0049] Figure 8 Schematic diagram of fragment target tracking results in the left image sequence in the embodiment;

[0050] Figure 9 Schematic diagram of the stereo matching results of left and right sequence images in the embodiment;

[0051] Figure 10 This is a schematic diagram of the distribution and numbering of control points in the fragmentation test control field in the embodiment.

[0052] Among them, 1. Variable background curtain, 2. Variable background light source, 3. Binocular high-speed camera, 4. Synchronous control system. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0054] Example 1

[0055] To address the image processing and spatial positioning issues of complex fragment clusters, this embodiment proposes a high-speed video measurement method for the three-dimensional motion parameters of a full-sequence fragment target. This method utilizes two or more high-speed cameras to synchronously record the instantaneous state of fragment dispersion at different moments. Through key technologies such as automated detection and tracking of massive amounts of tiny fragments, multi-view stereo matching of fragments against weakly textured backgrounds, and high-precision three-dimensional reconstruction and motion state inversion, it achieves automated measurement of the three-dimensional motion parameters of the full fragment sequence. This application enables the rapid construction of a video measurement system under both indoor and outdoor conditions, achieving high-precision inversion of the three-dimensional motion parameters of the full fragment sequence.

[0056] This application provides a high-speed video measurement and analysis method for inverting the three-dimensional motion parameters of fragments over a full sequence. This method primarily involves detecting and tracking massive amounts of high-speed fragment targets under dynamic background conditions, multi-view matching of fragments with anomalous properties against weakly textured backgrounds, and high-precision three-dimensional reconstruction and motion inversion of fragments. This method enables the rapid construction of video measurement systems in a variety of complex environments, both in the laboratory and in the field, to achieve high-precision inversion of the three-dimensional motion parameters of fragments over long time sequences.

[0057] To address the problem of high-precision detection and tracking of fragment targets, a fragment image processing framework is provided, including image preprocessing methods, an automatic fragment detection method combining adaptive background modeling and three-frame difference method, and a fragment target tracking method combining generalized labeled multi-Bernoulli trajectory estimation (GLMB) and Hungarian association algorithm; to address the problem of multi-perspective matching of massive heterogeneous fragments under weak texture background, the target search area is greatly reduced by using epipolar constraints, and an adaptive window matching method based on tracking prior is used to solve the problem of insufficient background texture information in the fragment target area, thereby achieving accurate heterogeneous fragment matching; to address the problem of three-dimensional reconstruction of the entire fragment sequence, the overall bundle adjustment method is used to optimize the exterior orientation elements of the high-speed camera, thereby obtaining high-precision three-dimensional coordinates of the entire fragment target sequence.

[0058] See also Figure 1 The method for high-speed video measurement of three-dimensional motion parameters of a fragment target in a full sequence provided by this embodiment includes the following parts:

[0059] S1, automatic detection and tracking of high-speed fragmentation targets;

[0060] S2, automatic stereo matching of fragments under weak texture background based on tracking prior;

[0061] S3, 3D reconstruction and motion inversion of fragments using the whole beam method.

[0062] The following describes the three parts separately.

[0063] 1. Automatic detection and tracking of high-speed fragmentation targets.

[0064] In order to achieve accurate detection and tracking of high-speed fragment targets in long sequence images under dynamic background, the sequence image processing is divided into three sub-parts: image preprocessing, automatic detection of fragment targets under dynamic background, and automatic tracking of abnormal fragment targets.

[0065] See also Figure 2 The present application provides a method for high-speed video measurement of the full sequence three-dimensional motion parameters of a fragment target, comprising the following sub-steps:

[0066] S111, obtaining video processing, performing image preprocessing, and calculating the global illumination change factor;

[0067] S112, using a three-frame difference method to perform difference, i.e., threshold segmentation, on the current frame and the two adjacent frames before and after the current frame;

[0068] S113, creating an adaptive background model for the current frame image, and performing a differential operation between the current frame image and the background model;

[0069] S114, based on the segmentation result obtained in S12, the difference result obtained in S13 and the global illumination change factor in S11, a binary image of the fragment target on the sequence image is obtained, and morphological processing and noise processing are performed.

[0070] The three sub-parts are described below.

[0071] (1) Image preprocessing: High-speed cameras often suffer from image quality degradation due to short exposure times when shooting at high frame rates. Therefore, a two-dimensional discrete zero-mean Gaussian function is used to eliminate noise. At the same time, to address the issues of image noise fusion with fragment targets and low light conditions, top-hat filtering is used to maintain the peak intensity of the fragment targets and eliminate other features. While suppressing noise, it partially restores important features such as the grayscale and boundaries of the fragments, providing support for subsequent target detection and tracking.

[0072] (2) Automatic detection of fragment targets under dynamic background: The fragment experiment site environment is complex, and the sequence images often contain gradual and sudden changes in illumination, dynamic changes in natural or artificial backgrounds, etc. In order to quickly and accurately extract fragment targets.

[0073] The background modeling method uses the background sample model B(x i ,y i ) is used as the dividing threshold to judge the foreground pixels and background pixels, B(x i ,y i ) contains background pixel values ​​and features, and the pixel I(x f ,y f) is a background pixel.

[0074]

[0075] Gaussian background modeling can adapt to the dynamic changes of the scene, but when the background model is updated at a fixed learning rate, it is not adaptable to dynamic environments. When the learning rate is too small, the background model cannot be updated in time, and the target is prone to "ghosting". When the value is too large, the update frequency of the background model is increased, which can easily cause some slow-moving objects in the image to be judged as background, resulting in false detection. Therefore, in the early stage of model update, in order to eliminate the interference information in the background model as quickly as possible, the model is usually updated at a faster rate; after the interference information in the background model is eliminated, a learning rate that is suitable for the actual scene is adopted. The difference between the target pixel and the adjacent pixels in the current frame and the background model is used as a reference to adaptively determine the learning rate to improve the target detection performance of the Gaussian background modeling method. Its basic principle is shown in Equation (2).

[0076]

[0077] Where k is the learning rate, a and b are constants, f is the current frame, ΔD is the difference between the current frame pixel and its 8 neighboring pixels and the background model, and F is the judgment condition for the number of sequence image frames.

[0078] Adaptive Gaussian background modeling can solve the problem of uneven illumination, but it is difficult for the background model to respond in real time to sudden changes in illumination, especially in fragmentation test scenarios, which are prone to sudden changes in illumination, resulting in a large number of false detections at the moment of sudden changes in illumination. This application incorporates a global illumination variation factor into Gaussian background modeling to eliminate the impact of sudden changes in illumination on fragmentation target detection. Adaptive learning rate-based Gaussian background modeling improves the algorithm's detection accuracy in dynamic environments by setting different learning rates and the number of models, making the acquired targets more complete.

[0079] However, for the edge of the fragment target, due to the constant change of the target position, there is still the problem of incomplete and discontinuous target outline. To solve the above problem, this application combines the advantage of the three-frame difference method in effectively extracting the target outline and integrates it with Gaussian background modeling to achieve the effect of supplementing the target outline.

[0080] After obtaining a binary image of the fragment targets in the sequence using the above method, image morphology methods are used to eliminate false detections and further eliminate non-fragment moving targets based on prior knowledge of the fragment targets, such as their size and shape. Finally, the two-dimensional pixel coordinates of the fragment targets are obtained by centroidal pixel coordinates.

[0081] (3) Automatic tracking of targets with abnormal fragments: Since fragments may flip over or be affected by other factors during flight, their shape will continue to change in the sequence image. This application determines the initial tracking area of ​​the fragment (point) movement through the trajectory connection (prediction) method. A low-complexity and robust potential probability hypothesis density filter (Cardinalized Probability Hypothesis Density, CPHD) filter is used to estimate the clutter rate and average detection probability in real time. This prior information is then introduced into the GLMB filter to generate the target tracking trajectory. Since there is a certain error between the trajectory estimate obtained above and the actual fragment target position (gravitational coordinates), the Hungarian algorithm is used for data association (matching), such as Figure 3 After the association process is completed, trajectories with excessively large diffusion angles or whose flight directions are opposite to or inconsistent with the overall diffusion direction are screened out and pruned to further improve and optimize the target tracking results.

[0082] See also Figure 3 ,The process of data association using the Hungarian algorithm includes the following sub-steps:

[0083] S121, performing adaptive GLMB tracking on the original data;

[0084] S122, input the obtained result (estimate) and the observation value of the current frame;

[0085] S123, calculating the Euclidean / Mahalanobis distance between the result and the measurement point, and the HOG feature (a local image feature) cosine similarity between the result and the measurement, calculating the Hungarian algorithm cost matrix between each result and the current frame measurement, and performing linear weighting;

[0086] S124, calculate observation→trajectory using Hungarian algorithm;

[0087] S125, the original observations are divided into matched M and unmatched U, and the adaptive GLMB algorithm tracking is re-executed on the unmatched U results until the number of unmatched U is lower than the set threshold and the loop ends. Otherwise, S122 is executed to form a cascade matching.

[0088] 2. Automatic stereo matching of fragments under weak texture background based on tracking prior.

[0089] After obtaining the coordinates and sequence correspondence of the fragments in the image, in order to calculate the three-dimensional coordinates of the fragments, it is necessary to perform stereo matching on the fragment targets taken from different angles. In order to improve the detection rate of fragments, the background in the experiment is usually a weak texture background such as a monochrome curtain or the sky. To solve the problem of accurate stereo matching of fragments under weak texture background, this application provides an automatic stereo matching method for fragments. Its basic process is as follows: Figure 4As shown, it includes the following sub-steps:

[0090] S21, for the fragmentation points of the camera image, the search area is determined using the epipolar geometry constraint, and the location area of ​​the same-name points in the left and right images is predicted at one time.

[0091] After obtaining the camera's internal and external orientation elements through camera calibration, the fundamental matrices of the left and right camera images can be calculated. This allows the use of epipolar constraints to reduce the search range for matching fragment targets from different viewing angles. Furthermore, since the distance from the photographic baseline to the fragment target can be estimated in advance, the search range on the epipolar line corresponding to the left camera image can be further narrowed, thereby improving matching efficiency and accuracy.

[0092] S22, based on the tracking prior adaptive window stereo matching method, finds the precise multi-view matching relationship of the clustered fragment area in the adaptive window and completes the matching of fragment homonymous points.

[0093] This application provides an adaptive window stereo matching method based on tracking prior, which uses robust tracking results as prior information for stereo matching, adaptively updates the matching window size through matching window information entropy, and ultimately achieves stereo matching of fragment targets.

[0094] For image block matching of single-frame fragments, the fragments have different shapes and the background information texture is single. A single-sized matching window is not suitable for accurate multi-view matching of a large number of heterogeneous fragments. A matching window that is too small will cause matching failure due to the lack of image information, while a large target window will reduce computational efficiency. Therefore, this application determines the size of the matching window by the information entropy of the matching window. At the beginning of fragment target matching, a small window is given to each fragment target point. Once a unique corresponding image area is matched and its information entropy value reaches a predefined threshold, the window size will stop growing. The correlation coefficient of the fragment target is calculated by zero-mean normalized cross-correlation:

[0095]

[0096] The window size of the target image block can be set to (2M+1)×(2M+1), f i,j and g i,j are the grayscale values ​​of the coordinates (i, j) in the target image and the image with the same name, and are the average grayscale values ​​of the target image block and the image block to be matched, respectively.

[0097] The robust tracking results are used as prior information for stereo matching to realize the stereo matching process. Figure 5 As shown, assuming that the two-dimensional coordinates of the fragment P in the left camera's i-1, i, i+1 frames are x i-1 ,x i ,xi+1 , the two-dimensional coordinates of P in the i-1, i, i+1 frames of the right camera are x i-1 ',x i ',x i+1 '. Then it is necessary to find the left and right correspondence in the epipolar area and use the adaptive window to achieve image block matching. Under normal circumstances, if x i The block area centered with x i If the normalized mutual correlation coefficient of the block area centered on ' is greater than a certain threshold, the fragment P of the i-th frame is matched between the left and right cameras. However, when multiple similar fragment targets appear in the same area at the same time, it is easy to produce unreliable regional correlation coefficients, which can easily lead to mismatching. In this case, the block areas of the previous and next frames can be matched in the same way. If the sum of the correlations of the image blocks of three consecutive frames of the left camera and the image blocks of three consecutive frames of the right camera is greater than a certain threshold, the stereo matching of the i-th frame can be considered reliable.

[0098] 3. Three-dimensional reconstruction and motion state inversion of fragments using the overall beam method.

[0099] The exterior orientation elements of the high-speed camera and the three-dimensional coordinates of the target point can be calculated by bundle adjustment using control points arranged in the scene. The overall bundle adjustment method simultaneously solves the three-dimensional coordinates of the fragment target at different times in the sequence of images. Its core mathematical model is the collinearity condition equation:

[0100]

[0101] Among them, (X p ,Y p ,Z p ) represents the object coordinates of the target point, (x, y) represents the image plane coordinates of the target point, (Δx, Δy) represents the correction number of the image plane coordinates, f represents the principal distance of the camera, (X S ,Y S ,Z S ) represents the camera's exterior orientation line element, a i ,b i ,c i (i∈[1,3]) is a function of the three exterior azimuth angle elements.

[0102] In the overall bundle adjustment, the coordinates of the control points are considered as true values, and the three-dimensional coordinates of the target points and the exterior orientation parameters of the camera are considered as unknown values. Therefore, the observation equation after linearization of formula (4) is:

[0103]

[0104] Where V is the residual matrix, t is the camera exterior orientation matrix, A is the parameter matrix of matrix t, X is the matrix consisting of target point coordinate corrections, B is the parameter matrix of matrix X, and L is the constant term in the error equation. The Levenberg-Marquardt algorithm is used to solve it and obtain the camera exterior orientation matrix and the 3D coordinates of the target point.

[0105] In order to speed up the calculation of the three-dimensional coordinates of the target point, the three-dimensional coordinates of each target tracking point are obtained by the forward intersection algorithm based on the collinearity equation under the premise of iteratively analyzing the internal and external parameters of each camera. The collinearity condition equation (4) can be transformed into:

[0106]

[0107] The above formula can be rearranged as:

[0108]

[0109] Among them, the coefficients are expressed as follows:

[0110]

[0111] Based on formula (8), four linear equations can be listed for a pair of image matching points with the same name, and the three-dimensional spatial coordinates of the fragment target point can be calculated through indirect adjustment.

[0112] After obtaining the three-dimensional coordinates of the fragment target in the entire image sequence, the low-pass filtering method is used to reduce the random error of the coordinate calculation. Then, the displacement, velocity, and acceleration parameters of the fragment target are further calculated based on the filtered coordinates to describe the motion state of the fragment target.

[0113] The following experiments verify and analyze the effect of this solution.

[0114] (1) Experimental scenario

[0115] To verify the algorithm presented above, a video measurement system consisting of two high-speed cameras was used to film the fragments in flight from the side. A variable light source was used to simulate illumination changes, and a swaying curtain was used to simulate dynamic background changes. The high-speed camera image size was 1280 × 1024 pixels, and the frame rate was set to 500 frames per second. A synchronization controller was used to ensure synchronization of the two camera images. The camera baseline was 2.5 meters, and the camera angle was approximately 30 degrees. The test scene was as follows: Figure 6 shown.

[0116] (2) Measurement results and accuracy verification

[0117] The provided method is used to detect and track fragment targets in the sequence images. From the time when all fragment targets exist in the image to the time when the fragments disappear in the last frame, a total of 45 frames are used, and the actual number of fragments fired is 33. In order to verify the accuracy of fragment detection, all sequence images are identified by visual method. The number of fragment targets automatically detected by the algorithm through visual recognition method is correct, which is TP (True Positive), the number of fragment targets automatically detected by the algorithm through visual recognition method is incorrect, which is FP (False Positive), and the number of fragments in the image obtained by visual method but not detected is FN (False Negative). The performance of the fragment target detection algorithm is evaluated by accuracy (Precision) and recall (Recall). The calculation formula of detection accuracy is shown in (9), and the calculation formula of detection recall is shown in (10).

[0118]

[0119]

[0120] The detection status of all fragments in the 45-frame sequence image as well as the accuracy and recall rate of target detection are shown in Table 1.

[0121] Table 1 Evaluation indicators for fragment detection in sequence images

[0122]

[0123] Take the image taken by high-speed camera A as an example, Figure 7 The detection results of the first four frames of high-speed camera A are shown. The red circles are the detected fragment targets.

[0124] In order to evaluate the fragment tracking effect, the number of fragment targets on the image is obtained by visual recognition method and is taken as the true value GT = TP + FN = 896. In the trajectory obtained by the tracking algorithm, the sum of the number of correctly tracked points is TP2, the sum of the number of incorrectly tracked points is FP2, and the sum of the number of untracked points is FN2. The sum of the number of tracking point ID switches on each trajectory is IDSw. Among them, the multi-target tracking accuracy MOTA = 1-(FP2+FN2+IDSw) / GT, Recall = TP2 / (TP2+FN2). The tracking accuracy indicators are shown in Table 2. The tracking results of high-speed camera B are shown in Figure 8 As shown, circles of different colors represent the trajectories corresponding to different fragments.

[0125] Table 2 Fragment tracking accuracy in sequence images

[0126]

[0127] After obtaining the two-dimensional coordinates and correspondence of the fragments on a single sequence of images, the correspondence between the fragment groups of the two cameras is obtained using an automatic stereo matching method of the fragment groups based on tracking priors. Figure 9 The stereo matching results for A and B are shown below. Stereo matching accuracy is measured by the number of correctly matched fragments (CM) and the number of incorrectly matched fragments (MM). In the image sequence, the number of CM and MM is 815 and 68, respectively. The fragment stereo matching accuracy = CM / CM + MM, which is 92.3%.

[0128] In order to evaluate the accuracy of the 3D reconstruction of the fragments, the 3D coordinates of the marker points were measured using a total station for accuracy assessment. Figure 10 The control field, control point distribution and numbering diagram are as follows. In order to evaluate the measurement accuracy of video measurement, points 2, 5, 9 and 12 are used as check points, and the remaining points are used as control points for bundle adjustment.

[0129] After obtaining the camera's internal and external orientation elements, a stereo measurement system was formed using high-speed cameras A and B. The differences between the measurement results of points 2, 5, 9, and 12 and the total station coordinates are shown in Table 3. The root mean square errors (RMSE) in the three directions were 0.839 mm, 0.515 mm, and 0.398 mm, respectively, all less than 1 mm.

[0130] Table 3 Coordinate differences between total station and video measurement at check points

[0131]

[0132] In summary, in order to solve the problem of non-contact measurement of the three-dimensional motion parameters of the full sequence of high-speed fragments, this application has constructed a set of full-sequence fragment 3D reconstruction methods. By using the adaptive Gaussian background modeling method to fuse the illumination factor and combining it with the three-frame difference method, rapid detection of massive fragment targets is achieved. By combining the fragment trajectory information with the epipolar constraint, robust matching of fragment targets under weak texture background is achieved. The camera pose is optimized using the overall bundle adjustment, and then the fragment targets in the sequence images are quickly reconstructed in three dimensions using forward intersection, achieving high-precision three-dimensional reconstruction and motion parameter inversion of the fragment targets.

[0133] Example 2

[0134] Based on Example 1, this embodiment provides a high-speed video measurement device for full-sequence three-dimensional motion parameters of fragment targets, including:

[0135] A variable background curtain 1, with a variable background light source 2 provided on one side;

[0136] A binocular high-speed camera 3 is arranged on the other side of the variable background curtain 1;

[0137] The synchronous control system 4 includes a memory and an actuator, and the memory stores instructions for executing the above-mentioned high-speed video measurement method for the full sequence three-dimensional motion parameters of the fragment target.

[0138] Example 3

[0139] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the above-mentioned method for high-speed video measurement of three-dimensional motion parameters of a full sequence of fragment targets.

[0140] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for high-speed video measurement of full-sequence three-dimensional motion parameters of fragment targets, characterized by: The steps include: Collect fragment sequence images taken by binocular high-speed cameras and perform filtering processing; A Gaussian background modeling method based on adaptive learning rate is used to obtain the two-dimensional pixel coordinates of fragments in the fragment sequence image; By estimating the clutter rate and average detection probability, a fragment tracking trajectory is generated, and the Hungarian algorithm is used to achieve cascade matching of fragments under the same high-speed camera viewing angle. Based on the adaptive matching window size and the epipolar geometry constraints of the binocular high-speed camera, the fragment stereo matching is achieved; The bundle method is used to construct the collinearity condition equations, and the three-dimensional coordinates of the fragments in the sequence images are obtained through indirect adjustment calculation. The process of generating the fragment tracking trajectory includes: The initial tracking area of ​​the fragment movement is obtained by trajectory splicing, the clutter rate and average detection probability are calculated using the CPHD filter, and the fragment tracking trajectory is generated using the GLMB filter. The fragment stereo matching process includes the following steps: Based on the epipolar geometry constraints of the binocular high-speed camera, the matching search area of ​​the fragments under different viewing angles is obtained; In the image block matching process of a single frame of fragments, the result of cascade matching is used as the prior information for stereo matching. The matching window size is adaptively updated through the matching window information entropy to achieve stereo matching of fragments.

2. The method for high-speed video measurement of three-dimensional motion parameters of a fragment target in full sequence according to claim 1 is characterized in that: The filtering process includes: The fragment sequence images are processed by filtering with a two-dimensional discrete zero-mean Gaussian function; Top hat filtering is used to process fragment sequence images.

3. The method for high-speed video measurement of three-dimensional motion parameters of a fragment target in a full sequence according to claim 1 is characterized in that: The Gaussian background modeling method specifically comprises the following steps: A background sample model based on the global illumination variation factor is used as the demarcation threshold to determine foreground and background pixels, and the two-dimensional pixel coordinates of the fragments are extracted through the three-frame difference method. The learning rate of the sample model is adaptively determined based on the difference between the target pixel and the adjacent pixels in the current frame and the background model.

4. The method for high-speed video measurement of three-dimensional motion parameters of a fragment target in full sequence according to claim 3 is characterized in that: The learning rate is calculated using the following formula: in, k is the learning rate, a , b is a constant, f Indicates the current frame, is the difference between the current frame pixel and its neighboring pixels and the background model, F is the number of image frames in the sequence.

5. The method for high-speed video measurement of full-sequence three-dimensional motion parameters of fragment targets according to claim 1 is characterized in that: After the cascade matching, the following steps are also included: Trajectories whose diffusion angle exceeds a preset value or whose flight direction is opposite to / not consistent with the overall diffusion direction are screened out and / or trimmed.

6. The method for high-speed video measurement of full-sequence three-dimensional motion parameters of fragment targets according to claim 1, characterized in that: After the fragment stereo matching, a reliability verification process is also included, including the following steps: The image block areas of the previous and next frames are matched. If the sum of the correlations between the image blocks of three consecutive frames under one foot of the high-speed camera and the image blocks of three consecutive frames under the right camera is greater than a preset threshold, the stereo matching of the current frame is considered reliable.

7. A high-speed video measurement device for the full sequence three-dimensional motion parameters of fragment targets, characterized by: include: A variable background curtain (1) with a variable background light source (2) provided on one side; A binocular high-speed camera (3) is arranged on the other side of the variable background curtain (1); A synchronous control system (4) comprises a memory and an actuator, wherein the memory stores instructions for executing the method for high-speed video measurement of full-sequence three-dimensional motion parameters of a fragment target according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The method comprises one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the method for high-speed video measurement of full-sequence three-dimensional motion parameters of a fragment target as described in any one of claims 1 to 6.

Images