A Method and System for Measuring the Model State in a Water Landing Test

Abstract

The present invention discloses a method and system for measuring the state of a model in a water landing test, belonging to the technical field of helicopter water landing. The method includes marking feature points on a model to be measured, recording the model's landing process, and simultaneously capturing the model with two high-speed cameras to obtain a continuous stereo image pair of the landing process. The method then detects the pixels of the feature points in each image of the stereo image pair and performs bidirectional verification to obtain a pair of feature points that pass the bidirectional verification. The method then calculates the parallax value of the feature point in the stereo image pair to obtain depth information for the feature point. The two-dimensional pixel coordinates of the feature point in the stereo image pair are converted into world space coordinates to obtain the three-dimensional coordinates of each feature point. Based on the obtained three-dimensional coordinates of each feature point, the model's posture changes are deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during motion. This method can effectively improve the accuracy and reliability of test data measurement.

Description

Technical Field

[0001] The present invention relates to the technical field of helicopter forced landing on water, and more particularly to a method and system for measuring a model state in a forced landing test. Background Art

[0002] A ditching refers to an emergency landing process that needs to be carried out when a helicopter is flying over the water due to an emergency such as engine failure or running out of fuel. It requires the helicopter to be able to land safely on the water and ensure that the crew and passengers can be evacuated safely.

[0003] With the widespread use of helicopters in water, higher requirements are being placed on their water landing performance. Since the helicopter water landing process involves the interaction between fluid mechanics and structural mechanics, numerical simulation analysis is very difficult. Experimental methods are a more suitable method for studying helicopter water landing problems, as they can obtain more effective and accurate test data.

[0004] Currently, helicopter ditching tests are plagued by high costs, high risks, and limited flexibility. Most methods rely on scaled-down models instead of real helicopters for ditching tests. This test primarily captures information such as the scaled-down model's speed, attitude, and floating time. This allows for analysis of the helicopter's ditching performance, including its surface floating motion patterns and optimal ditching procedures. This allows for the definition of the range of motion parameters that ensure the helicopter does not capsize, which is of great significance to research on ditching tests.

[0005] During helicopter scale model water landing tests, photoelectric encoders, gyroscopes, and other sensors installed on the model provide test data on the model's state. This data is then processed and analyzed to study the helicopter's water landing performance. The helicopter scale model water landing tests place high demands on sensors, requiring them to be highly accurate, waterproof, drop-resistant, and lightweight to obtain more realistic test data.

[0006] Since the sensor will inevitably be damaged during the scaled model water immersion test, the measurement data will have a large deviation, which will affect the measurement accuracy and efficiency of the data during the scaled model water immersion test. Summary of the Invention

[0007] In response to the problems existing in the above-mentioned fields, the present invention proposes a method and system for measuring the state of a model in a forced landing test. By determining the depth information of feature points, the three-dimensional information of each feature point during the model's motion is obtained. The depth information can help distinguish the foreground and background, and improve the recognition accuracy of significant areas. It is more comprehensive than traditional two-dimensional feature point tracking, and helps to more accurately analyze the motion trajectory and posture changes of the model during the landing process.

[0008] To solve the above technical problems, the present invention discloses a method for measuring the state of a model in a ditching test, comprising the following steps:

[0009] The feature points of the model to be measured are marked, and the model's water entry process is recorded. Two high-speed cameras are used to simultaneously capture the process, obtaining a continuous stereo image pair of the water entry process.

[0010] Detecting pixels of feature points of each image in the stereo image pair and performing bidirectional verification to obtain feature point pairs that pass bidirectional verification;

[0011] According to the feature point pair that has passed the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair;

[0012] According to the depth information of the feature point, the two-dimensional pixel coordinates of the feature point in the stereo image pair are converted into world space coordinates to obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the posture change of the model is deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during motion.

[0013] Preferably, obtaining the depth information of the feature point comprises the following steps:

[0014] Read the stereo image pair taken by two high-speed cameras simultaneously and perform stereo correction on it;

[0015] Calculating a disparity map of the corrected stereo image pair using a stereo matching algorithm. The disparity map represents the pixel differences between corresponding feature points of the stereo image pair. Optimizing the stereo matching algorithm by adjusting the disparity level and the size of the matching block.

[0016] According to the disparity map, the disparity value of the feature point in the stereo image pair is converted into the depth information of the feature point by multiplying it by the baseline length between the high-speed cameras and dividing it by the pixel spacing.

[0017] Preferably, obtaining the three-dimensional coordinates of each feature point includes the following steps:

[0018] The calculation formula for depth information is:

[0019]

[0020] Where B is the baseline distance between cameras, f is the focal length of the cameras, d is the parallax, and D is the depth;

[0021] According to the depth information of the feature points, the two-dimensional image coordinates are converted into coordinates in the three-dimensional camera coordinate system:

[0022]

[0023] Z c =D

[0024] Among them, (u,v) is the pixel coordinate, (X c ,Y c ,Z c ) is the camera coordinate;

[0025] The conversion formula for converting 3D camera coordinates to world space coordinates is:

[0026]

[0027] Among them, (u,v) is the pixel coordinate, (x,y) is the image coordinate, (X c ,Y c ,Z c ) is the camera coordinate, (X w ,Y w ,Z w ) are world space coordinates, is the camera internal parameter, is the camera external parameter, d x is the change in the image coordinates of the feature point in the x direction, d y is the change in the image coordinates of the feature point in the y direction.

[0028] Preferably, constructing the rotation matrix comprises the following steps:

[0029] According to the obtained three-dimensional coordinates of each feature point, the three-dimensional coordinates of all feature points in the model movement process are summarized to determine the three-dimensional point cloud of the model movement process;

[0030] By classifying the 3D point cloud, the 3D coordinates of the same feature point at different times are grouped into one data set, and m data sets are obtained, that is, the number of feature points is m, m = 1, 2, ..., j, ..., m;

[0031] Take any dataset j in m and get the spatial coordinates of the jth feature point at different times as p1, p2, ..., p n , where p i =(x i ,y i ,z i ) T Represents the three-dimensional coordinates of the feature point on the image at time i;

[0032] For each pair of adjacent positions p i and p i+1 , by calculating the centroid of each pair of adjacent positions and performing de-centroiding, the adjacent positions p are calculated i and pi+1 The corresponding center of mass coordinate c i and c i+1 They are:

[0033]

[0034] For adjacent position p i and p i+1 The corresponding centroid coordinates are de-centred to obtain the adjacent position p i and p i+1 Relative to the center of mass coordinate c i and c i+1 The decentralized position q i and q i+1 They are:

[0035] q i =p i -c i ;

[0036] q i+1 =p i+1 -c i+1 ;

[0037] Get the de-centroided position q i and q i+1 The covariance matrix H i for:

[0038]

[0039] The obtained de-centroided position q i and q i+1 The covariance matrix H i , as the covariance matrix of the j-th feature point;

[0040] By performing singular value decomposition on the covariance matrix of the j-th feature point, the rotation matrix R of the feature point at different times is obtained. i :

[0041] H i =U i ·S i ·V i T ;

[0042]

[0043] Among them, V i and U i Represents the orthogonal matrix corresponding to the adjacent de-centroided positions of the feature point, S i Represents the diagonal matrix corresponding to the feature point; when det(R i )=-1, Vi The last column of the R i is a valid rotation matrix.

[0044] Preferably, the obtaining of dynamic state parameters of the model during motion specifically includes:

[0045] The dynamic state parameters include attitude parameters and speed parameters, wherein the attitude parameters include pitch angle, yaw angle, and roll angle;

[0046] Convert the rotation matrix to Euler angles and decompose the rotation matrix R in ZYX order i , and the rotation angles around the Z, Y, and X axes are obtained, which are the yaw angles α i , pitch angle β i and roll angle γ i ;

[0047] When the solved rotation matrix is When , we get the rotation matrix R i The calculation formula for the converted Euler angle is:

[0048]

[0049] β i =arctan(-r 31 );

[0050]

[0051] Calculate the instantaneous velocity of the jth feature point through the displacement and time of adjacent moments;

[0052] The instantaneous velocity calculation formula for each feature point is:

[0053]

[0054] Where Δp i is the displacement change corresponding to the time change Δt, and the frame rate of the camera is F;

[0055] The speed of the model at this time is expressed as the average speed of adjacent points;

[0056] The calculation formula of model speed is:

[0057]

[0058] Where N is the number of feature points for calculating velocity;

[0059] Similarly, the data set corresponding to each feature point is solved, and finally the dynamic state parameters of the model during motion are output.

[0060] Preferably, obtaining the feature point pairs that pass the bidirectional verification comprises the following steps:

[0061] Detect the feature points of each image through the target detection algorithm, automatically detect and identify multiple feature points in the stereo image pair, and determine the position information corresponding to each feature point;

[0062] Based on the ORB algorithm, each detected feature point is described and the corresponding feature points with the same BRIEF descriptor are preliminarily matched. For each detected feature point, the ORB algorithm calculates the intensity center of the surrounding pixels, also known as the centroid, and uses the calculated centroid as the direction of the feature point.

[0063] The descriptor is constructed by comparing the brightness of a pair of pixels around the feature point. For each feature point, the ORB algorithm randomly selects multiple pixel pairs and generates a binary value for each pair of pixels. These binary values are concatenated to form a long string, namely the BRIEF descriptor.

[0064] For each pair of matched feature points, a bidirectional verification is performed to obtain a pair of feature points that pass the bidirectional verification by checking whether the matching results of the feature points in each other's images are consistent.

[0065] Preferably, it also includes:

[0066] The dynamic state parameters of the model during motion collected by the high-speed camera are denoted as model state parameters 1;

[0067] The image of the model's landing process is collected by the sensor, and the dynamic state parameters of the model during the motion process based on the sensor acquisition are obtained, which are recorded as model state parameters 2;

[0068] If the model state parameter 1 fits the model state parameter 2, the fitting result is determined as the final model state parameter output.

[0069] Preferably, a model state measurement system for a ditching test is further included, comprising:

[0070] The model data acquisition module is used to mark the feature points of the model to be measured and record the model's water entry process. Two high-speed cameras are used to simultaneously capture the model and obtain a continuous stereo image pair of the water entry process.

[0071] A feature point depth information acquisition module is used to detect the pixels of the feature points of each image in the stereo image pair and perform bidirectional verification to obtain feature point pairs that pass the bidirectional verification; based on the feature point pairs that pass the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair;

[0072] The model state parameter acquisition module is used to convert the two-dimensional pixel coordinates of the feature point in the stereo image pair into world space coordinates based on the depth information of the feature point, and obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the model's posture changes are deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during motion.

[0073] Preferably, a computer device is further included, the computer device comprising a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, causing the processor to perform the following steps:

[0074] The feature points of the model to be measured are marked, and the model's water entry process is recorded. Two high-speed cameras are used to simultaneously capture the process, obtaining a continuous stereo image pair of the water entry process.

[0075] Detecting pixels of feature points of each image in the stereo image pair and performing bidirectional verification to obtain feature point pairs that pass bidirectional verification;

[0076] According to the feature point pair that has passed the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair;

[0077] According to the depth information of the feature point, the two-dimensional pixel coordinates of the feature point in the stereo image pair are converted into world space coordinates to obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the posture change of the model is deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during motion.

[0078] Preferably, the present invention further comprises a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the following steps:

[0079] The feature points of the model to be measured are marked, and the model's water entry process is recorded. Two high-speed cameras are used to simultaneously capture the process, obtaining a continuous stereo image pair of the water entry process.

[0080] Detecting pixels of feature points of each image in the stereo image pair and performing bidirectional verification to obtain feature point pairs that pass bidirectional verification;

[0081] According to the feature point pair that has passed the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair;

[0082] According to the depth information of the feature point, the two-dimensional pixel coordinates of the feature point in the stereo image pair are converted into world space coordinates to obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the posture change of the model is deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during motion.

[0083] Compared with the prior art, the present invention has the following beneficial effects:

[0084] The present invention proposes a method for measuring the state of a model in a forced landing test. This method addresses the deficiencies in the sensor's measurement data during the test. By marking the model's feature points and recording the model's landing process, a continuous pair of stereo images of the landing process is obtained. The pixels of the feature points of each image in the stereo image pair are detected, and each detected feature point is further matched and bidirectionally verified. The bidirectional verification can significantly improve the accuracy and robustness of feature point matching. This verification mechanism can effectively eliminate incorrectly matched point pairs, further ensuring the reliability and consistency of the matching results and providing high-quality input data for subsequent three-dimensional point cloud reconstruction. Based on the acquired bidirectionally verified feature point pairs, the depth information of the feature points is determined to obtain the three-dimensional coordinate information of each feature point during the model's motion. The depth information can help distinguish the foreground and background, improve the recognition accuracy of significant areas, and is more comprehensive than traditional two-dimensional feature point tracking. The obtained three-dimensional coordinate information helps to more accurately analyze the model's motion trajectory and posture changes during the landing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 This is a flow chart of the model state measurement method in the ditching test proposed by the present invention;

[0086] Figure 2 The framework of the model state measurement method in the ditching test proposed by the present invention;

[0087] Figure 3 A schematic diagram of the relative positions of the high-speed camera and the model provided by the present invention;

[0088] Figure 4 Schematic diagram of the model feature points provided by the present invention;

[0089] Figure 5 Target detection flow chart provided by an embodiment of the present invention;

[0090] Figure 6 A flow chart of a dual-target stereo vision algorithm provided by an embodiment of the present invention;

[0091] Figure 7 This is a flow chart of the model state measurement algorithm provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0092] The following is a combination of the embodiments of the present invention Figure 1-Figure 7 , the technical solutions in the embodiments of the present invention are clearly and completely described. It should be understood that the terms used in the present invention are only used to describe specific implementation methods and are not intended to limit the present invention.

[0093] like Figure 1 As shown, the present invention proposes a method for measuring the state of a model in a ditching test, comprising the following steps:

[0094] S1: Mark the feature points of the model to be measured and record the model's water entry process. Use two high-speed cameras to simultaneously capture the process and obtain a continuous stereo image pair of the water entry process.

[0095] S2: Detect the pixels of the feature points of each image in the stereo image pair and perform bidirectional verification to obtain feature point pairs that pass the bidirectional verification;

[0096] S3: Based on the feature point pair that has passed the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair;

[0097] S4: Based on the depth information of the feature point, the two-dimensional pixel coordinates of the feature point in the stereo image pair are converted into world space coordinates to obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the posture change of the model is deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during the movement process.

[0098] Specifically, in step S1, obtaining a pair of continuous stereo images of the water landing process includes:

[0099] Based on the conventional test conditions of the helicopter scale model ditching test, two high-speed camera types that meet the test requirements are selected;

[0100] The high-speed camera is installed at the end of the test pool. Based on the test site environment, the focal length, angle and position of the high-speed camera are finely adjusted through measurement and calculation to capture the entire process of the model entering the water.

[0101] Adjust the lighting conditions of the test site by using additional lighting equipment or adjusting the occlusion of natural light, calibrate the intrinsic and extrinsic parameters of the high-speed camera, and calibrate the two high-speed cameras, where the intrinsic parameters include focal length, optical center and distortion coefficient, and the extrinsic parameters include rotation matrix and translation vector;

[0102] Debug two high-speed cameras and perform image acquisition to obtain continuous stereo image pairs of the water impact process;

[0103] Mark the feature points of the model, select areas with obvious features on the model as feature points; mark a certain number of feature points on the model.

[0104] In step S2, obtaining a pair of feature points that pass bidirectional verification includes the following steps:

[0105] Detect the feature points of each image through the target detection algorithm, automatically detect and identify multiple feature points in the image, and determine the location information corresponding to each feature point;

[0106] Based on the ORB algorithm, each detected feature point is described and the corresponding feature points with the same BRIEF descriptor are preliminarily matched. For each detected feature point, the ORB algorithm calculates the intensity center of the surrounding pixels, also known as the centroid, and uses the calculated centroid as the direction of the feature point.

[0107] The descriptor is constructed by comparing the brightness of a pair of pixels around the feature point. For each feature point, the ORB algorithm randomly selects multiple pixel pairs and generates a binary value for each pair of pixels. These binary values are concatenated to form a long string, namely the BRIEF descriptor.

[0108] For each pair of matched feature points, a bidirectional verification is performed to obtain a pair of feature points that pass the bidirectional verification by checking whether the matching results of the feature points in each other's images are consistent.

[0109] In step S3, the depth information of the feature point is obtained, including the following steps:

[0110] Read the stereo image pair taken by two high-speed cameras simultaneously and perform stereo correction on it;

[0111] The disparity map of the corrected stereo image pair is calculated using a stereo matching algorithm. The disparity map represents the pixel differences between corresponding feature points in the left and right images. The stereo matching algorithm is optimized by adjusting the disparity level and the size of the matching block.

[0112] According to the disparity map, the disparity value is converted into the depth information of the feature points of any image in the stereo image pair by multiplying it by the baseline length between the high-speed cameras and dividing it by the pixel spacing.

[0113] In step S4, the three-dimensional coordinates of each feature point are obtained, including the following steps:

[0114] The calculation formula for depth information is:

[0115]

[0116] Where B is the baseline distance between cameras, f is the focal length of the cameras, d is the parallax, and D is the depth;

[0117] According to the depth information of the feature points, the two-dimensional image coordinates are converted into coordinates in the three-dimensional camera coordinate system:

[0118]

[0119] Z c =D

[0120] Among them, (u,v) is the pixel coordinate, (X c ,Y c ,Z c ) is the camera coordinate;

[0121] The conversion formula for converting 3D camera coordinates to world space coordinates is:

[0122]

[0123] Among them, (u,v) is the pixel coordinate, (x,y) is the image coordinate, (X c ,Y c ,Z c ) is the camera coordinate, (X w ,Y w ,Z w ) are world space coordinates, is the camera internal parameter, is the camera external parameter, d x is the change in the image coordinates of the feature point in the x direction, d y is the change in the image coordinates of the feature point in the y direction.

[0124] Constructing a rotation matrix includes the following steps:

[0125] According to the obtained three-dimensional coordinates of each feature point, the three-dimensional coordinates of all feature points in the model movement process are summarized to determine the three-dimensional point cloud of the model movement process;

[0126] By classifying the 3D point cloud, the 3D coordinates of the same feature point at different times are grouped into one data set, and m data sets are obtained, that is, the number of feature points is m, m = 1, 2, ..., j, ..., m;

[0127] Take any dataset j in m and get the spatial coordinates of the jth feature point at different times as p1, p2, ..., p n , where p i =(x i ,y i ,z i ) T Represents the three-dimensional coordinates of the feature point on the image at time i;

[0128] For each pair of adjacent positions p i and p i+1 , by calculating the centroid of each pair of adjacent positions and performing de-centroiding, the adjacent positions p are calculated i and p i+1 The corresponding center of mass coordinate c i and c i+1 They are:

[0129]

[0130] For adjacent position p i and p i+1 The corresponding centroid coordinates are de-centred to obtain the adjacent position p i and p i+1 Relative to the center of mass coordinate c i and c i+1 The decentralized position q i and q i+1 They are:

[0131] q i =p i -c i ;

[0132] q i+1 =p i+1 -c i+1 ;

[0133] Get the de-centroided position q i and q i+1 The covariance matrix H i for:

[0134]

[0135] The obtained de-centroided position q i and q i+1 The covariance matrix H i , as the covariance matrix of the j-th feature point;

[0136] By performing singular value decomposition on the covariance matrix of the j-th feature point, the rotation matrix R of the feature point at different times is obtained. i :

[0137] H i =U i ·S i ·V i T ;

[0138]

[0139] Among them, V i and U iRepresents the orthogonal matrix corresponding to the adjacent de-centroided positions of the feature point, S i Represents the diagonal matrix corresponding to the feature point; when det(R i )=-1, V i The last column of the R i is a valid rotation matrix.

[0140] Obtain the dynamic state parameters of the model during motion, including:

[0141] The dynamic state parameters include attitude parameters and speed parameters, wherein the attitude parameters include pitch angle, yaw angle and roll angle;

[0142] Convert the rotation matrix to Euler angles and decompose the rotation matrix R in ZYX order i , and the rotation angles around the Z, Y, and X axes are obtained, which are the yaw angles α i , pitch angle β i and roll angle γ i ;

[0143] When the solved rotation matrix is When , we get the rotation matrix R i The calculation formula for the converted Euler angle is:

[0144]

[0145] β i =arctan(-r 31 );

[0146]

[0147] Calculate the instantaneous velocity of the jth feature point through the displacement and time of adjacent moments;

[0148] The instantaneous velocity calculation formula for each feature point is:

[0149]

[0150] Where Δp i is the displacement change corresponding to the time change Δt, and the frame rate of the camera is F;

[0151] The speed of the model at this time is expressed as the average speed of adjacent points;

[0152] The calculation formula of model speed is:

[0153]

[0154] Where N is the number of feature points for calculating velocity;

[0155] Similarly, the data set corresponding to each feature point is solved, and finally the dynamic state parameters of the model during motion are output.

[0156] In order to further verify the efficiency of the proposed model state measurement method in the ditching test, the present invention also installs necessary data sensors on the model to obtain images of the model's landing process through the data sensors.

[0157] The dynamic state parameters of the model during motion collected by the high-speed camera are denoted as model state parameters 1;

[0158] The image of the model's landing process is collected by the sensor, and the dynamic state parameters of the model during the motion process based on the sensor acquisition are obtained, which are recorded as model state parameters 2;

[0159] If the model state parameter 1 fits the model state parameter 2, the dynamic state parameters of the model during motion captured by the high-speed camera are optimized, and the fitting results are used as the final output model state parameters to further verify the accuracy of the test data collected by the sensor.

[0160] The present invention also proposes a model state measurement system for a ditching test, comprising:

[0161] The model data acquisition module is used to mark the feature points of the model to be measured and record the model's water entry process. Two high-speed cameras are used to simultaneously capture the model and obtain a continuous stereo image pair of the water entry process.

[0162] A feature point depth information acquisition module is used to detect the pixels of the feature points of each image in the stereo image pair and perform bidirectional verification to obtain feature point pairs that pass the bidirectional verification; based on the feature point pairs that pass the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair;

[0163] The model state parameter acquisition module is used to convert the two-dimensional pixel coordinates of the feature point in the stereo image pair into world space coordinates based on the depth information of the feature point, and obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the model's posture changes are deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during motion.

[0164] The proposed method sequentially detects, preliminarily matches, and bidirectionally verifies the pixels of feature points in each of the consecutive stereo image pairs during the immersion process, obtaining bidirectionally verified feature point pairs. Based on the bidirectionally verified feature point pairs, the disparity map of the stereo image pair is determined to obtain the depth information of the feature points. Based on the depth information and the 2D pixel coordinates of the feature points, the 2D image coordinates are converted to 3D camera coordinates, and then the 3D camera coordinates are converted to world space coordinates, obtaining the 3D information of the feature points during the model's motion. This method is more comprehensive than traditional 2D feature point tracking methods. The bidirectional verification significantly improves the accuracy and robustness of feature point matching. This verification mechanism effectively eliminates incorrectly matched point pairs, further ensuring the reliability and consistency of the matching results and providing high-quality input data for subsequent 3D point cloud reconstruction. By determining the depth information of the feature points, the 3D coordinate information of each feature point during the model's motion is obtained. This depth information helps distinguish foreground from background, improving the accuracy of identifying salient areas. This method is more comprehensive than traditional 2D feature point tracking, and the 3D coordinate information facilitates more accurate analysis of the model's motion trajectory and posture changes during immersion.

[0165] Example

[0166] like Figure 2 As shown, in order to verify the effectiveness of the method proposed in the present invention, the proposed model state measurement method in the forced landing test is compared and analyzed with the embodiment provided by the present invention. This embodiment takes a model posture tracking method in the forced landing test as an example, and specifically includes the following steps:

[0167] Step 1: Install test equipment, conduct ditching tests, and collect test data;

[0168] Based on the conventional test conditions of the helicopter scale model water landing test, a high-speed camera type that meets the test requirements is selected; a high-speed camera is installed at the terminal position of the test pool. Combined with the test site environment, the camera's focal length, angle, and position are finely adjusted through measurement and calculation to ensure that the coverage of the two high-speed cameras can be seamlessly connected to jointly capture the entire landing process of the model. The relative position diagram of the high-speed camera and the model is shown in the figure below. Figure 3As shown in Figures (a) and (b) in the figure, Figure (a) is a front schematic diagram of the relative position of the high-speed camera and the model, and Figure (b) is a side schematic diagram of the relative position of the high-speed camera and the model; the lighting conditions of the test site are adjusted by using additional lighting equipment or adjusting the occlusion of natural light to ensure that the high-speed camera can capture high-quality images in various lighting environments; the two high-speed cameras are calibrated by accurately calibrating the camera's intrinsic parameters (focal length, optical center, distortion coefficient) and extrinsic parameters (rotation matrix and translation vector) to ensure that their positioning and orientation in space are consistent, thereby ensuring the geometric consistency between the dual-view images; the two high-speed cameras are connected to a computer terminal and debugged to ensure that the two high-speed cameras can simultaneously capture images and transmit the data in real time to the terminal for storage and subsequent processing.

[0169] Mark the feature points of the scaled model, and the feature points are distributed as follows Figure 4 As shown; select areas with obvious features on the model as feature points, which are easy to identify in the image; mark a sufficient number of feature points on the model pattern without too much concentration or overlap in the image to ensure that the model status can be accurately tracked from different angles and distances; use high-contrast and high-stability marking materials to ensure that it can maintain traceability even in high-speed movement and complex backgrounds, and prevent the markers from falling off or deforming due to water impact or model movement during the test, thereby ensuring the continuity and reliability of data collection.

[0170] Step 2: Based on the target detection algorithm, process the image information collected by the camera. The flow chart is as follows: Figure 5 As shown, the specific steps include:

[0171] Step 2.1: Perform image preprocessing. You can grayscale, remove noise, sharpen, resize, crop, and perform other operations on the noise and bad images that may be contained in the captured multi-view image set. This will improve the image quality and enhance the robustness of the image processing algorithm to different conditions (such as lighting changes, perspective changes, etc.), and prepare feature point data for the subsequent dual stereo vision algorithm.

[0172] Step 2.2: Obtain the corresponding depth map. The depth map provides additional spatial information to the RGB image, which helps improve the performance of saliency detection, especially in complex backgrounds or under different lighting conditions. Depth information can help distinguish the foreground from the background and improve the recognition accuracy of salient areas. First, read the stereo image pair captured by the left and right cameras; then, stereo rectification is performed on the image pair to remove lens distortion and align the images; then, a stereo matching algorithm (such as PSMNet) is used to calculate the disparity map, which represents the pixel difference between corresponding points in the left and right images; then, a depth map is generated based on the disparity map. This step usually involves converting the disparity value into depth information by multiplying the baseline length between the high-speed cameras and dividing by the pixel spacing; during this process, the parameters of the stereo matching algorithm, such as the number of disparity levels and the size of the matching block, need to be adjusted to optimize the results; then, the generated depth map is subjected to bilateral filtering to remove noise and fill discontinuous areas.

[0173] Step 2.3: To optimize the model structure and improve the ability to detect feature points, a deep learning model with four key modules is provided. The specific content of each module is as follows:

[0174] Module 1: Building a backbone network. The backbone network can capture the high-level semantic information and spatial structure of the image, providing a strong feature foundation for subsequent fusion and detection. Convolutional neural networks (CNNs) such as ResNet and Transformer architectures such as VisionTransformer (ViT) have proven their powerful performance in image processing.

[0175] Module 2: Considering the heterogeneous representation problems between different modalities, a cross-modal fusion module is designed that can handle these differences and utilize the correlation and complementarity between different modalities; attention mechanisms or feature alignment techniques can be used to enhance the fusion of features from different modalities, such as using dynamic weight fusion or channel attention to emphasize the features of salient targets.

[0176] Module 3: To enhance the model's adaptability to scale changes, a multi-scale fusion module is used that can integrate features extracted at different scales. Feature Pyramid Network (FPN) or similar structures can be used to achieve cross-level feature fusion, which can retain more detailed information and improve the detection ability of small targets.

[0177] Module 4: Select a suitable decoder. The decoder needs to convert the fused features into a saliency map, which usually involves upsampling and feature reconstruction. Therefore, a transposed convolution operation is used to gradually amplify the feature map while retaining spatial and semantic information.

[0178] To optimize model performance, a combination of multiple loss functions, such as Intersection over Union (IoU) loss and Non-Corrupted Encoding (NCE) loss, can be used to simultaneously optimize the model's localization accuracy and discrimination capabilities. IoU is a metric that measures the similarity between predicted results and true labels and is widely used in semantic segmentation tasks. IoU loss helps the model better learn the boundaries of salient objects, thereby improving detection accuracy. NCE loss is a method for learning the probability distribution that distinguishes positive and negative samples and is commonly used in metric learning and other tasks that require distinguishing between different categories.

[0179] At the same time, the training strategy can be changed. During the training process, data enhancement techniques such as random cropping, rotation, and scaling can be used to improve the model's generalization ability for small markers. Considering the difficulty of detecting small targets, focal loss or similar techniques can be used to balance the category imbalance between positive and negative samples.

[0180] Step 3: After improving the accuracy of feature point detection through target detection algorithm, use binocular stereo vision algorithm to construct three-dimensional space, obtain the three-dimensional space coordinate information of feature points, and then use model state algorithm to calculate the state change of the model. The flow chart is as follows: Figure 6 As shown, it specifically includes the following steps:

[0181] Step 3.1: Use two cameras to capture images of the same scene from different angles.

[0182] Step 3.2: Determine the intrinsic parameters (focal length, principal point, distortion coefficient) and extrinsic parameters (position and orientation) of each camera.

[0183] Step 3.3: Detect the feature points of each image, i.e., target detection. This method can automatically detect and identify multiple objects in the image without human intervention and provide accurate location information of the objects. Deep learning-based methods can achieve real-time or near-real-time detection and can adapt to different lighting conditions, background noise, and changes in viewing angles, with strong robustness.

[0184] Step 3.4: Use the ORB (OrientedFAST and RotatedBRIEF) algorithm to describe the feature points, perform a preliminary match on corresponding feature points with the same BRIEF descriptor, and use bidirectional verification to improve the accuracy of the match. For each detected feature point, the ORB algorithm calculates the intensity center (also known as the centroid) of the surrounding pixels. This centroid is used as the orientation of the feature point, making ORB invariant to rotation. BRIEF is a binary descriptor that constructs a descriptor by comparing the brightness of a pair of pixels around the key point. For each key point, ORB randomly selects multiple pixel pairs and generates a binary value (0 or 1) for each pair, indicating which pixel in the pair has a higher brightness. These binary values are concatenated to form a long string, namely the BRIEF descriptor. On the basis of preliminary matching, bidirectional verification is performed, that is, for each pair of matched feature points, it is checked whether their matching results in each other's images are consistent; only those feature point pairs that pass bidirectional verification are considered reliable matching pairs and are ultimately retained as matched feature point pairs; bidirectional verification can significantly improve the accuracy and robustness of feature point matching. Through this verification mechanism, incorrectly matched point pairs can be effectively eliminated, further ensuring the reliability and consistency of the matching results, and providing high-quality input data for subsequent 3D point cloud reconstruction.

[0185] Step 3.5: Use a binocular stereo vision algorithm to obtain the 3D coordinates of the matching points. The 3D coordinates of all matching points are combined to form a preliminary 3D point cloud of the scene. Two high-speed cameras are used to capture the same scene from different angles. By comparing corresponding points in the two images, the disparity between these points is calculated. Disparity is the difference in horizontal distance between the projections of the same spatial point in the two images. Using the disparity and the baseline distance between the high-speed cameras, the depth information of each corresponding point is calculated using triangulation principles.

[0186] After the depth information of the feature points is obtained, the two-dimensional image coordinates can be converted into coordinates in the three-dimensional camera coordinate system, that is, the coordinates of the feature points in the three-dimensional space.

[0187] Step 3.6: Use deep learning algorithms and statistical filtering technology to optimize the preliminary 3D point cloud to form a complete 3D point cloud.

[0188] Using deep learning algorithms, especially convolutional neural networks (CNN) or graph convolutional networks (GCN), to analyze and process preliminary 3D point clouds can learn complex feature representations from point cloud data, thereby improving the quality and accuracy of point clouds.

[0189] Applying statistical filtering techniques, such as Gaussian filtering or bilateral filtering, to smooth the point cloud to reduce noise and preserve the edge features of the point cloud can help identify and remove outliers that have a significantly different density from the surrounding point cloud.

[0190] The point clouds processed by deep learning algorithms and statistical filtering techniques are fused to form a more complete and consistent 3D point cloud. During the fusion process, clustering algorithms or surface reconstruction techniques, such as Poisson surface reconstruction or Marching Cubes algorithm, may be used to generate a smoother and more continuous surface.

[0191] Step 4: Process the three-dimensional coordinate information of the feature points based on the model state measurement algorithm and output the model state parameter 1. The flow chart is as follows: Figure 7 As shown, the specific steps include:

[0192] Step 4.1: Classify the 3D point cloud, that is, classify the 3D coordinates of the same feature point at different times into one data set, and then m (the number of feature points) data sets can be obtained.

[0193] Step 4.2: Calculate the centroid of each pair of adjacent positions and perform de-centroiding

[0194] Calculate the centroid coordinates for each pair of adjacent positions, and then perform de-centroiding after obtaining the centroid coordinates to obtain the de-centroided position corresponding to the adjacent position;

[0195] Step 4.3: Construct a rotation matrix to deduce the pose change of the model.

[0196] Get the covariance matrix of the de-centroided position and use it as the covariance matrix of the j-th feature point;

[0197] By performing singular value decomposition on the covariance matrix of the j-th feature point, the rotation matrix of the feature point at different times is obtained.

[0198] Step 4.4: Convert the rotation matrix into Euler angles to represent the pose change of the model.

[0199] Assume that the rotation matrix R is decomposed in ZYX order i , and the rotation angles around the Z, Y, and X axes are obtained, which are the yaw angles α i , pitch angle β i , roll angle γ i .

[0200] Step 4.5: Determine the instantaneous velocity of each feature point by calculating the instantaneous velocity of the point using the displacement and time of adjacent moments. The velocity of the model at that moment is expressed as the average velocity of the adjacent points.

[0201] Step 4.6: Repeat steps 4.2 to 4.5 in step 4 above to solve each feature point data set, analyze and process the solutions of all feature points, and finally obtain the dynamic state parameters of the model during motion, that is, model state parameter 1.

[0202] Step 5: Based on the data collected by the sensor, the data is processed to obtain the model state parameter 2;

[0203] The model state parameter 2 is compared and analyzed with the model state parameter 1, and the model state parameter 1 is optimized. By judging whether the model state data is fitted, the final model state parameter is output when it is fitted, and the accuracy of the test data collected by the sensor is verified at the same time.

[0204] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

[0205] In addition, unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those skilled in the art to which the present invention belongs. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods related to the documents. In the event of any conflict with any incorporated document, the content of this specification shall prevail.

Claims

1. A method for measuring the state of a model in a ditching test, characterized in that: The following steps are involved: The feature points of the model to be measured are marked, and the model's water entry process is recorded. Two high-speed cameras are used to simultaneously capture the process, obtaining a continuous stereo image pair of the water entry process. Detecting pixels of feature points of each image in the stereo image pair and performing bidirectional verification to obtain feature point pairs that pass bidirectional verification; According to the feature point pair that has passed the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair; Based on the depth information of the feature point, the two-dimensional pixel coordinates of the feature point in the stereo image pair are converted into world space coordinates to obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the posture change of the model is deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during the motion process; The step of obtaining the depth information of the feature point comprises the following steps: Read the stereo image pair taken by two high-speed cameras simultaneously and perform stereo correction on it; Calculating a disparity map of the corrected stereo image pair using a stereo matching algorithm. The disparity map represents the pixel differences between corresponding feature points of the stereo image pair. Optimizing the stereo matching algorithm by adjusting the disparity level and the size of the matching block. According to the disparity map, the disparity value of the feature point in the stereo image pair is converted into the depth information of the feature point by multiplying it by the baseline length between the high-speed cameras and dividing it by the pixel spacing; The step of obtaining the feature point pairs that pass the bidirectional verification comprises the following steps: Detect the feature points of each image through the target detection algorithm, automatically detect and identify multiple feature points in the stereo image pair, and determine the position information corresponding to each feature point; Based on the ORB algorithm, each detected feature point is described and the corresponding feature points with the same BRIEF descriptor are preliminarily matched. For each detected feature point, the ORB algorithm calculates the intensity center of the surrounding pixels, also known as the centroid, and uses the calculated centroid as the direction of the feature point. The descriptor is constructed by comparing the brightness of a pair of pixels around the feature point. For each feature point, the ORB algorithm randomly selects multiple pixel pairs and generates a binary value for each pair of pixels. These binary values are concatenated to form a long string, namely the BRIEF descriptor. For each pair of matched feature points, a bidirectional verification is performed to obtain a pair of feature points that pass the bidirectional verification by checking whether the matching results of the feature points in each other's images are consistent.

2. The method for measuring the model state in a ditching test according to claim 1, characterized in that: The step of obtaining the three-dimensional coordinates of each feature point comprises the following steps: The calculation formula for depth information is: Where B is the baseline distance between cameras, f is the focal length of the cameras, d is the parallax, and D is the depth; According to the depth information of the feature points, the two-dimensional image coordinates are converted into coordinates in the three-dimensional camera coordinate system: Z c =D Among them, (u,v) is the pixel coordinate, (X c ,Y c ,Z c ) is the camera coordinate; The conversion formula for converting 3D camera coordinates to world space coordinates is: Among them, (u,v) is the pixel coordinate, (x,y) is the image coordinate, (X c ,Y c ,Z c ) is the camera coordinate, (X w ,Y w ,Z w ) are world space coordinates, is the camera internal parameter, is the camera external parameter, d x is the change in the image coordinates of the feature point in the x direction, d y is the change in the image coordinates of the feature point in the y direction.

3. The method for measuring the model state in a ditching test according to claim 2, characterized in that: The construction of the rotation matrix comprises the following steps: According to the obtained three-dimensional coordinates of each feature point, the three-dimensional coordinates of all feature points in the model movement process are summarized to determine the three-dimensional point cloud of the model movement process; By classifying the 3D point cloud, the 3D coordinates of the same feature point at different times are grouped into one data set, and m data sets are obtained, that is, the number of feature points is m, m = 1, 2, ..., j, ..., m; Take any dataset j in m and get the spatial coordinates of the jth feature point at different times, which are p1, p2, ..., p n , where p i =(x i ,y i ,z i ) T Represents the three-dimensional coordinates of the feature point on the image at time i; For each pair of adjacent positions p i and p i+1 , by calculating the centroid of each pair of adjacent positions and performing de-centroiding, the adjacent positions p are calculated i and p i+1 The corresponding center of mass coordinate c i and c i+1 They are: For adjacent position p i and p i+1 The corresponding centroid coordinates are de-centred to obtain the adjacent position p i and p i+1 Relative to the center of mass coordinate c i and c i+1 The decentralized position q i and q i+1 They are: q i =p i -c i ; q i+1 =p i+1 -c i+1 ; Get the de-centroided position q i and q i+1 The covariance matrix H i for: The obtained de-centroided position q i and q i+1 The covariance matrix H i , as the covariance matrix of the j-th feature point; By performing singular value decomposition on the covariance matrix of the j-th feature point, the rotation matrix R of the feature point at different times is obtained. i : Among them, V i and U i Represents the orthogonal matrix corresponding to the adjacent de-centroided positions of the feature point, S i Represents the diagonal matrix corresponding to the feature point; when det(R i )=-1, V i The last column of the R i is a valid rotation matrix.

4. The method for measuring the model state in a ditching test according to claim 3, characterized in that: The acquisition of dynamic state parameters during the model movement process specifically includes: The dynamic state parameters include attitude parameters and speed parameters, wherein the attitude parameters include pitch angle, yaw angle, and roll angle; Convert the rotation matrix to Euler angles and decompose the rotation matrix R in ZYX order i , and the rotation angles around the Z, Y, and X axes are obtained, which are the yaw angles α i , pitch angle β i and roll angle γ i ; When the solved rotation matrix is When , we get the rotation matrix R i The calculation formula for the converted Euler angle is: β i =arctan(-r 31 ); Calculate the instantaneous velocity of the jth feature point through the displacement and time of adjacent moments; The instantaneous velocity calculation formula for each feature point is: Where Δp i is the displacement change corresponding to the time change Δt, and the frame rate of the camera is F; The speed of the model at this time is expressed as the average speed of adjacent points; The calculation formula of model speed is: Where N is the number of feature points for calculating velocity; Similarly, the data set corresponding to each feature point is solved, and finally the dynamic state parameters of the model during motion are output.

5. The method for measuring the model state in a ditching test according to claim 1, characterized in that: Also includes: The dynamic state parameters of the model during motion collected by the high-speed camera are denoted as model state parameters 1; The image of the model's landing process is collected by the sensor, and the dynamic state parameters of the model during the motion process based on the sensor acquisition are obtained, which are recorded as model state parameters 2; If the model state parameter 1 fits the model state parameter 2, the fitting result is determined as the final model state parameter output.

6. A model state measurement system for a ditching test, characterized in that: include: The model data acquisition module is used to mark the feature points of the model to be measured and record the model's water entry process. Two high-speed cameras are used to simultaneously capture the model and obtain a continuous stereo image pair of the water entry process. A feature point depth information acquisition module is used to detect the pixels of the feature points of each image in the stereo image pair and perform bidirectional verification to obtain feature point pairs that pass the bidirectional verification; based on the feature point pairs that pass the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair; The model state parameter acquisition module is used to convert the two-dimensional pixel coordinates of the feature point in the stereo image pair into world space coordinates based on the depth information of the feature point, and obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the model posture change is deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during the motion process; The step of obtaining the depth information of the feature point comprises the following steps: Read the stereo image pair taken by two high-speed cameras simultaneously and perform stereo correction on it; Calculating a disparity map of the corrected stereo image pair using a stereo matching algorithm. The disparity map represents the pixel differences between corresponding feature points of the stereo image pair. Optimizing the stereo matching algorithm by adjusting the disparity level and the size of the matching block. According to the disparity map, the disparity value of the feature point in the stereo image pair is converted into the depth information of the feature point by multiplying it by the baseline length between the high-speed cameras and dividing it by the pixel spacing; The step of obtaining the feature point pairs that pass the bidirectional verification comprises the following steps: Detect the feature points of each image through the target detection algorithm, automatically detect and identify multiple feature points in the stereo image pair, and determine the position information corresponding to each feature point; Based on the ORB algorithm, each detected feature point is described and the corresponding feature points with the same BRIEF descriptor are preliminarily matched. For each detected feature point, the ORB algorithm calculates the intensity center of the surrounding pixels, also known as the centroid, and uses the calculated centroid as the direction of the feature point. The descriptor is constructed by comparing the brightness of a pair of pixels around the feature point. For each feature point, the ORB algorithm randomly selects multiple pixel pairs and generates a binary value for each pair of pixels. These binary values are concatenated to form a long string, namely the BRIEF descriptor. For each pair of matched feature points, a bidirectional verification is performed to obtain a pair of feature points that pass the bidirectional verification by checking whether the matching results of the feature points in each other's images are consistent.

7. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: The feature points of the model to be measured are marked, and the model's water entry process is recorded. Two high-speed cameras are used to simultaneously capture the process, obtaining a continuous stereo image pair of the water entry process. Detecting pixels of feature points of each image in the stereo image pair and performing bidirectional verification to obtain feature point pairs that pass bidirectional verification; According to the feature point pair that has passed the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair; Based on the depth information of the feature point, the two-dimensional pixel coordinates of the feature point in the stereo image pair are converted into world space coordinates to obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the posture change of the model is deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during the motion process; The step of obtaining the depth information of the feature point comprises the following steps: Read the stereo image pair taken by two high-speed cameras simultaneously and perform stereo correction on it; Calculating a disparity map of the corrected stereo image pair using a stereo matching algorithm. The disparity map represents the pixel differences between corresponding feature points of the stereo image pair. Optimizing the stereo matching algorithm by adjusting the disparity level and the size of the matching block. According to the disparity map, the disparity value of the feature point in the stereo image pair is converted into the depth information of the feature point by multiplying it by the baseline length between the high-speed cameras and dividing it by the pixel spacing; The step of obtaining the feature point pairs that pass the bidirectional verification comprises the following steps: Detect the feature points of each image through the target detection algorithm, automatically detect and identify multiple feature points in the stereo image pair, and determine the position information corresponding to each feature point; Based on the ORB algorithm, each detected feature point is described and the corresponding feature points with the same BRIEF descriptor are preliminarily matched. For each detected feature point, the ORB algorithm calculates the intensity center of the surrounding pixels, also known as the centroid, and uses the calculated centroid as the direction of the feature point. The descriptor is constructed by comparing the brightness of a pair of pixels around the feature point. For each feature point, the ORB algorithm randomly selects multiple pixel pairs and generates a binary value for each pair of pixels. These binary values are concatenated to form a long string, namely the BRIEF descriptor. For each pair of matched feature points, a bidirectional verification is performed to obtain a pair of feature points that pass the bidirectional verification by checking whether the matching results of the feature points in each other's images are consistent.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs the following steps: The feature points of the model to be measured are marked, and the model's water entry process is recorded. Two high-speed cameras are used to simultaneously capture the process, obtaining a continuous stereo image pair of the water entry process. Detecting pixels of feature points of each image in the stereo image pair and performing bidirectional verification to obtain feature point pairs that pass bidirectional verification; According to the feature point pair that has passed the bidirectional verification, the depth information of the feature point is obtained by calculating the disparity value of the feature point in the stereo image pair; Based on the depth information of the feature point, the two-dimensional pixel coordinates of the feature point in the stereo image pair are converted into world space coordinates to obtain the three-dimensional coordinates of each feature point; based on the obtained three-dimensional coordinates of each feature point, the posture change of the model is deduced by constructing a rotation matrix to obtain the dynamic state parameters of the model during the motion process; The step of obtaining the depth information of the feature point comprises the following steps: Read the stereo image pair taken by two high-speed cameras simultaneously and perform stereo correction on it; Calculating a disparity map of the corrected stereo image pair using a stereo matching algorithm. The disparity map represents the pixel differences between corresponding feature points of the stereo image pair. Optimizing the stereo matching algorithm by adjusting the disparity level and the size of the matching block. According to the disparity map, the disparity value of the feature point in the stereo image pair is converted into the depth information of the feature point by multiplying it by the baseline length between the high-speed cameras and dividing it by the pixel spacing; The step of obtaining the feature point pairs that pass the bidirectional verification comprises the following steps: Detect the feature points of each image through the target detection algorithm, automatically detect and identify multiple feature points in the stereo image pair, and determine the position information corresponding to each feature point; Based on the ORB algorithm, each detected feature point is described and the corresponding feature points with the same BRIEF descriptor are preliminarily matched. For each detected feature point, the ORB algorithm calculates the intensity center of the surrounding pixels, also known as the centroid, and uses the calculated centroid as the direction of the feature point. The descriptor is constructed by comparing the brightness of a pair of pixels around the feature point. For each feature point, the ORB algorithm randomly selects multiple pixel pairs and generates a binary value for each pair of pixels. These binary values are concatenated to form a long string, namely the BRIEF descriptor. For each pair of matched feature points, a bidirectional verification is performed to obtain a pair of feature points that pass the bidirectional verification by checking whether the matching results of the feature points in each other's images are consistent.

Images