A wind tunnel video image stabilization method combining YoloV11 and homographic transformation

Abstract

The application discloses a wind tunnel video image stabilization method combining YoloV11 and homographic transformation, relates to the fields of target detection and image processing, and mainly adopts a calibration algorithm to calibrate internal parameters and distortion parameters of a camera, and corrects distortion of collected image data through the calibration parameters; then a certain amount of mark point data sets are collected in an experimental environment to train a YoloV11 detection model; then the detection model is used in combination with an improved traditional image processing mode to automatically detect mark points in a first frame of wind tunnel video images, and the mark points are used as initial positions to predict and update subsequent mark point positions by using a Kalman filtering algorithm; finally, the homographic matrix of the subsequent mark points and the initial mark point positions is calculated, and the homographic transformation is used to realize the image stabilization of the wind tunnel video images. The application realizes the wind tunnel video image stabilization in a pure visual mode, avoids the complicated design of a structure, and has better image stabilization effect and simpler operation mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of object detection and image processing. More specifically, the present application relates to a wind tunnel video stabilization method combining YoloV11 and homography transformation. BACKGROUND

[0002] In the process of wind tunnel model attitude measurement test, due to the existence of high-speed airflow in the wind tunnel and the vibration of high-speed camera during video shooting, the measured model will appear a certain degree of vibration state in the image, which has poor visualization effect and is not conducive to stable shooting of model video and high-precision measurement of attitude.

[0003] The traditional video stabilization method mainly suppresses physical vibration through fixed high-rigidity support or active vibration isolation platform, but this method cannot eliminate the distortion effect caused by high-speed airflow on the field of view, and has high implementation cost and complex structure. In addition, there are also methods of extracting scene feature points in the image, such as SIFT (Scale Invariant Feature Transform), ORB (OrientedFAST and Rotated BRIEF) features, etc. By calculating the feature matching between image frames, the homography constraint is obtained, and the influence caused by airflow or camera shaking is eliminated, so as to correct the image and achieve the purpose of video stabilization. However, this method depends on rich feature information in the scene, and needs stable features. In actual wind tunnel environment, there are often more feature mismatching or unstable feature points, and insufficient features, which affect the final stabilization effect.

[0004] In addition to the traditional physical method and feature detection method, deep learning method can also be used, such as CNN, optical flow network to predict inter-frame displacement or generative adversarial network homography matrix estimation. Although these methods can obtain good stabilization effect, they depend on a large amount of wind tunnel scene data for training, which is not suitable for actual wind tunnel environment. Specifically, the existing technology patent application number CN202510825446.6, a video anti-shake processing method, medium and device, estimates the homography transformation matrix by extracting the feature point information in the image, and combines global motion compensation with local grid optimization, which can process video frame images under severe shaking scene, restore the picture content, and obtain stable video frame images, avoiding missing monitoring pictures, but the problem is that the environmental conditions change greatly during the wind tunnel test, it is difficult to extract stable feature point information in the image, and the error tracking feature point is easy to cause the calculation error of homography transformation matrix, resulting in failure of image anti-shake.

[0005] For example, patent application number: CN201910670613.9, a video deblurring method based on deep learning, this method is no longer based on the traditional feature matching strategy and homography matrix estimation, but pixel-level mapping relationship estimation, such transformation can solve the problem of local cannot be fitted by the same homography matrix caused by discontinuous depth change, thus better results are obtained in real video, but the problem is that: this method needs stable real stable image frame as training data and the initial training model is trained on the public DeepStab data set, which is quite different from the actual wind tunnel environment; Secondly, it is difficult to obtain real stable image frame data in the wind tunnel environment, and it is difficult to construct a large amount of data for model training.

[0006] For example, patent application number: CN202010475187.6, a video deblurring method and system based on local path optimization, by feature point detection and feature point matching calculation on each frame of image of the original video, the homography matrix between all adjacent two frames of images is obtained, according to the homography matrix, the spatial variation of the original path of the entire original video is obtained, and the preliminary reconstruction matrix of each image frame is obtained by smoothing optimization processing on the original path; In the spatial variation of the entire original video, the relative path is used, the preliminary reconstruction matrix is iteratively processed to obtain the final reconstruction matrix of each image frame; The final reconstruction matrix of each image frame is used to perform image transmission transformation processing on each image frame in the original video to obtain the deblurred video image, but the problem is that: there is less feature information in the wind tunnel environment, and it is difficult to obtain enough feature matching point pairs after feature extraction and feature matching. In addition, in the wind tunnel environment, it is difficult to obtain feature matching point pairs in the same plane, which will lead to image transmission transformation failure.

[0007] In summary, it can be seen that the video deblurring method in the prior art cannot be applied to the wind tunnel video stabilization because:

[0008] 1. The environment in the wind tunnel changes greatly, the image feature points obtained are unstable, which leads to feature tracking failure;

[0009] 2. There are less feature points in the wind tunnel, and it is difficult to obtain enough feature matching point pairs in the same plane;

[0010] 3. It is difficult to obtain real stable image frame data in the wind tunnel, and it is difficult to construct the required data set for training. SUMMARY

[0011] An object of the present application is to solve at least the above problems and / or deficiencies, and to provide at least the advantages to be described later.

[0012] In order to achieve the objects and other advantages of the present application, a wind tunnel video image stabilization method combining YoloV11 and homographic transformation is provided, comprising:

[0013] S1, using a camera to shoot wind tunnel video data, and using OpenCV to parse the image information of each frame in the video;

[0014] S2, by calibrating the internal parameters and distortion parameters of the camera, completing the distortion correction of the wind tunnel image;

[0015] S3, taking the first frame image in the wind tunnel video as the initial image, using YoloV11 marker point detection and recognition algorithm to extract the center position of the marker point in the initial image as the initial marker point;

[0016] S4, based on the position of the current marker point and the position of the last frame marker point, based on the Kalman filter algorithm to complete the continuous tracking of the marker point;

[0017] S5, calculating the homographic transformation matrix of the current frame image to the initial image, to map the marker point of the next continuous frame to the marker point position in the last frame image, to complete the image stabilization output of the video through the homographic correction of the image;

[0018] In S3, the workflow of the marker point tracking algorithm comprises:

[0019] S31, using the trained YoloV11 model to obtain the ROI image including the marker point;

[0020] S32, based on the global adaptive Gamma correction method, the ROI image is contrast enhanced to obtain the image ;

[0021] S33, based on the multi-gradient threshold sub-pixel contour extraction method, the contour information of the marker point in the image is obtained;

[0022] S34, based on the contour information of the marker point, the least square method is used to obtain the center coordinates corresponding to the marker point.

[0023] Preferably, in S1, when shooting the wind tunnel video, the initial video image data should be shot when the wind tunnel is in the windless state.

[0024] Preferably, in S2, the distortion correction process of the wind tunnel image comprises:

[0025] S20, setting a chessboard format calibration board, and collecting a predetermined number of calibration image data;

[0026] S21. Identify key points in the calibration board using a checkerboard detection algorithm and record the correspondence between the actual physical coordinates of the calibration board and the pixel coordinates of the image.

[0027] S22. Calculate the camera's internal parameters K according to Zhang Zhengyou's planar calibration method;

[0028] S23. The distortion parameter D of the camera is calculated based on the internal parameter K and the LM optimization algorithm;

[0029] S24. Using the internal parameter K and the distortion parameter D, and combining them with the distortion correction function, the image coordinates are remapped to obtain the distortion-corrected image coordinates, and then the distortion-corrected image is generated.

[0030] Preferably, in S31, the trained YOLOv11 model is loaded using OpenCV, and the YOLOv11 model is retrained by adding negative samples to complete the fine-tuning optimization of the YOLOv11 model output. This results in a ROI image containing labeled points. It is characterized by the following formula:

[0031] ;

[0032] In the above formula, Represents the original image. This represents the bounding box of the marker points output by the YOLOv11 model.

[0033] Preferably, in S32, the workflow of the globally adaptive Gamma correction method includes:

[0034] S320, to After normalization, the mean gray value m within the ROI range is calculated;

[0035] S321, Use the following formula to... Enhancement processing is performed:

[0036] ;

[0037] In the above formula, Indicates after normalization The pixel value of any pixel in the array. This represents the image pixel value after enhancement, where g represents the enhancement parameter. When m > 0.6, then... When m>0.3, then .

[0038] Preferably, in S33, the workflow of the sub-pixel contour extraction method based on multi-gradient thresholds includes:

[0039] S330. Obtain multiple inter-class variance values ​​using the following formula. And the maximum inter-class variance The corresponding grayscale value is used as the segmentation threshold. T ,and :

[0040] ;

[0041] In the above formula, In the image In the middle, the pixel value is greater than or equal to the segmentation threshold. T The ratio of the number of pixels in a given area to the total number of pixels in the image. This indicates that the pixel value is less than the segmentation threshold. T The ratio of the number of pixels in a given area to the total number of pixels in the image. Representing an image The average pixel value of all pixels in the image. This indicates that the value is greater than or equal to the segmentation threshold. T The average pixel value, Indicates less than the segmentation threshold T The average pixel value;

[0042] S332, Set the segmentation threshold T Set as multi-gradient threshold segmentation range And set the threshold lookup step size ;

[0043] S333, Let the current segmentation threshold be... Tc ,and ,Will The image is larger than Tc The pixel value is set to 255, and vice versa, to obtain a binarized image. ;

[0044] S334. Detect binarized images using the Canny edge detection algorithm. Edge information in;

[0045] S335, Based on any edge point D i To determine whether there are other edge points within the range of 8 adjacent pixels. D i Whether it is continuous;

[0046] S336. Define each continuous edge point as a single contour information, and filter out erroneous contours by the area of ​​the contour region, thereby obtaining the true contour information of the marked points.

[0047] S337, Settings and judge TcCheck if it exceeds 255. If it does not exceed, return to S333; otherwise, proceed to S338.

[0048] S338. Count the number of times the contour of each marker point appears during the search process. If the number of times any contour appears is less than 3, delete the corresponding contour.

[0049] S339. Sum the contour data of the marker points obtained in each round, and then obtain the sub-pixel contour coordinates of the marker points by averaging.

[0050] Preferably, in S4, the continuous tracking method for the marker points is as follows:

[0051] S40. Calculate the movement velocity of the marker point based on the current position of the marker point and the position of the marker point in the previous frame;

[0052] S41. Predict the position D(x,y) where the marker point should appear in the next frame using a uniform motion model, and use the YOLOV11 model at D(x,y) to detect the marker point in the next frame.

[0053] S42. Update the predicted marker position using the detected marker position of the next frame, and so on to complete the continuous tracking of the marker.

[0054] The present invention has at least the following beneficial effects:

[0055] Firstly, compared with the traditional method of isolating camera vibration through mechanical structure, this invention uses a purely visual method to achieve wind tunnel video stabilization, avoiding complex structural design, while also achieving better stabilization effect and simpler operation.

[0056] Secondly, compared with existing methods that achieve video stabilization through feature points or directly through deep learning, this invention combines YOLOv11 with traditional image processing. Using a model pre-trained in an experimental environment, it can detect elliptical feature markers, thus achieving video stabilization even when environmental features are insufficient or training data is inadequate.

[0057] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0058] Figure 1 Hardware principle block diagram for this invention;

[0059] Figure 2 This is a schematic diagram of the chessboard calibration plate of the present invention. Detailed Implementation

[0060] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0061] This invention employs a combination of YOLOv11 and homography transformation to perform image stabilization on wind tunnel videos. The implementation of this method depends on the corresponding hardware system, such as... Figure 1 As shown, the hardware system required by this invention mainly includes: camera 1, lens 2, camera support platform 3, server 4, wind tunnel detection target 5, and marker module (the marker module includes four circular markers: A, B, C, and D). The system setup method is as follows: Figure 1 The camera is fixed on the mounting platform and connected to the server. The lens focal length is adjusted by observing the camera's imaging effect on the server until the target and circular markers are clearly visible. In practice, the camera is fixed on the mounting platform and connected to the server. The lens focal length is adjusted by observing the camera's imaging effect on the server until the target and all circular markers are clearly visible.

[0062] After the hardware system is built, this invention provides a wind tunnel video stabilization method that combines YOLOv11 and homography transformation. By pasting four marker points in a plane of the wind tunnel and combining YOLOv11 detection with an improved traditional image processing algorithm, it ensures that the features of the marker points can be stably acquired and all marker points can be tracked during the experiment. Furthermore, a certain number of marker point images can be captured in any environment as a training dataset for the YOLOv11 algorithm. This solves the problems of difficulty in obtaining stable image data in actual wind tunnel environments and difficulty in constructing image datasets, while realizing the stabilization of high-speed camera video images in wind tunnel environments.

[0063] The core content of the wind tunnel video stabilization method combining YOLOv11 and homography transformation mainly includes: First, a calibration algorithm is used to calibrate the camera's intrinsic and distortion parameters, and the acquired image data is then distorted using the calibrated parameters. Next, a certain number of marker point datasets are collected in the experimental environment to train a YOLOv11 detection model. Using this detection model and an improved traditional image processing method, marker points in the first frame of the wind tunnel video image are automatically detected. These marker points are then used as initial positions, and Kalman filtering is applied to predict and update the positions of subsequent marker points. Finally, by calculating the homography matrix between the positions of subsequent marker points and the initial marker point positions, homography transformation is used to stabilize the wind tunnel video image. The specific processing flow includes:

[0064] 1. Wind tunnel video image data acquisition

[0065] The wind tunnel was captured using a camera, and each frame was parsed using OpenCV. It's important to note that the initial video images should be captured when the wind tunnel is windless and the overall hardware environment is relatively static. This allows for setting static initial images during subsequent processing.

[0066] 2. Image distortion correction

[0067] This step mainly involves calibrating the camera parameters and using these parameters to correct distortion in the captured wind tunnel video image data. Specifically, during image formation, the camera lens exhibits a certain degree of distortion; therefore, it is necessary to calibrate the camera's internal parameters and distortion parameters to correct the distortion of the wind tunnel images. The specific implementation process is as follows:

[0068] (1) Prepare a... Figure 2 A checkerboard calibration board with a known geometric pattern is used, and a predetermined number of calibration image data are collected;

[0069] (2) Identify key points in the calibration board using the checkerboard detection algorithm and record the correspondence between the actual physical coordinates (3D) of the calibration board and the pixel coordinates (2D) of the image.

[0070] (3) Calculate the camera's internal parameters K according to Zhang Zhengyou's planar calibration method;

[0071] (4) The camera distortion parameter D is calculated using the internal parameters obtained in the previous step and the LM (Levenberg-Marquardt) optimization algorithm;

[0072] (5) Using the internal parameters and distortion parameters, and combined with the distortion correction function (undistort) in OpenCV (Open Source Computer Vision Library), the image coordinates are remapped, the distortion-corrected image coordinates are calculated, and the distortion-corrected image is generated.

[0073] 3. Initial position setting for wind tunnel images

[0074] The first frame of the wind tunnel video is used as the initial static image. A marker detection and recognition algorithm based on the YOLOv11 algorithm is used to extract the center positions of elliptical markers in the image. (It should be noted that in practical applications, the markers of the detected target are actually circular, but due to the camera mounting angle and the perspective transformation principle of imaging, the markers in the acquired image are no longer regular circular structures; they will be referred to as elliptical markers hereafter.) Specifically, the marker detection and recognition algorithm's processing flow involves automatically detecting the positions of elliptical markers in the image using the YOLOv11 algorithm after obtaining the distortion-corrected image, fitting the elliptical center coordinates of the markers using an improved sub-pixel contour detection algorithm, and finally using a Kalman filter algorithm to track each marker.

[0075] In practice, traditional elliptical marker detection typically involves a series of conventional image processing algorithms, such as image preprocessing, thresholding, contrast enhancement, contour detection, and ellipse fitting, to detect elliptical markers in an image. This approach fails to effectively distinguish true markers when there are many interfering points in the environment. Therefore, to address this issue, this step employs the YOLOv11 deep learning algorithm to detect the location of elliptical markers in the image. The specific implementation is as follows:

[0076] (1) Data preparation and labeling

[0077] A certain number of elliptical markers are pasted in the experimental environment. The shape of the markers is similar to... Figure 1 As shown, a certain number of labeled point images were captured using a camera, and the data was labeled using AnyLabeling annotation software. The labeled data was in standard YOLO format. Specifically, each image corresponded to a .txt file, with the annotation format being (category name, center x, y_center, width, height) normalized coordinates).

[0078] (2) YoloV11 model training

[0079] The labeled dataset from the previous step is divided into training, testing, and validation sets according to a certain ratio (e.g., 70%, 20%, 10%). Using the training methods and models provided by the YOLOv11 algorithm, a YOLOv11 model that can be used to detect elliptical marker targets in images is trained.

[0080] (3) Elliptical target detection

[0081] The system acquires raw elliptical marker image data in the wind tunnel environment using a camera, loads the aforementioned detection model using OpenCV, and uses the model to infer the location information of the elliptical markers in the image.

[0082] (4) Model fine-tuning

[0083] To address potential misidentifications in the real-world environment described in the previous step, this data is added to negative samples for retraining, thereby optimizing the model's detection performance.

[0084] 4. Marker tracking and video stabilization

[0085] The marker tracking in this step involves using a marker tracking algorithm to track the positions of markers in consecutive frames based on the initial marker positions.

[0086] The marker tracking algorithm detects the elliptical marker locations obtained from the YOLOv11 model, extracts the corresponding marker image data at these locations, and generates a Region of Interest (ROI) image containing only the markers. Then, it combines an improved traditional image processing algorithm to identify and fit the center coordinates of the elliptical markers, specifically including: adaptive ROI image Gamma enhancement, sub-pixel contour extraction with multi-gradient thresholds, and ellipse center fitting. Finally, the Kalman filter algorithm is used to achieve marker recognition and continuous tracking. The specific implementation is as follows:

[0087] (1) Acquisition of ROI image with markers

[0088] Assuming the original image is The bounding boxes of marker points obtained by the YOLOv11 algorithm are Then you can create an ROI image that only contains marker points. Represented as:

[0089] (1)

[0090] The above formula means to extract the original image. middle The image data of the area is used as the marker point image.

[0091] (2) Adaptive ROI image Gamma enhancement

[0092] Traditional global adaptive image gamma enhancement is performed by calculating the grayscale mean, variance, etc. of the entire image and automatically setting the gamma enhancement coefficient to enhance the entire image globally.

[0093] However, due to uneven lighting in the wind tunnel, some marker points may be too bright while others are too dark. Therefore, using a global approach will inevitably lead to failure in image enhancement for some ROIs. Local adaptive enhancement, on the other hand, calculates a Gamma value for each pixel based on the characteristics of its local neighborhood (such as a 5x5 or 7x7 window). This method introduces more noise within the ROI.

[0094] To more accurately extract the center position of the elliptical marker points, this invention proposes a globally adaptive Gamma correction method based on the Region of Interest (ROI) to enhance the contrast of the image. The specific implementation is as follows:

[0095] ① Normalize the ROI image and calculate the mean gray value within the ROI range;

[0096] ② If the mean is greater than 0.6, it means the image is too bright; adjust the settings accordingly. If the mean is less than 0.3, the image is too dark; adjust the settings accordingly. The specific transformations are as follows:

[0097] (2)

[0098] in, This represents the pixel value of any pixel in the normalized ROI image, and g represents the enhancement parameters set during enhancement. This represents the image pixel value after enhancement operations.

[0099] ③ Perform the above two steps sequentially on the ROI image of each marked point, and then save the transformed ROI image data for the next step of subpixel contour detection.

[0100] (3) Sub-pixel contour extraction with multi-gradient thresholds

[0101] Traditional contour extraction algorithms obtain pixel-level contour information from the binary image obtained after single threshold segmentation. This approach suffers from incomplete contour extraction and low contour accuracy. This invention proposes a sub-pixel contour extraction method based on multi-gradient thresholding to obtain accurate contour information of marked points. The specific implementation is as follows:

[0102] ① Calculate the maximum inter-class variance using the OTSU thresholding algorithm. To obtain a suitable segmentation threshold Images that enhance contrast The image is converted to a binary image as shown in the following formula:

[0103] (3)

[0104] in, Represents the inter-class variance. Representing an image In the middle, the pixel value is greater than or equal to the segmentation T The ratio of the number of pixels in the image to the total number of pixels in the image. This indicates that the pixel value is less than the segmentation value. TThe ratio of the number of pixels in the image to the total number of pixels in the image. Representing an image The average pixel value of all pixels in the image. This indicates that the value is greater than or equal to the segmentation threshold. T The average pixel value, Indicates less than the segmentation threshold T The average pixel value.

[0105] for For each gray value, an inter-class variance value can be obtained using equation (3). According to the mathematical meaning of between-class variance, the larger the between-class variance, the greater the distinction between the background and the target. Therefore, it is only necessary to find the maximum between-class variance. The corresponding grayscale value is used as the optimal segmentation threshold. T .

[0106] ② Based on the optimal segmentation threshold T Set the multi-gradient threshold segmentation range as follows: .if If so, then the minimum threshold is set to 0; if Therefore, the maximum threshold is set to 255, and the threshold lookup step size is set. .

[0107] ③ Set the current segmentation threshold .from Tc The threshold is used to start the contour search, and the search method is as follows: First, set the threshold... Images larger than the segmentation threshold Tc The pixel value is set to 255, and vice versa, to obtain a binarized image. The Canny edge detection algorithm is used to detect binarized images. The edge information is then analyzed. Next, the continuity of an edge point is determined by whether another edge point exists within its eight adjacent pixels. Finally, each continuous edge point is defined as a single contour, and smaller or larger erroneous contours are filtered out based on the area of ​​the contour region, thus accurately obtaining the true contour information of the elliptical marker points.

[0108] ④ Repeat the contour search method from the previous step, increasing the threshold set for each search by one step size based on the current segmentation threshold. .

[0109] ⑤ The search ends when the current threshold exceeds the maximum threshold. Count the number of times each elliptical marker outline appears during the search. If the outline appears less than 3 times, it is considered an interference point and deleted.

[0110] ⑥ Sum the contour data of the elliptical markers obtained in each round, and then obtain the sub-pixel contour coordinates of the elliptical markers by averaging them, so as to fit the center coordinates of the elliptical markers.

[0111] (4) Ellipse center fitting

[0112] Based on the outline information of the marked points obtained in the previous step, the coordinates of the center of the ellipse can be obtained using the least squares method. Assume the equation of the ellipse is: Where a, b, c, d, e, and f are the parameters to be solved for the equation of the ellipse. The equation is solved algebraically using the least squares method:

[0113] (4)

[0114] in, n This indicates the number of contour points on the contour. i Indicates the serial number of the contour point ( ), This is the function to be optimized. By finding the minimum value of this function using the least squares method, all the parameter values ​​of the ellipse equation can be obtained. The coordinates of the ellipse's center can then be obtained from the ellipse's parametric equation, as shown in the following formula:

[0115] (5)

[0116] (6)

[0117] in, This indicates the center coordinates of the ellipse marker point.

[0118] (5) Homography Transformation Video Stabilization

[0119] Homography-based video stabilization is achieved by calculating the homography matrix of two consecutive frames using the center coordinates of corresponding marker points. By setting the position of the first frame as the initial position and mapping subsequent frames to that initial position, video stabilization can be accomplished.

[0120] In practice, assume the coordinates of the detected marker points in the two frames of images are as follows: and ,in The center coordinates of one set of marker points are respectively and Therefore, the homography transformation equation can be constructed as follows:

[0121] (7)

[0122] in, and It is a homogeneous coordinate representation of two central coordinate points, where H is a 3x3 homography matrix. ~ These are the parameters to be solved. Since the last row in the above transformation process is equal to 1, the H matrix actually only has 8 degrees of freedom, requiring at least 4 sets of marker points. The homography transformation matrix of the two images can be solved using DLT (Direct Linear Transform).

[0123] Therefore, in practical applications, video stabilization in this step can be achieved by calculating the homography transformation matrix from the current frame image to the initial image. This matrix can then be used to map the marker points in the current frame image to the marker point positions in the initial frame image, as follows:

[0124] ① Represent the coordinates of each pixel in the original image as homogeneous coordinates. .

[0125] ② Multiply the homography coordinates by the homography matrix H to obtain the homography coordinates of the target image. :

[0126] (8)

[0127] ③ Convert the homogeneous coordinates of the target image to pixel coordinates. If the pixel coordinates exceed the size range of the target image, the calculation result is ignored, and the next pixel coordinate is calculated.

[0128] ④ If the pixel coordinates from the previous step are within the range of the target image, then the pixel values ​​of the target image are calculated using bilinear interpolation.

[0129] ⑤ After homography transformation, the original image is presented from the perspective of the target image, realizing homography correction of the image, that is, achieving the effect of video stabilization.

[0130] The above solution is merely an illustration of a preferred example and is not limited thereto. When implementing this invention, appropriate substitutions and / or modifications can be made according to the user's needs.

[0131] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be applied to various fields suitable for the present invention. Other modifications can be readily made by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and examples shown and described herein.

Claims

1. A wind tunnel video stabilization method combining YOLOv11 and homography transformation, characterized in that, include: S1. Use a camera to capture wind tunnel video data, and use OpenCV to parse the image information of each frame of the video; S2. By calibrating the camera's internal parameters and distortion parameters, distortion correction of the wind tunnel image is completed; S3. Using the first frame of the wind tunnel video as the initial image, the center position of the marker point in the initial image is extracted using the YOLOv11 marker point detection and recognition algorithm, and used as the initial marker point; S4. Based on the current position of the marker point and the position of the marker point in the previous frame, the marker point is continuously tracked using the Kalman filter algorithm. S5. Calculate the homography transformation matrix from the current frame image to the initial image, so as to map the marker points of the next consecutive frame to the marker point positions in the previous frame image, so as to complete the stable output of the video through image homography correction. In S3, the workflow for using the YOLOv11 marker detection and recognition algorithm includes: S31. Use a trained YOLOV11 model to obtain the ROI image including the marked points; S32. Contrast enhancement of the ROI image based on a globally adaptive Gamma correction method to obtain the image... ; S33. A sub-pixel contour extraction method based on multi-gradient thresholds is used to obtain the image. Contour information of the marker points; S34. Based on the contour information of the marker points, the center coordinates corresponding to the marker points are obtained by using the least squares method.

2. The wind tunnel video stabilization method combining YOLOv11 and homography transformation as described in claim 1, characterized in that, In S1, when shooting wind tunnel video, the initial video image data should be taken when the wind tunnel is in a windless state.

3. The wind tunnel video stabilization method combining YOLOv11 and homography transformation as described in claim 1, characterized in that, In S2, the distortion correction process for wind tunnel images includes: S20. Set up a calibration board in chessboard format and collect a predetermined number of calibration image data; S21. Identify key points in the calibration board using a checkerboard detection algorithm and record the correspondence between the actual physical coordinates of the calibration board and the pixel coordinates of the image. S22. Calculate the camera's internal parameters K according to Zhang Zhengyou's planar calibration method; S23. The distortion parameter D of the camera is calculated based on the internal parameter K and the LM optimization algorithm; S24. Using the internal parameter K and the distortion parameter D, and combining them with the distortion correction function, the image coordinates are remapped to obtain the distortion-corrected image coordinates, and then the distortion-corrected image is generated.

4. The wind tunnel video stabilization method combining YOLOv11 and homography transformation as described in claim 1, characterized in that, In S31, the trained YOLOv11 model is loaded using OpenCV, and then retrained by adding negative samples to achieve fine-tuning optimization of the YOLOv11 model's output. This results in a ROI image containing labeled points. It is characterized by the following formula: ； In the above formula, Represents the original image. This represents the bounding box of the marker points output by the YOLOv11 model.

5. The wind tunnel video stabilization method combining YOLOv11 and homography transformation as described in claim 1, characterized in that, In S32, the workflow of the globally adaptive Gamma correction method includes: S320, to After normalization, the mean gray value m within the ROI range is calculated; S321, Use the following formula to... Enhancement processing is performed: ； In the above formula, Indicates after normalization The pixel value of any pixel in the array. This represents the image pixel value after enhancement, where g represents the enhancement parameter. When m > 0.6, then... When m>0.3, then .

6. The wind tunnel video stabilization method combining YOLOv11 and homography transformation as described in claim 1, characterized in that... In S33, the workflow of the sub-pixel contour extraction method based on multi-gradient thresholds includes: S330. Obtain multiple inter-class variance values ​​using the following formula. And the maximum inter-class variance The corresponding grayscale value is used as the segmentation threshold. T ,and : ； In the above formula, In the image In the middle, the pixel value is greater than or equal to the segmentation threshold. T The ratio of the number of pixels in a given area to the total number of pixels in the image. This indicates that the pixel value is less than the segmentation threshold. T The ratio of the number of pixels in a given area to the total number of pixels in the image. Representing an image The average pixel value of all pixels in the image. This indicates that the value is greater than or equal to the segmentation threshold. T The average pixel value, This indicates that the value is less than the segmentation threshold. T The average pixel value; S332, Set the segmentation threshold T Set as multi-gradient threshold segmentation range And set the threshold lookup step size ; S333, Let the current segmentation threshold be... Tc ,and ,Will The image is larger than Tc The pixel value is set to 255, and vice versa, to obtain a binarized image. ; S334. Detect binarized images using the Canny edge detection algorithm. Edge information in; S335, Based on any edge point D i To determine whether there are other edge points within the range of 8 adjacent pixels. D i Whether it is continuous; S336. Define each continuous edge point as a single contour information, and filter out erroneous contours by the area of ​​the contour region, thereby obtaining the true contour information of the marked points. S337, Settings and judge Tc Check if it exceeds 255. If it does not exceed, return to S333; otherwise, proceed to S338. S338. Count the number of times the outline of each marker point appears during the search process. If any outline appears less than 3 times, delete the corresponding outline. S339. Sum the contour data of the marker points obtained in each round, and then obtain the sub-pixel contour coordinates of the marker points by averaging.

7. The wind tunnel video stabilization method combining YOLOv11 and homography transformation as described in claim 1, characterized in that, In S4, the continuous tracking method for marker points is as follows: S40. Calculate the movement velocity of the marker point based on the current position of the marker point and the position of the marker point in the previous frame; S41. Predict the position D(x,y) where the marker point should appear in the next frame using a uniform motion model, and use the YoloV11 model at D(x,y) to complete the detection of the marker point in the next frame. S42. Update the predicted marker position using the detected marker position of the next frame, and so on to complete the continuous tracking of the marker.

Images