A method and system for detecting output force and torque of a drone

Abstract

The application provides a kind of unmanned aerial vehicle output force and torque detection method and system, it is related to unmanned aerial vehicle control technical field, the method includes: obtaining the image data of each rotor of unmanned aerial vehicle, and the image data is preprocessed, obtain the continuous frame of preset length;Multiple center area images of each rotor are extracted from the continuous frame by frame difference method and multi-frame average method;Each center area image is extracted, the shaft contour of the center shaft of rotor is obtained, and the shaft contour is elliptical fitting, to obtain the center point coordinates of the shaft of rotor;By feature matching algorithm, the key points of the preset mark image of each rotor are detected, and the key point is obtained according to the center coordinates of the key points when the rotor rotates;According to the center coordinates and the center point coordinates of the shaft, the output force and torque of the unmanned aerial vehicle are determined.The application extracts accurate image data from image data in high dynamic environment, improves the accuracy of the output force and torque measurement of unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control technology, and more specifically, to a method and system for detecting the output force and torque of a UAV. Background Technology

[0002] Drones are being used more and more widely in fields such as aerial photography, logistics, and agriculture, especially multi-rotor drones. In order to ensure the flight stability and accuracy of multi-rotor drones, it is necessary to control them by the output force and torque generated by the high-speed rotation of different rotors. Therefore, it is necessary to make reasonable calculations of the drone's output force and torque.

[0003] In related technologies, in order to provide more accurate data, a force and torque calculation method based on computer vision is adopted. Image processing technology is used to extract rotor motion features, and aerodynamic models are combined to calculate forces and torques in real time. However, the rapid movement of the rotor can easily lead to image blurring, affecting the accuracy of feature extraction. At the same time, complex background interference and the diversity of rotor structures make the target detection robustness insufficient, resulting in large errors in the calculated output forces and torques, which in turn affect the flight stability of the UAV. Summary of the Invention

[0004] The problem addressed by this invention is how to improve the measurement accuracy of the output force and torque of a drone.

[0005] To address the aforementioned problems, this invention provides a method and system for detecting the output force and torque of a UAV.

[0006] In a first aspect, the method for detecting the output force and torque of a UAV according to the present invention includes: Image data of each rotor of the UAV is acquired, and the image data is preprocessed to obtain continuous frames of a preset length. Multiple central region images of each rotor are extracted from the consecutive frames using frame difference and multi-frame averaging methods. Extract the image of each central region to obtain the axis profile of the central axis of the rotor, and then perform ellipse fitting on the axis profile to obtain the coordinates of the center point of the rotor axis; By using a feature matching algorithm, key points of a preset marked image of each rotor are detected, and the center coordinates of the key points are obtained based on the key points when the rotor rotates. The output force and torque of the UAV are determined based on the coordinates of the center of the circle and the coordinates of the center point of the rotating shaft.

[0007] Optionally, the preprocessing of the image data to obtain consecutive frames of a preset length includes: The image data is converted to obtain a grayscale image of the image data. The grayscale image is cropped according to the preset length to obtain grayscale images of multiple consecutive video frames; Each grayscale image is smoothed by Gaussian filtering, and the smoothed grayscale images of each video frame are combined to form the continuous frame.

[0008] Optionally, the step of extracting multiple center region images of each rotor from the consecutive frames using frame differencing and multi-frame averaging includes: Based on the difference in grayscale values ​​at the same pixel point between any two consecutive video frames in the continuous frame, multiple difference images are obtained; The difference image is binarized according to a first threshold range to obtain the binarized image; The rotor's central region image in each video frame of the consecutive frames is obtained by smoothing each binarized image using a multi-frame averaging method.

[0009] Optionally, the step of extracting the rotation axis profile of the central rotation axis of the rotor from each of the central region images includes: A morphological dilation operation is performed on the central region image to obtain the dilated central region image; Based on the expanded central region image, a binarization operation is performed according to the second threshold range to obtain the result image; Edge detection is performed on the resulting image to obtain the profile of the central axis of the rotor.

[0010] Optionally, the step of performing ellipse fitting on the shaft profile to obtain the coordinates of the rotor shaft center point includes: Obtain the contour area of ​​the rotation axis contour in each of the central region images; Based on the contour area, the rotation axis contours of each of the central region images are sorted to obtain the rotation axis contour with the largest contour area; The shaft profile with the largest profile area is taken as the target profile of the rotor. The target profile is fitted using an ellipse fitting algorithm to obtain the coordinates of the rotor's center point.

[0011] Optionally, detecting key points of a preset marked image for each rotor using a feature matching algorithm includes: Preset key points are extracted from a preset template image using the ORB algorithm; The preset key points are matched with the preset marker image of the rotor by brute force matching to obtain the key points of the preset marker image.

[0012] Optionally, obtaining the center coordinates of the key points when the rotor rotates based on the key points includes: By using a fitting algorithm, a preset number of key points are randomly selected for fitting, and the center coordinates and radii of multiple fitted circles are obtained. Obtain the nearest distance between each key point and the fitted circle, and extract interior points from the key points based on the nearest distance; The fitted circle with the most interior points is taken as the final circle, and least-squares circle fitting is performed on the final circle to obtain the center coordinates of the key points in each central region image.

[0013] Optionally, determining the output force and torque of the UAV based on the coordinates of the center of the circle and the coordinates of the center point of the rotation axis includes: Two video frames are extracted from the consecutive frames at a preset interval; The center coordinates are tracked by Kalman filtering. Based on the angles formed by the center coordinates of the central region images in the two video frames and the coordinates of the center point of the rotating shaft, the instantaneous speed and forward / reverse rotation information of each rotor are determined. The output force and torque of the UAV are determined based on the instantaneous speed of the rotor and the forward and reverse rotation information.

[0014] Optionally, determining the output force and torque of the UAV based on the instantaneous speed of the rotor and the forward / reverse rotation information includes: The real-time rotational speed of each rotor is determined based on the instantaneous speed of the rotor and the forward / reverse rotation information; The total thrust of the UAV is determined based on the real-time rotational speed and thrust coefficient of the rotor, and the total thrust is used as the output force. Based on the distance between each rotor and the center of the UAV, combined with the real-time rotational speed of the rotor and the thrust coefficient, the total pitch moment, total roll moment and total yaw moment of the UAV are determined, and the total pitch moment, total roll moment and total yaw moment are used as the torque of the UAV.

[0015] Secondly, the UAV output force and torque detection system of the present invention includes: The image acquisition unit is used to acquire image data of each rotor of the UAV and preprocess the image data to obtain continuous frames of a preset length. The image processing unit is used to extract multiple central region images of each rotor from the consecutive frames using frame difference and multi-frame averaging methods. The center point coordinate extraction unit is used to extract the center region image of each of the central regions to obtain the axis profile of the rotor's central axis, and then perform ellipse fitting on the axis profile to obtain the center point coordinates of the rotor's axis. The center coordinate extraction unit is used to detect key points of a preset marked image of each rotor using a feature matching algorithm, and to obtain the center coordinates of the key points when the rotor rotates based on the key points. The calculation unit is used to determine the output force and torque of the UAV based on the coordinates of the circle center and the coordinates of the center point of the rotation axis.

[0016] The method and system for detecting the output force and torque of a UAV of the present invention acquires continuous image data of the UAV rotor and performs preprocessing to obtain clear image data of the UAV rotor, presented in the form of continuous frames. Since the continuous frames consist of multiple consecutive video frames, the frame difference method is used to extract the image of the rotor center region from the continuous frames, effectively highlighting the dynamic characteristics of the rotor while suppressing interference from the background and blades, making the extraction of the rotor center region more accurate, thereby improving the accuracy of image recognition of the rotor. Then, by fitting an ellipse to the contour of the rotor center axis, the coordinates of the axis center point are accurately determined, providing a stable reference point for measurement. In addition, a feature matching algorithm is used to detect key points and track their center coordinates. By tracking the center coordinates, subtle changes in the rotor during rotation are captured, further improving the accuracy of capturing the rotor motion state. Finally, by combining the center coordinates and the axis center point coordinates, the output force and torque of the UAV are calculated according to aerodynamic principles. With accurate coordinate data and dynamic tracking information, the force and torque can be calculated more accurately. This invention extracts precise image data from image data in highly dynamic environments, accurately perceives the rotor motion state, and improves the accuracy of measuring the output force and torque of UAVs, thereby providing strong technical support for the flight control and performance optimization of UAVs. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a method for detecting the output force and torque of a UAV in one embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a quadcopter drone in one embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a UAV output force and torque detection system in one embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0019] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0020] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first," "second," etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0021] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0022] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0023] To address the problems existing in the aforementioned related technologies, this embodiment provides a method and system for detecting the output force and torque of a UAV.

[0024] Combination Figure 1 As shown, this embodiment of the invention provides a method for detecting the output force and torque of a UAV, including: Image data of each rotor of the UAV is acquired and the image data is preprocessed to obtain continuous frames of a preset length.

[0025] Specifically, by acquiring image data of each rotor of the UAV and preprocessing this data, a series of continuous, high-quality frames can be obtained. These continuous frames not only demonstrate the dynamic changes of the rotor but also enhance image clarity and contrast through preprocessing, such as grayscale conversion and noise reduction. The purpose of this step is to ensure the data quality for subsequent analysis, laying the foundation for accurate detection of rotor motion. The use of continuous frames helps capture the complete motion cycle of the rotor, thereby enabling more accurate analysis of its motion characteristics.

[0026] Multiple central region images of each rotor are extracted from the consecutive frames using frame difference and multi-frame averaging methods.

[0027] Specifically, the frame difference method is used to extract the central region image of the rotor from consecutive frames to highlight the dynamic features of the rotor. By comparing the differences between consecutive frames to identify the motion region, interference from the background and blades is effectively suppressed. This makes the extraction of the rotor's central region more accurate and improves the precision of image recognition.

[0028] The image of each central region is extracted to obtain the axis profile of the central axis of the rotor. Then, the axis profile is fitted with an ellipse to obtain the coordinates of the center point of the rotor axis.

[0029] Specifically, ellipse fitting is performed on the profile of the rotor's central axis to accurately determine the coordinates of the axis's center point. Ellipse fitting can adapt to the geometry of the rotor axis, providing accurate fitting results even when the axis profile is irregular or has slight deviations, providing a stable reference point for measurement, thereby enabling more accurate analysis of the rotor's motion state. In a preferred embodiment of the invention, incomplete profiles can be repaired using morphological dilation operations, and the circular profile of the rotor's central axis can be extracted using the Canny edge detection method. Ellipse fitting technology can accurately determine the coordinates of the rotor axis's center point, providing a stable reference point for measurement.

[0030] By using a feature matching algorithm, key points of a preset marked image for each rotor are detected, and the center coordinates of the key points are obtained based on the key points when the rotor rotates.

[0031] Specifically, by detecting and tracking key points using a feature matching algorithm, subtle changes in the rotor during rotation can be captured. A preset marker image is a specific marker, such as a circular marker, placed on the rotor. Identifying and tracking this specific marker using the feature matching algorithm provides crucial information about its motion state, thereby enabling more accurate analysis of the rotor's motion characteristics. In a preferred embodiment of the invention, the ORB algorithm can be used to detect key points in each central region image, and the center coordinates of the key points during rotor rotation can be obtained based on these key points. Furthermore, it is worth mentioning that combining Kalman filtering with dynamic tracking of the circular marker's center can effectively filter out random errors, ensuring stable positioning of the rotor marker points in consecutive frames.

[0032] The output force and torque of the UAV are determined based on the coordinates of the center of the circle and the coordinates of the center point of the rotating shaft.

[0033] Specifically, the output force and torque of the UAV are determined based on the coordinates of the center of the circle and the center point of the rotation axis. Using precise coordinate data and dynamic tracking information, the force and torque are calculated according to aerodynamic principles. This method not only improves the accuracy of the calculations but also makes the measurement results more reliable.

[0034] The method for detecting the output force and torque of a UAV in this invention acquires and preprocesses continuous image data of the UAV rotor to obtain clear image data of the UAV rotor, presented in the form of continuous frames. Since each continuous frame consists of multiple consecutive video frames, the frame difference method is used to extract the rotor center region image from the continuous frames, effectively highlighting the dynamic characteristics of the rotor while suppressing interference from the background and blades, making the extraction of the rotor center region more accurate and improving the accuracy of rotor image recognition. Then, by fitting an ellipse to the contour of the rotor's central axis, the coordinates of the axis's center point are accurately determined, providing a stable reference point for measurement. Furthermore, a feature matching algorithm is used to detect key points and track their center coordinates. By tracking the center coordinates, subtle changes in the rotor's rotation are captured, further improving the accuracy of capturing the rotor's motion state. Finally, combining the center coordinates and the axis center point coordinates, the UAV's output force and torque are calculated based on aerodynamic principles. Through precise coordinate data and dynamic tracking information, the force and torque can be calculated more accurately. This invention extracts precise image data from image data in highly dynamic environments, accurately perceives the rotor motion state, and improves the accuracy of measuring the output force and torque of UAVs, thereby providing strong technical support for the flight control and performance optimization of UAVs.

[0035] Optionally, the preprocessing of the image data to obtain consecutive frames of a preset length includes: The image data is converted to obtain a grayscale image of the image data. The grayscale image is cropped according to the preset length to obtain grayscale images of multiple consecutive video frames; Each grayscale image is smoothed by Gaussian filtering, and the smoothed grayscale images of each video frame are combined to form the continuous frame.

[0036] Specifically, the original image data is converted into grayscale images. Then, multiple consecutive video frames are cropped into grayscale images according to a preset length. Finally, each grayscale image is smoothed using Gaussian filtering to form consecutive frames. In the preprocessing stage, the acquired color video frames are first converted into grayscale images. Since grayscale images only contain brightness information and not color information, data dimensionality and computational complexity can be reduced. Next, according to the camera's installation position, the video frames are cropped from the midpoint between the two rotors on each side to obtain video frames containing only one rotor. The video frames of all four rotors are processed in parallel. Finally, Gaussian filtering is applied to smooth the grayscale images, eliminating noise interference and subtle textures. In the Gaussian filtering formula, the weights of the Gaussian kernel are determined by the standard deviation of the Gaussian distribution. This standard deviation determines the degree of blurring during filtering, thereby effectively reducing image noise and improving the accuracy of subsequent image processing. In a preferred embodiment of the invention, combined with... Figure 2 As shown, a high-speed camera with a frame rate of 400 frames per second can be used to collect data from a quadcopter drone. The system is equipped with two high-speed cameras, with lenses parallel to the rotors, mounted below the midpoint of the two rotors on each side. Each camera captures images of the two rotors on its corresponding side, ensuring complete coverage of the rotor images. Simultaneously, the images are converted to grayscale and Gaussian filtered. Converting the acquired images to grayscale reduces data dimensionality and computational complexity; then, Gaussian filtering is applied to smooth the images, eliminating noise interference and subtle textures, ultimately yielding continuous frames. Furthermore, based on the camera's mounting position, the continuous frames are cropped from the midpoint between the two rotors on each side, resulting in continuous frames containing only one rotor. The continuous frames corresponding to each of the four rotors are then processed in parallel.

[0037] In this embodiment of the invention, grayscale conversion and Gaussian filtering preprocessing not only reduce the complexity of data processing but also improve image quality, making the rotor motion information more clearly visible. This significantly improves the accuracy of rotor motion information extraction.

[0038] Optionally, the step of extracting multiple center region images of each rotor from the consecutive frames using frame differencing and multi-frame averaging includes: Based on the difference in grayscale values ​​at the same pixel point between any two consecutive video frames in the continuous frame, multiple difference images are obtained; The difference image is binarized according to a first threshold range to obtain the binarized image; The rotor's central region image in each video frame of the consecutive frames is obtained by smoothing each binarized image using a multi-frame averaging method.

[0039] Specifically, frame differencing is used to extract multiple central region images of each rotor from consecutive frames. This process involves calculating the difference in grayscale values ​​at the same pixel point between two consecutive video frames, resulting in multiple difference images. These difference images reflect the rotor's positional changes at consecutive time points, thus highlighting the rotor's dynamic changes. Next, the difference images are binarized according to a first threshold range, converting the images into binary images, simplifying the image data and highlighting the rotor's motion area. In a preferred embodiment of the invention, each frame of binary images is converted to floating-point format, and the pixel values ​​of 15 consecutive frames of binary images are accumulated pixel-by-pixel. The average value is then taken, and the result is converted into an unsigned integer image. This multi-frame averaging method smooths each binarized image, effectively removing interference from dynamic backgrounds and rotor blades, thereby obtaining the central region image of the rotor in each video frame.

[0040] In a preferred embodiment of the present invention, the calculation formula for the frame difference method is as follows: ; in, Indicates a point in time Pixels in the difference image grayscale value; Indicates the current frame pixels grayscale value; Indicates the previous frame pixels grayscale value; This represents absolute value operations.

[0041] The formula for calculating the binarization operation is as follows: ; in, Represents the pixels in the binarized image The value; The threshold for binarization; A fixed value assigned to a pixel to meet certain conditions; high threshold. and low threshold This forms the first threshold range.

[0042] Binary images of consecutive frames are (No. (a binary image of a frame), then continuous The average image of the frames is expressed as: ; where represents the gray value of the image after multi-frame averaging at the pixel point ; N represents the total number of consecutive frames for averaging.

[0043] In the embodiments of the present invention, by combining the frame difference method and the multi-frame averaging method, the central region of the rotor can be effectively extracted from a complex background, which not only improves the accuracy of rotor central region extraction, but also enhances the robustness of the system, enabling it to work stably under different flight conditions. Through the binarization operation, the rotor motion region in the image can be clearly separated from the background, providing clearer image data for subsequent morphological dilation operation and ellipse fitting, thereby improving the measurement accuracy and reliability of the entire system.

[0044] Optionally, the extraction of each of the central region images to obtain the shaft contour of the central axis of the rotor includes: Performing a morphological dilation operation on the central region image to obtain the dilated central region image; Performing a binarization operation on the dilated central region image according to a second threshold range to obtain a result image; Performing edge detection on the result image to obtain the shaft contour of the central axis of the rotor.

[0045] Specifically, when performing a morphological dilation operation on the central region image, an elliptical structuring element of an appropriate size is used to perform multiple dilations on the image, effectively expanding the target region, filling holes, and smoothing the boundaries, thereby repairing the problem of incomplete contours caused by image noise or discontinuous object edges. In the preferred embodiment of the present invention, the defined elliptical structuring element has a width of 9 and a height of 4. By operating through each position in the structuring element and the corresponding pixel value of the input average image, the maximum value is taken and assigned to the corresponding position of the result image. The dilation operation is repeated to gradually expand the target region.

[0046] The calculation formula of the morphological dilation operation is as follows: ; where the dilated central region image is ; are the pixel coordinates of the image; is the defined elliptical structuring element with a width of 9 and a height of 4. Through operation, each position in the structuring element and the corresponding pixel value of the input average image are traversed, and the maximum value is taken and assigned to the result image The corresponding position; by repeating the expansion operation 3 times, the target area is gradually expanded.

[0047] Subsequently, based on the dilated central region image, binarization is performed according to a second threshold range to obtain a binarized image. In this binarized image, pixel values ​​have only two possibilities: 0 and 1, further simplifying the image data and highlighting the rotor shaft outline. Finally, edge detection is performed on the binarized image, typically using the Canny edge detection method, to obtain the rotor's central shaft outline. This allows for the identification of edge information in the image, providing accurate contour data for subsequent ellipse fitting.

[0048] In another preferred embodiment of the present invention, the calculation formula for the binarization operation is as follows: ; in, ; High threshold and low threshold Forming the second threshold range, The result image.

[0049] In this example, preferably, the edge detection method in S3 is the Canny edge detection method.

[0050] For the input result image Calculate the gradient magnitude using the Sobel operator. The formula for calculating the gradient magnitude is: ; in, and They are and The gradient of the direction is obtained using the Sobel operator: ; ; in, and These represent the pixel values ​​of the adjacent pixels to the right and left of the (x,y) pixel, respectively. and These represent the pixel values ​​of the adjacent pixels below and above the (x,y) pixel, respectively.

[0051] For gradient magnitude Non-maximum suppression is performed to refine the edges and retain only local maxima: ; Use high threshold and low threshold Binarization classification is performed on the results after non-maximum suppression to obtain the final edge image containing the rotation axis contour: ; in, ; , This represents the gradient magnitude after nonmaximum suppression. Represents the edge image.

[0052] In this embodiment of the invention, morphological dilation enhances the contour features of the rotor shaft in the image, enabling subsequent binarization and edge detection steps to more accurately identify and extract the shaft contour. Binarization simplifies the image data, allowing the edge detection algorithm to operate more efficiently and more easily distinguish the shaft contour from the background, significantly improving the completeness and accuracy of contour extraction.

[0053] Optionally, the step of performing ellipse fitting on the shaft profile to obtain the coordinates of the rotor shaft center point includes: Obtain the contour area of ​​the rotation axis contour in each of the central region images; Based on the contour area, the rotation axis contours of each of the central region images are sorted to obtain the rotation axis contour with the largest contour area; The shaft profile with the largest profile area is taken as the target profile of the rotor. The target profile is fitted using an ellipse fitting algorithm to obtain the coordinates of the rotor's center point.

[0054] Specifically, the system calculates the contour area of ​​the rotor axis profile in each central region image. Then, based on the calculated contour area, all rotor axis profiles are sorted to identify the profile with the largest area. This largest contour is selected as the target profile for the rotor, based on the assumption that the rotor axis profile is the most significant and largest feature region in the image. Next, since the rotor axis is typically elliptical in the image, an ellipse fitting algorithm is used to fit the target profile. During the fitting process, the contours are first sorted by area, and the largest contour is selected. Then, the ellipse fitting algorithm is used to fit this contour to obtain the coordinates of the ellipse's center point.

[0055] In a preferred embodiment of the present invention, let the extracted set of closed contours be... , where each contour It is a set of points, represented as: ; in, Represents outline The coordinates of the point in the middle; Sort the contours by area and select the contour with the largest area. The formula for calculating its area is: ; The points are arranged in a counter-clockwise direction. , The area of ​​the contour is represented by m, and the contour area is represented by m. The number of midpoints.

[0056] For the selected set of contour points Perform least-squares ellipse fitting to obtain the coordinates of the center point of the fitted ellipse. Long half shaft and short half shaft .

[0057] In this embodiment of the invention, by sorting the contour areas and selecting the largest contour as the target contour, it can be ensured that the fitting process is targeted at the actual rotor shaft, thereby improving the accuracy of the measurement. The ellipse fitting algorithm can adapt to the geometry of the rotor shaft, providing accurate fitting results even when the shaft contour is irregular or has slight deviations.

[0058] Optionally, detecting key points of a preset marked image for each rotor using a feature matching algorithm includes: Preset key points are extracted from a preset template image using the ORB algorithm; The preset key points are matched with the preset marker image of the rotor by brute force matching to obtain the key points of the preset marker image.

[0059] Specifically, the preset template image is a pre-selected standard image containing circular marks on the rotor. Preset key points are extracted from this image using the ORB algorithm and used as a matching benchmark. The key points in the preset template image are matched with the preset mark image on the current rotor to determine the key points of the preset mark image. The preset mark image is typically a circular mark image, used to detect and recognize the circular marks on the rotor. Specifically, firstly, the ORB (Oriented Fast and Rotated BRIEF) algorithm is used to extract preset key points from the preset template image. After extracting the key points, a brute-force matching method is used to match these preset key points with the preset mark image on the rotor, such as a circular mark. Brute-force matching is a simple feature matching method that finds the best match by calculating the distance between key point descriptors. In a preferred embodiment of the invention, the feature matching algorithm is as follows: The ORB algorithm processes the input image. Perform keypoint detection to obtain a set of keypoints: ; Among them, each key point Includes its two-dimensional coordinates ,direction and descriptors ; descriptor It is a vector composed of binary features, used for subsequent matching; Let the set of key points of the preset template image be... The key point set of the rotor image is Key points for each rotor image The Hamming distance between the target image and all keypoints in the template image is calculated using a brute-force matching algorithm. ; in, and They are respectively and binary descriptor, For the length of the descriptor; For each Select the template keypoint with the smallest Hamming distance. : ; in, This represents the set of all key points in the target image. Representing key points in the reference image Key points in the target image with the minimum Hamming distance.

[0060] Use the nearest neighbor ratio test to filter matching pairs and remove incorrect matches: ; in, and Let these represent the minimum and second smallest Hamming distances of the current keypoint, respectively. This is an empirical threshold (usually taken as 0.7). The final output is the set of matched keypoint pairs after filtering: .

[0061] In this embodiment of the invention, the combined use of the ORB algorithm and brute-force matching enables efficient and accurate extraction and identification of key feature points of the rotor from images. Precise key point detection helps improve the accuracy of subsequent center coordinate calculations, thereby affecting the measurement of rotor speed and direction. This not only improves the accuracy and reliability of the measurement but also ensures its real-time performance.

[0062] Optionally, obtaining the center coordinates of the key points when the rotor rotates based on the key points includes: By using a fitting algorithm, a preset number of key points are randomly selected for fitting, and the center coordinates and radii of multiple fitted circles are obtained. Obtain the nearest distance between each key point and the fitted circle, and extract interior points from the key points based on the nearest distance; The fitted circle with the most interior points is taken as the final circle, and least-squares circle fitting is performed on the final circle to obtain the center coordinates of the key points in each central region image.

[0063] Specifically, firstly, a preset number of key points are randomly selected using a fitting algorithm to obtain the center coordinates and radii of multiple fitted circles. The RANSAC (Random Sample Consensus) algorithm is typically used as an iterative method to estimate model parameters from a dataset containing outliers. In a preferred embodiment of the invention, three points are randomly selected for fitting, and the center coordinates and radii of the candidate circles are calculated. Next, the nearest distance between each key point and the fitted circle is obtained, and inliers are extracted from the key points based on this distance; points closer to the fitted circle are typically marked as inliers if the distance is less than a preset threshold (e.g., 20 pixels). Then, by comparing the number of inliers, the fitted circle with the most inliers is selected as the final circle, representing the best fitted circle for the marked points on the rotor. Finally, least-squares circle fitting is performed on the final circle to obtain the center coordinates of the key points within each central region image. The calculation formula for the RANSAC circle fitting algorithm is as follows: Suppose that three points are randomly selected from the set of key points. The coordinates of the center of the circle are: ; ; in, , and Represents the coordinates of the center point of the fitted ellipse, and the radius. Represents the radius of the fitted ellipse, radius .

[0064] In this embodiment of the invention, the rotor's rotation center can be effectively identified from key points by combining the RANSAC algorithm and least-squares circle fitting. This method not only handles noise and outliers in the image but also accurately fits the rotor's rotation trajectory, thereby improving the accuracy of the circle center coordinate calculation.

[0065] Optionally, determining the output force and torque of the UAV based on the coordinates of the center of the circle and the coordinates of the center point of the rotation axis includes: Two video frames are extracted from the consecutive frames at a preset interval; The center coordinates are tracked by Kalman filtering. Based on the angles formed by the center coordinates of the central region images in the two video frames and the coordinates of the center point of the rotating shaft, the instantaneous speed and forward / reverse rotation information of each rotor are determined. The output force and torque of the UAV are determined based on the instantaneous speed of the rotor and the forward and reverse rotation information.

[0066] Specifically, for the dynamic tracking of the center, the initial coordinates of the circular marker's center can be used as the initial values ​​of the state variables, including estimates of position, velocity, and acceleration. A Kalman filter-based state transition model is used to predict the possible position of the center at the next moment. The center coordinates detected using feature matching algorithms and geometric fitting methods are used as observations, input into the Kalman filter, and compared with the predicted values. Based on the error between the predicted and observed values, the Kalman gain is used to correct the state variables, filtering out random errors caused by noise or jitter, thus improving the positioning accuracy of the center coordinates. The state prediction and filtering correction are repeated to continuously and dynamically track the center position of the rotor's circular marker.

[0067] The calculation method for Kalman filtering is as follows: The coordinates of the center of the rotor are represented by a six-dimensional state vector: ; in, , The current coordinates of the center of the circle; , The velocity component at the center of the circle; , The acceleration component is the center of the circle; The observed value z represents the position of the center of the circle. ; The state transition equation is: ; ; in, Indicates process noise. Represents the state transition matrix. Indicates at time point k 1 is a state variable. This represents the state variable at time point k.

[0068] The observation equation is: ; in, For the observation matrix, This is the observation noise; H is: ; The calculation formula for the prediction stage of Kalman filtering is as follows: ; ; in, Indicates time The predicted state; The prediction error covariance matrix represents the uncertainty of the prediction results. Let represent the state transition matrix, and Q represent the process noise covariance matrix.

[0069] The calculation formula for the Kalman filter update stage is as follows:

[0070] in, Kalman gain; For the updated optimal state estimate; Let R be the updated error covariance matrix, and let R be the observation noise covariance matrix.

[0071] Two video frames are extracted from consecutive frames at a preset interval, for example, every 50 frames. Then, the angle formed by the coordinates of the center of the image in the central region of these two video frames and the coordinates of the center point of the rotation axis is calculated. The instantaneous velocity and forward / reverse rotation information of each rotor are calculated using the instantaneous velocity calculation formula. The instantaneous velocity calculation formula is:

[0072] in, Let be the Euclidean distance between the coordinates of the center of the rotor and the coordinates of the center of the shaft. This is the time interval between two frames.

[0073] When calculating the forward and reverse rotation information, the specific data from the camera that acquired the image is used to calculate the forward and reverse rotation of the rotor. For example, if the camera frame rate is 400 frames per second and the maximum rotor speed is 8000 revolutions per minute (i.e., 133.33 revolutions per second), the angle of rotation of the rotor per second is... The time interval between two frames is seconds, corresponding to a rotation angle of Therefore, the rotor angle difference between any two frames must be less than [a certain value]. After obtaining the coordinates of the center point of the rotor in step S3, the polar angle difference between the center coordinates of the rotor and the center coordinates of the rotor is calculated every 50 frames. If the polar angle difference is positive (clockwise), the rotor is rotating in the forward direction; if the polar angle difference is negative (counterclockwise), the rotor is rotating in the reverse direction.

[0074] The method for calculating the polar angle difference is as follows: Center point of rotation Using the origin as the reference point, calculate the circular marker points on the rotor. Polar angles in the current frame and the previous frame The formula for calculating the polar angle is: ; in, It is a two-parameter arctangent function, which ensures that the angle result lies in the... ; Let the polar angle of the marker point in the previous frame be... The polar angle of the current frame marker point is Then the angle difference for: ; Due to the angle exist Within the range, jumps may occur (e.g., from...). arrive Therefore, it is necessary to... Normalize to ensure the results are within the range of Within the range: ; like Clockwise, the rotor rotates in the correct direction. Then, the rotor rotates counterclockwise, reversing its direction.

[0075] In this embodiment of the invention, by accurately measuring the instantaneous speed and forward / reverse rotation information of the rotor, the output force and torque of the UAV can be calculated more precisely. This method not only improves the accuracy of the calculation but also makes the measurement results more reliable, providing strong technical support for the flight control and performance optimization of the UAV.

[0076] Optionally, determining the output force and torque of the UAV based on the instantaneous speed of the rotor and the forward / reverse rotation information includes: The real-time rotational speed of each rotor is determined based on the instantaneous speed of the rotor and the forward / reverse rotation information; The total thrust of the UAV is determined based on the real-time rotational speed and thrust coefficient of the rotor, and the total thrust is used as the output force. Based on the distance between each rotor and the center of the UAV, combined with the real-time rotational speed of the rotor and the thrust coefficient, the total pitch moment, total roll moment and total yaw moment of the UAV are determined, and the total pitch moment, total roll moment and total yaw moment are used as the torque of the UAV.

[0077] Specifically, first, the system calculates the real-time rotational speed of each rotor based on its instantaneous speed and rotational direction. For example, if the camera frame rate is 400 frames per second and the maximum rotor speed is 8000 revolutions per minute (133.33 revolutions per second), the rotor speed can be determined by calculating the angle difference between the rotors between two frames. Next, using the real-time rotor speed and thrust coefficient, the system calculates the total thrust of the UAV and uses this total thrust as the UAV's output force. Furthermore, based on the distance between each rotor and the center of the UAV, combined with the real-time rotor speed and thrust coefficient, the system further determines the UAV's total pitch moment, total roll moment, and total yaw moment.

[0078] In a preferred embodiment of the present invention, combined with Figure 2 As shown, a quadcopter drone is used as an example. Assume the quadcopter has an X-shaped configuration, and the body coordinate system is a forward-right-down (FRD) right-handed system, with its x-axis pointing forward, y-axis pointing to the right side of the drone, and z-axis pointing downwards. The world coordinate system is a north-east-ground (NED) right-handed system, with the z-axis perpendicular to the ground and pointing downwards. The rotors are numbered counter-clockwise from the top right corner as 1, 2, 3, 4, etc. Figure 2 As shown.

[0079] Thrust of each rotor Its rotational speed It is proportional to the square of: ; in, Each represents one of the four rotors. It is the thrust coefficient; Torque of each rotor for: ; in, This indicates the distance between the rotor and the center of gravity of the drone; Total pitch moment for: ; Total Rolling Moment for: ; Total Yaw Moment for: ; The resultant force of the UAV in the world coordinate system is calculated by superimposing the total thrust with gravity after mapping through the attitude rotation matrix. Specifically: ; in, The total thrust of the quadrotor in the airframe coordinate system. , , These are the resultant forces in the x, y, and z directions in the world coordinate system. These are roll, pitch, and yaw angles, respectively. For the quality of drones, This is the acceleration due to gravity.

[0080] In this embodiment of the invention, by accurately measuring the instantaneous speed and forward / reverse rotation information of the rotor, the output force and torque of the UAV can be calculated more precisely. This method not only improves the accuracy of force and torque calculations but also makes the flight control of the UAV more precise and reliable. By monitoring the force and torque of the UAV in real time, a better understanding of the UAV's performance under different flight conditions can be achieved.

[0081] Combination Figure 3 As shown, the present invention also provides a UAV output force and torque detection system, comprising: The image acquisition unit is used to acquire image data of each rotor of the UAV and preprocess the image data to obtain continuous frames of a preset length. The image processing unit is used to extract multiple central region images of each rotor from the consecutive frames using frame difference and multi-frame averaging methods. The center point coordinate extraction unit is used to extract the center region image of each of the central regions to obtain the axis profile of the rotor's central axis, and then perform ellipse fitting on the axis profile to obtain the center point coordinates of the rotor's axis. The center coordinate extraction unit is used to detect key points of a preset marked image of each rotor using a feature matching algorithm, and to obtain the center coordinates of the key points when the rotor rotates based on the key points. The calculation unit is used to determine the output force and torque of the UAV based on the coordinates of the circle center and the coordinates of the center point of the rotation axis.

[0082] The advantages of the UAV output force and torque detection system of the present invention compared with the prior art are the same as those of the above-mentioned UAV output force and torque detection method compared with the prior art, and will not be repeated here.

[0083] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the method for detecting the output force and torque of a UAV as described above.

[0084] Alternatively, a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the following operations: Acquire image data of each rotor of the UAV, and preprocess the image data to obtain continuous frames of a preset length; Multiple central region images of each rotor are extracted from the consecutive frames using frame difference and multi-frame averaging methods. Extract the image of each central region to obtain the axis profile of the central axis of the rotor, and then perform ellipse fitting on the axis profile to obtain the coordinates of the center point of the rotor axis; By using a feature matching algorithm, key points of a preset marked image for each rotor are detected, and the center coordinates of the key points are obtained based on the key points when the rotor rotates. The output force and torque of the UAV are determined based on the coordinates of the center of the circle and the coordinates of the center point of the rotating shaft.

[0085] The advantages of the computer-readable storage medium of the present invention compared to the prior art are the same as the advantages of the above-mentioned UAV output force and torque detection method compared to the prior art, and will not be repeated here.

[0086] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for detecting the output force and torque of a UAV, characterized in that, include: Image data of each rotor of the UAV is acquired, and the image data is preprocessed to obtain continuous frames of a preset length. Multiple central region images of each rotor are extracted from the consecutive frames using frame difference and multi-frame averaging methods. Extract the image of each central region to obtain the axis profile of the central axis of the rotor, and then perform ellipse fitting on the axis profile to obtain the coordinates of the center point of the rotor axis; By using a feature matching algorithm, key points of a preset marked image of each rotor are detected, and the center coordinates of the key points are obtained based on the key points when the rotor rotates. The output force and torque of the UAV are determined based on the coordinates of the center of the circle and the coordinates of the center point of the rotating shaft. Specifically, this includes: extracting two video frames from the consecutive frames at a preset interval; tracking the center of the circle using Kalman filtering; determining the instantaneous speed and forward / reverse rotation information of each rotor based on the angles formed by the center of the circle coordinates of the central region images in the two video frames and the coordinates of the center point of the rotating shaft; and determining the output force and torque of the UAV based on the instantaneous speed and forward / reverse rotation information of the rotor. The step of determining the output force and torque of the UAV based on the instantaneous speed of the rotor and the forward / reverse rotation information includes: Based on the instantaneous speed of the rotors and the forward / reverse rotation information, the real-time rotational speed of each rotor is determined; based on the real-time rotational speed of the rotors and the thrust coefficient, the total thrust of the UAV is determined, and the total thrust is used as the output force; based on the distance between each rotor and the center of the UAV, combined with the real-time rotational speed of the rotors and the thrust coefficient, the total pitch moment, total roll moment, and total yaw moment of the UAV are determined, and the total pitch moment, total roll moment, and total yaw moment are used as the torque of the UAV.

2. The method for detecting the output force and torque of a UAV according to claim 1, characterized in that, The preprocessing of the image data to obtain continuous frames of a preset length includes: The image data is converted to obtain a grayscale image of the image data. The grayscale image is cropped according to the preset length to obtain grayscale images of multiple consecutive video frames; Each grayscale image is smoothed by Gaussian filtering, and the smoothed grayscale images of each video frame are combined to form the continuous frame.

3. The method for detecting the output force and torque of a UAV according to claim 1, characterized in that, The step of extracting multiple center region images of each rotor from the consecutive frames using frame differencing and multi-frame averaging includes: Based on the difference in grayscale values ​​at the same pixel point between any two consecutive video frames in the continuous frame, multiple difference images are obtained; The difference image is binarized according to a first threshold range to obtain a binarized image; The rotor's central region image in each video frame of the consecutive frames is obtained by smoothing each binarized image using a multi-frame averaging method.

4. The method for detecting the output force and torque of a UAV according to claim 1, characterized in that, The step of extracting the image of each central region to obtain the rotation axis profile of the rotor's central axis includes: A morphological dilation operation is performed on the central region image to obtain the dilated central region image; Based on the expanded central region image, a binarization operation is performed according to the second threshold range to obtain the result image; Edge detection is performed on the resulting image to obtain the profile of the central axis of the rotor.

5. The method for detecting the output force and torque of a UAV according to claim 1, characterized in that, The step of performing ellipse fitting on the rotor shaft profile to obtain the coordinates of the rotor shaft center point includes: Obtain the contour area of ​​the rotation axis contour in each of the central region images; Based on the contour area, the rotation axis contours of each of the central region images are sorted to obtain the rotation axis contour with the largest contour area; The shaft profile with the largest profile area is taken as the target profile of the rotor. The target profile is fitted using an ellipse fitting algorithm to obtain the coordinates of the rotor's center point.

6. The method for detecting the output force and torque of a UAV according to claim 1, characterized in that, The step of detecting key points of a preset marked image for each rotor using a feature matching algorithm includes: Preset key points are extracted from a preset template image using the ORB algorithm; The preset key points are matched with the preset marker image of the rotor by brute force matching to obtain the key points of the preset marker image.

7. The method for detecting the output force and torque of a UAV according to claim 1, characterized in that, The step of obtaining the center coordinates of the key points of the rotor during rotation based on the key points includes: By using a fitting algorithm, a preset number of key points are randomly selected for fitting, and the center coordinates and radii of multiple fitted circles are obtained. Obtain the nearest distance between each key point and the fitted circle, and extract interior points from the key points based on the nearest distance; The fitted circle with the most interior points is taken as the final circle, and least-squares circle fitting is performed on the final circle to obtain the center coordinates of the key points in each central region image.

8. A UAV output force and torque detection system applied to the UAV output force and torque detection method of claim 1, characterized in that, include: The image acquisition unit is used to acquire image data of each rotor of the UAV and preprocess the image data to obtain continuous frames of a preset length. The image processing unit is used to extract multiple central region images of each rotor from the consecutive frames using frame difference and multi-frame averaging methods. The center point coordinate extraction unit is used to extract the center region image of each of the central regions to obtain the axis profile of the rotor's central axis, and then perform ellipse fitting on the axis profile to obtain the center point coordinates of the rotor's axis. The center coordinate extraction unit is used to detect key points of a preset marked image of each rotor using a feature matching algorithm, and to obtain the center coordinates of the key points when the rotor rotates based on the key points. The calculation unit is used to determine the output force and torque of the UAV based on the coordinates of the circle center and the coordinates of the center point of the rotation axis.

Images