Target detection method based on high-speed visual camera

Abstract

The invention provides a target detection method based on a high-speed vision camera, and belongs to the technical field of computer vision, and the method comprises the steps: collecting a target image through the high-speed vision camera carried on an unmanned plane; graying and de-noising the target image to obtain a pre-processed target image; dynamically adjusting a threshold based on an improved adaptive threshold algorithm, eliminating shadow interference in the preprocessed target image, and obtaining an optimized binary target image; contour detection and polygon approximation are carried out according to the optimized binary target image, target feature information is extracted, the matching degree of the target feature information and a preset visual identifier is detected, and the landing platform of the unmanned aerial vehicle is locked. According to the method, the target image can be effectively identified and processed in an uneven illumination, shadow shielding or complex background environment, so that the requirement of real-time detection in a dynamic environment is met.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a target detection method based on a high-speed vision camera. Background Technology

[0002] Unmanned vehicles, abbreviated as "drones" or URMs, are devices operated using radio remote control equipment and onboard program control devices, or operated autonomously, either completely or intermittently, by an onboard computer.

[0003] During autonomous landing, the UAV's visual measurement unit (VMU) identifies and detects the landing platform, automatically controlling the UAV to land. However, current VMUs typically employ fixed-threshold algorithms for image recognition. For example, they extract target features from the landing platform image using color filtering (such as RGB separation) or fixed-threshold binary methods. However, these methods suffer from low image processing efficiency and high false detection rates in environments with uneven lighting, shadows, or complex backgrounds, and cannot meet the demands of real-time detection in dynamic environments. Furthermore, the delay between the VMU and UAV control systems results in poor real-time control, leading to landing deviations.

[0004] Therefore, a target detection method based on a high-speed vision camera is proposed. Summary of the Invention

[0005] To address some or all of the technical problems existing in the prior art, the present invention provides a target detection method based on a high-speed vision camera, which solves the problems of low efficiency and high false detection rate in traditional technology for image recognition and detection of landing platforms, resulting in deviations in UAV landing.

[0006] The technical solution of the present invention is as follows:

[0007] A target detection method based on a high-speed vision camera is provided, including:

[0008] Target images are acquired using a high-speed vision camera mounted on a drone;

[0009] The target image is subjected to grayscale conversion and noise reduction to obtain a preprocessed target image;

[0010] The threshold is dynamically adjusted based on an improved adaptive thresholding algorithm to eliminate shadow interference in the preprocessed target image and obtain an optimized binary target image.

[0011] Based on the optimized binary target image, contour detection and polygon approximation are performed to extract target feature information, and the matching degree between the target feature information and the preset visual identifier is detected to lock the landing platform of the UAV.

[0012] Furthermore, in the target detection method based on a high-speed vision camera provided in this embodiment of the invention, the improved adaptive threshold algorithm is expressed as:

[0013] T(x,y)=u(x,y)·(1+k·σ(x,y));

[0014] Where T(x,y) represents the adaptive threshold at pixel (x,y) in the image, u(x,y) represents the mean gray level within a set window centered at pixel (x,y), σ(x,y) represents the standard deviation of gray level within a set window centered at pixel (x,y), and k represents the dynamic adjustment coefficient.

[0015] Furthermore, in the target detection method based on a high-speed vision camera provided in the embodiments of the present invention, the preset visual identifier is designed based on the double square characteristic.

[0016] Furthermore, in the target detection method based on a high-speed visual camera provided in the embodiments of the present invention, the preset visual identifier is based on a double square structure and an arrow graphic design, including: a first rectangular frame, a second rectangular frame located in the middle of the second rectangular frame, a first arrow graphic located in the first rectangular frame, and a second arrow graphic located in the middle of the second rectangular frame.

[0017] Furthermore, in the target detection method based on a high-speed visual camera provided in this embodiment of the invention, the preset visual identifier is based on a double square structure and an H graphic design, including: a first rectangular frame, a second rectangular frame located in the middle of the second rectangular frame, and an H graphic located in the middle of the second rectangular frame.

[0018] Furthermore, in the target detection method based on a high-speed vision camera provided in this embodiment of the invention, the method further includes:

[0019] Calculate the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data, and roll angle data of the UAV relative to the landing platform;

[0020] The drone's speed and acceleration are collected in real time by using an inertial measurement unit mounted on the drone.

[0021] Based on the relative position prediction model, the relative pose information between the UAV and the landing platform is updated in real time using an interpolation algorithm, based on the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data, and roll angle data of the UAV relative to the landing platform, as well as the speed and acceleration of the UAV.

[0022] After receiving the landing command, the drone plans its trajectory based on the relative pose information and controls the drone to land on the landing platform in stages.

[0023] The main advantages of the technical solution of this invention are as follows:

[0024] The target detection method based on a high-speed vision camera of the present invention eliminates shadow interference in the preprocessed target image by improving the adaptive threshold algorithm, obtains an optimized binary target image, and performs contour detection and polygon approximation based on the optimized binary target image, thereby realizing the extraction of target feature information in the target image. By detecting the matching degree between the target feature information and the preset visual identifier, the method can lock the UAV landing platform. This solves the problems of low efficiency and high feature false detection rate in traditional technologies that use color filtering or fixed threshold binary methods for image processing. The above method uses an improved adaptive threshold algorithm for image processing, enabling UAVs to effectively identify and process target images in environments with uneven lighting, shadow occlusion, or complex backgrounds, thereby meeting the needs of real-time detection in dynamic environments. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the target detection method based on a high-speed vision camera provided by the present invention.

[0027] Figure 2 This is a schematic diagram of a target image provided by the target detection method based on a high-speed vision camera according to the present invention.

[0028] Figure 3 This is a schematic diagram of an optimized binary target image for the target detection method based on a high-speed vision camera provided by the present invention.

[0029] Figure 4 This is a schematic diagram of a feature contour image for the target detection method based on a high-speed vision camera provided by the present invention.

[0030] Figure 5 This is a schematic diagram of target feature information for the target detection method based on a high-speed vision camera provided by the present invention.

[0031] Figure 6 This is a schematic diagram of the preset visual identifier for the target detection method based on a high-speed vision camera provided by the present invention;

[0032] Figure 7 This is a flowchart illustrating the UAV landing platform based on the target detection method using a high-speed vision camera provided by the present invention.

[0033] Figure 8 This is a schematic diagram of the UAV control structure for the target detection method based on a high-speed vision camera provided by the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0035] The technical solutions provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0036] This invention provides a target detection method based on a high-speed vision camera, with reference to... Figure 1 ,include:

[0037] Target images are acquired using a high-speed vision camera mounted on a drone;

[0038] The target image is converted to grayscale and denoised to obtain a preprocessed target image;

[0039] An improved adaptive thresholding algorithm is used to dynamically adjust the threshold, eliminate shadow interference in the preprocessed target image, and obtain an optimized binary target image.

[0040] Based on the optimized binary target image, contour detection and polygon approximation are performed to extract target feature information, and the matching degree between the target feature information and the preset visual identifier is detected to lock the landing platform of the UAV.

[0041] In the above embodiments, a target image is acquired by a high-speed vision camera mounted on the UAV, and the target image is grayscaled and denoised to obtain a preprocessed target image; the threshold is dynamically adjusted based on an improved adaptive threshold algorithm to eliminate shadow interference in the preprocessed target image and obtain an optimized binary target image; contour detection and polygon approximation are performed based on the optimized binary target image to extract target feature information, and the matching degree between the target feature information and the preset visual identifier is detected to lock the landing platform of the UAV.

[0042] In one embodiment, reference Figure 2 and Figure 3 The target image is grayscaled and denoised, and the threshold is dynamically adjusted based on an improved adaptive thresholding algorithm to eliminate shadow interference in the preprocessed target image, thereby obtaining an optimized binary target image.

[0043] In one embodiment, reference Figure 4 and Figure 5 Based on the optimized binary target image, contour detection and polygon approximation are performed to obtain the feature contour image, and non-target contours are excluded to obtain the target feature information.

[0044] In one embodiment, the preset visual signage is designed based on the characteristics of a double square.

[0045] In one embodiment, reference Figure 6 The preset visual identity is based on a double square structure and arrow graphic design, specifically including: a first rectangle, a second rectangle located in the middle of the second rectangle, a first arrow graphic located in the first rectangle, and a second arrow graphic located in the middle of the second rectangle.

[0046] based on Figure 6 The preset visual markers shown include rectangles that can be used to calculate the drone's position relative to the target, and arrows that provide angle information. When the drone is far from the target, its position can be estimated using the outer first rectangle and arrow. When the drone is close to the target, the outer first marker may no longer be within the camera's field of view; in this case, the drone's position can be estimated using the inner second rectangle and arrow. This allows for a more accurate estimation of the drone's position relative to the target.

[0047] In one embodiment, the preset visual signage is based on a multi-stage design to achieve seamless switching between near and far distances, with a position calculation error of ≤5cm.

[0048] In one embodiment, the preset visual identity is based on a double square structure and an H graphic design, specifically including: a first rectangular frame, a second rectangular frame located inside the second rectangular frame, and an H graphic located inside the second rectangular frame.

[0049] In one embodiment, the preset visual identifier is based on a concentric circle structure or a QR code graphic design.

[0050] The beneficial effects of the above technologies are as follows: Based on the improved adaptive threshold algorithm, shadow interference in the preprocessed target image is eliminated to obtain an optimized binary target image. Contour detection and polygon approximation are then performed based on the optimized binary target image, enabling the extraction of target feature information from the target image. By detecting the matching degree between the target feature information and the preset visual identifier, the landing platform of the UAV can be locked. This solves the problems of low efficiency and high false detection rate of traditional image processing methods using color filtering or fixed threshold binary methods. Compared with traditional technologies, the target detection method based on a high-speed visual camera uses an improved adaptive threshold algorithm for image processing, enabling the UAV to effectively identify and process target images in environments with uneven lighting, shadow occlusion, or complex backgrounds, thereby meeting the needs of real-time detection in dynamic environments.

[0051] Furthermore, in this embodiment of the invention, the improved adaptive threshold algorithm is expressed as:

[0052] T(x,y)=u(x,y)·(1+k·σ(x,y));

[0053] Where T(x,y) represents the adaptive threshold at pixel (x,y) in the image, u(x,y) represents the mean gray level within a set window centered at pixel (x,y), σ(x,y) represents the standard deviation of gray level within a set window centered at pixel (x,y), and k represents the dynamic adjustment coefficient.

[0054] The dynamic adjustment coefficient ranges from 0.1 to 0.3.

[0055] The size of the window can be set according to actual needs.

[0056] In this embodiment of the invention, the improved adaptive thresholding algorithm dynamically calculates the threshold of each pixel using local mean and standard deviation. By dynamically adjusting the local threshold, it can eliminate shadow interference in the preprocessed target image, obtain a high-quality optimized binary target image, and improve the robustness of image binarization.

[0057] Furthermore, in an optional embodiment of the present invention, the target detection method based on a high-speed vision camera provided by the present invention further includes:

[0058] Calculate the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data, and roll angle data of the UAV relative to the landing platform;

[0059] The drone's speed and acceleration are collected in real time by using an inertial measurement unit mounted on the drone.

[0060] Based on the relative position prediction model, the relative pose information between the UAV and the landing platform is updated in real time using an interpolation algorithm, based on the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data, and roll angle data of the UAV relative to the landing platform, as well as the speed and acceleration of the UAV.

[0061] After receiving the landing command, the drone plans its trajectory based on the relative pose information and controls the drone to land on the landing platform in stages.

[0062] In this embodiment of the invention, based on the landing platform position of the UAV in the target image, the rotation matrix and translation vector of the high-speed vision camera relative to the landing platform are solved by a conventional pose calculation algorithm. Then, the horizontal offset data, vertical height data, yaw angle data, pitch angle data and roll angle data of the UAV relative to the landing platform are calculated based on the rotation matrix and translation vector.

[0063] In this embodiment of the invention, the relative pose information between the UAV and the landing platform includes: relative displacement information and relative attitude information; wherein, the relative displacement information includes: horizontal offset data and vertical altitude data of the UAV relative to the landing platform; the relative attitude information includes: yaw angle data, pitch angle data and roll angle data of the UAV relative to the landing platform.

[0064] When the UAV navigates to the vicinity of the landing platform, it activates its high-speed vision camera to acquire target images and processes them to determine the landing platform's location. During this image processing period, the UAV's flight state continuously changes. The relative pose information between the UAV and the landing platform calculated based on the image processing results is outdated. If this past relative pose information is directly transmitted to the UAV for flight control, the UAV's state will always be lagging behind in time, leading to decreased landing accuracy. Therefore, in this embodiment of the invention, considering the delay caused by image processing, a relative position prediction model and interpolation algorithm are used to predict the relative pose information of the next frame, updating the relative pose information between the UAV and the landing platform in real time, thereby improving landing accuracy.

[0065] The target detection method based on a high-speed vision camera provided in this invention collects the speed and acceleration of the UAV in real time through an inertial measurement unit, and predicts and updates the relative pose information between the UAV and the landing platform based on a relative position prediction model and an interpolation algorithm. This improves the real-time performance of UAV control, effectively reduces the control delay of the UAV, and improves the control accuracy of the UAV landing on the landing platform.

[0066] Furthermore, in this embodiment of the invention, the relative position prediction model is expressed as:

[0067]

[0068] Among them, P t+1 P represents the relative pose of the UAV and the landing platform at time t+1. t V represents the relative pose of the UAV and the landing platform at time t. t Let a represent the velocity of the drone at time t. t Let t represent the acceleration of the drone at time t, and Δt represent the time difference between time t+1 and time t.

[0069] In this embodiment of the invention, the relative pose of the UAV and the landing platform at the next moment is predicted by the above-mentioned relative position prediction model, which can effectively solve the delay problem caused by the image processing process and improve the control accuracy of the UAV landing based on the landing platform.

[0070] Furthermore, in this embodiment of the invention, the segmented control of the drone to land on the landing platform includes: controlling the drone to fly directly above the landing platform, controlling the drone to track the landing platform, and controlling the drone to reduce its flight altitude and land on the landing platform.

[0071] In this embodiment of the invention, the time taken for the UAV to perform trajectory planning based on relative pose information after receiving the landing command is ≤2 seconds, and the control response time of the UAV is <50ms.

[0072] In this embodiment of the invention, the segmented control of the UAV to land on the landing platform through the above-described method can improve the control accuracy of the UAV landing on the landing platform.

[0073] Furthermore, in an optional embodiment of the present invention, after the UAV receives the landing command, trajectory planning is performed based on the relative pose information, and the UAV is controlled to land on the landing platform in stages, including:

[0074] After the drone receives the landing command, a mission assessment is performed to determine whether the drone currently meets the conditions for executing the command.

[0075] When it is determined that the UAV meets the command execution conditions, the UAV’s trajectory is planned based on the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data and roll angle data relative to the landing platform. The UAV is controlled to fly directly above the landing platform and track the landing platform in real time.

[0076] When the altitude data between the drone and the landing platform is detected to be less than a preset altitude threshold, the drone is controlled to land on the landing platform.

[0077] In this embodiment of the invention, after the UAV receives the landing command, it performs a mission evaluation to determine whether executing the landing command will affect its own safety and whether the execution conditions are met. If the execution conditions are met, the UAV plans and optimizes its trajectory based on the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data, and roll angle data of the UAV relative to the landing platform. The UAV is controlled to fly directly above the landing platform and tracks the landing platform in real time. When the altitude data between the UAV and the landing platform is detected to be less than a preset altitude threshold, the UAV is controlled to land on the landing platform.

[0078] In this embodiment of the invention, by performing a mission evaluation after the UAV receives the landing command, the landing command is analyzed, avoiding the impact on the flight safety of the UAV by directly executing the landing command without evaluation. Furthermore, the UAV is controlled to track the landing platform based on the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data, and roll angle data of the UAV relative to the landing platform. The UAV only lands when its altitude data is lower than a preset altitude threshold. Through segmented control, the landing safety of the UAV is effectively guaranteed.

[0079] Furthermore, in an optional embodiment of the present invention, after the UAV receives the landing command, trajectory planning is performed based on the relative pose information, and the UAV is controlled to land on the landing platform in stages, which further includes:

[0080] If the target is lost during the drone's flight while tracking the landing platform, the drone should be controlled to adjust its flight altitude and search for the landing platform again.

[0081] refer to Figure 7 In this embodiment of the invention, after locking onto the landing platform and obtaining the landing command, the UAV is controlled to fly directly above the landing platform based on the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data and roll angle data of the UAV relative to the landing platform, and the UAV is tracked in real time. When the altitude data between the UAV and the landing platform is detected to be less than a preset altitude threshold, the UAV is controlled to land on the landing platform.

[0082] In this embodiment of the invention, the drone tracks the landing platform. If the target of the landing platform tracked by the drone is lost for more than 3 seconds, the drone is controlled to increase its flight altitude by 200 centimeters and search for the landing platform again.

[0083] In this embodiment of the invention, the flight altitude of the drone must not exceed the flight altitude threshold.

[0084] In this embodiment of the invention, when the target of the landing platform tracked by the UAV is lost, the flight altitude of the UAV is adjusted to re-search for the landing platform, thereby realizing a secondary search for the landing platform when the tracked target is lost, and thus realizing automatic tracking and detection of the landing platform.

[0085] refer to Figure 8 Furthermore, in an optional embodiment of the present invention, after the UAV receives the landing command, trajectory planning is performed based on the relative pose information, and the UAV is controlled to land on the landing platform in stages, including:

[0086] The outer loop controller receives the trajectory planning command, which is then parsed and processed by the outer loop controller to obtain the trajectory control command, and the inner loop controller controls the flight of the UAV.

[0087] The first attitude trajectory information of the UAV is collected in real time by the internal loop sensor. The deviation between the first attitude trajectory information and the trajectory control command is analyzed to obtain the first compensation command. The internal loop controller optimizes the trajectory control command to control the flight of the UAV.

[0088] The second attitude trajectory information of the UAV is collected in real time by the external loop sensor. The deviation between the second attitude trajectory information and the trajectory planning command is analyzed to obtain the second compensation command. The external loop controller then optimizes the trajectory planning command to control the flight of the UAV.

[0089] In this embodiment of the invention, the UAV is controlled by a dual-loop control structure to land on the landing platform according to the obtained trajectory planning instructions.

[0090] In this embodiment of the invention, the outer loop controller parses and processes the trajectory planning command to obtain the trajectory control command, and controls the UAV flight through the inner loop controller.

[0091] In this embodiment of the invention, the inner loop sensor collects the first attitude trajectory information of the UAV in real time, analyzes the deviation between the first attitude trajectory information and the trajectory control command, obtains the first compensation command, and the inner loop controller optimizes the trajectory control command to control the flight of the UAV, thereby stabilizing the flight attitude of the UAV.

[0092] In this embodiment of the invention, the external loop sensor collects the second attitude trajectory information of the UAV in real time, analyzes the deviation between the second attitude trajectory information and the trajectory planning command, obtains the second compensation command, and the external loop controller optimizes the trajectory planning command to control the flight of the UAV, so as to adjust the flight trajectory of the UAV in real time.

[0093] In this embodiment of the invention, the UAV is controlled to fly according to the flight path plan through a dual-loop structure of an outer loop controller and an outer loop sensor, and an inner loop controller and an inner loop sensor. This effectively stabilizes the UAV's flight attitude and corrects the flight trajectory in real time, thereby achieving dynamic tracking of the landing platform.

[0094] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Additionally, the terms "front," "back," "left," "right," "upper," and "lower" in this document refer to the placement shown in the accompanying drawings.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method based on a high-speed vision camera, characterized in that, include: Target images are acquired using a high-speed vision camera mounted on a drone; The target image is subjected to grayscale conversion and noise reduction to obtain a preprocessed target image; The threshold is dynamically adjusted based on an improved adaptive thresholding algorithm to eliminate shadow interference in the preprocessed target image and obtain an optimized binary target image. Based on the optimized binary target image, contour detection and polygon approximation are performed to extract target feature information, and the matching degree between the target feature information and the preset visual identifier is detected to lock the landing platform of the UAV.

2. The target detection method based on a high-speed vision camera according to claim 1, characterized in that, The improved adaptive threshold algorithm is expressed as follows: T(x,y)=u(x,y)·(1+k·σ(x,y)); Where T(x,y) represents the adaptive threshold at pixel (x,y) in the image, u(x,y) represents the mean gray level within a set window centered at pixel (x,y), σ(x,y) represents the standard deviation of gray level within a set window centered at pixel (x,y), and k represents the dynamic adjustment coefficient.

3. The target detection method based on a high-speed vision camera according to claim 1, characterized in that, The preset visual logo is designed based on the characteristics of a double square.

4. The target detection method based on a high-speed vision camera according to claim 1, characterized in that, The preset visual identifier is based on a double square structure and arrow graphic design, including: a first rectangular frame, a second rectangular frame located in the middle of the second rectangular frame, a first arrow graphic located in the first rectangular frame, and a second arrow graphic located in the middle of the second rectangular frame.

5. The target detection method based on a high-speed vision camera according to claim 1, characterized in that, The preset visual identifier is based on a double square structure and an H graphic design, including: a first rectangular frame, a second rectangular frame located in the middle of the second rectangular frame, and an H graphic located in the middle of the second rectangular frame.

6. The target detection method based on a high-speed vision camera according to claim 1, characterized in that, The method further includes: Calculate the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data, and roll angle data of the UAV relative to the landing platform; The drone's speed and acceleration are collected in real time by using an inertial measurement unit mounted on the drone. Based on the relative position prediction model, the relative pose information between the UAV and the landing platform is updated in real time using an interpolation algorithm, based on the horizontal offset data, vertical altitude data, yaw angle data, pitch angle data, and roll angle data of the UAV relative to the landing platform, as well as the speed and acceleration of the UAV. After receiving the landing command, the drone plans its trajectory based on the relative pose information and controls the drone to land on the landing platform in stages.

Images