Unmanned aerial vehicle flight control method based on target identification

Abstract

The invention discloses an unmanned aerial vehicle flight control method based on target recognition. The method comprises the following steps: calling a plurality of cameras by using multiple processes to collect original images; first contour detection is carried out on the collected images, whether first contour information is extracted or not is judged, and if the first contour information is extracted, the area of a first contour and the length-width ratio of a first enclosing rectangle are calculated; if the target is not extracted, preliminarily judging that the target is not identified; judging whether the first contour area and the length-width ratio of the first external rectangle meet a first preset condition or not, and if the first contour area and the length-width ratio of the first external rectangle meet the first preset condition, outputting an identification result; if the first preset condition is not met, judging that the target is not recognized; and selecting an identification result according to the identification result switching condition to control the flight of the unmanned aerial vehicle. According to the invention, the limitation that a camera with a single focal length cannot adapt to target detection tasks under different distances is solved, and the identification distance of the target detection tasks is increased.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) flight control, and more specifically, to a UAV flight control method based on target recognition. Background Art

[0002] A thread is the smallest unit of program execution flow and the basic unit of processor scheduling and dispatching. A process is an independent unit for resource allocation and scheduling in the operating system and is the carrier for application execution.

[0003] Multithreading technology is limited to a single CPU or a single core in a multi-core CPU, failing to fully utilize the advantages of multi-core computing resources. Multi-process technology addresses these limitations of multi-threaded programming. It can launch a process on each CPU, or on each core of a multi-core CPU, and can even create appropriate threads within each process, maximizing computing resources to solve problems.

[0004] The accuracy and speed of drone aerial target search are limited by the focal length, field of view, resolution, and frame rate of the drone's detection equipment. Currently, most drone detection devices (pods) carry a high-performance visible light camera or other type of detection lens. To achieve high long-range recognition and detection performance, the camera must have automatic or manual zoom capabilities, which increases equipment cost and requires highly stable and responsive automatic zoom algorithms, making them unsuitable for target detection tasks at varying distances. Summary of the Invention

[0005] The present invention provides a flight control method for an unmanned aerial vehicle (UAV) based on target recognition, so as to solve at least one of the problems existing in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a UAV flight control method based on target recognition, the method comprising:

[0008] Use multiple processes to call multiple cameras to collect original images;

[0009] Performing first contour detection on each of the multiple images collected to determine whether the first contour information is extracted. If the first contour information is extracted, calculating the first contour area and the aspect ratio of the first circumscribed rectangle; if the first contour information is not extracted, preliminarily determining that the target is not recognized;

[0010] determining whether the aspect ratio of the first contour area and the first circumscribed rectangle satisfies a first preset condition, and outputting a recognition result if the aspect ratio of the first contour area and the first circumscribed rectangle satisfies the first preset condition; and determining that the target is not recognized if the aspect ratio of the first contour area and the first circumscribed rectangle does not satisfy the first preset condition;

[0011] Select the recognition result based on the recognition result switching condition to control the drone flight.

[0012] Optionally, the method also includes performing a first preprocessing on the collected original images before performing the first contour detection on the collected multiple images respectively. The first preprocessing includes performing Gaussian filtering on the collected multiple original images, extracting preset color areas in the images under direct lighting conditions after Gaussian filtering, performing image binarization on the extracted areas, and performing corrosion and expansion on the images after image binarization in sequence.

[0013] Optionally, the method further includes determining that the target is not recognized if a preset color area in the image is not extracted under direct lighting conditions.

[0014] Optionally, the preliminary determination that the target is not identified includes

[0015] If the first contour information is not extracted, the image is adjusted to a backlight state, preset color areas in the image are extracted in the backlight state, and the extracted areas are subjected to a second preprocessing.

[0016] Optionally, the second preprocessing of the extracted region includes:

[0017] The extracted areas are subjected to image binarization processing respectively, and the binarized images are subjected to erosion and dilation processing in sequence.

[0018] Optionally, the method further includes

[0019] If the preset color area is not extracted in the backlight state, it is determined that the target is not recognized.

[0020] Optionally, the method further includes

[0021] Perform second contour detection on the images after the second preprocessing to determine whether the second contour information is extracted.

[0022] Optionally, the determination of whether the second contour information is extracted includes:

[0023] If the second contour information is extracted, the second contour area and the aspect ratio of the second circumscribed rectangle are calculated; if the second contour information is not extracted, it is determined that the target is not recognized and no result is output.

[0024] Optionally, the method further includes

[0025] Determine whether the aspect ratio of the second contour area and the second circumscribed rectangle meets the second preset condition. If the aspect ratio of the second contour area and the second circumscribed rectangle meets the second preset condition, output the recognition result; if the aspect ratio of the second contour area and the second circumscribed rectangle does not meet the second preset condition, it is determined that the target is not recognized and no result is output.

[0026] Optionally, the recognition result switching condition includes:

[0027] Compare the distance between the drone and the target with the preset distance and select the corresponding recognition result.

[0028] The beneficial effects of the present invention are as follows:

[0029] The present invention adopts a method of multi-process calling multiple cameras to identify targets and extract target information, switches flags according to the long and short focal lengths of the cameras, controls the flight control of the UAV, solves the limitation that cameras with a single focal length cannot adapt to target detection tasks at different distances, and improves the recognition distance of target detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0031] Figure 1 A flow chart showing a target recognition algorithm of the present invention is shown;

[0032] Figure 2 A flow chart showing two processes of the present invention calling cameras with different focal lengths;

[0033] Figure 3 A logic diagram showing multiple processes calling multiple cameras to share data in the present invention is shown. DETAILED DESCRIPTION

[0034] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the following detailed description is illustrative rather than restrictive and should not be used to limit the scope of protection of the present invention.

[0035] An embodiment of the present invention provides a method for controlling a drone flight based on target recognition, the method comprising:

[0036] Use multiple processes to call multiple cameras to collect original images;

[0037] A first preprocessing is performed on the multiple collected original images. The first preprocessing includes performing Gaussian filtering on the images respectively, extracting preset color areas from the filtered images under direct lighting conditions, binarizing the images to which the preset color areas have been extracted, and sequentially performing erosion and dilation on the binarized images. If the preset color areas are not extracted under direct lighting conditions, it is determined that the target is not recognized.

[0038] Performing a first contour detection on the image after the first preprocessing to determine whether the first contour information is extracted, and if so, calculating the first contour area and the aspect ratio of the first circumscribed rectangle; if not, preliminarily determining that the target is not recognized;

[0039] Determine whether the aspect ratio of the first contour area to the first circumscribed rectangle meets a first preset condition. If so, output a recognition result. If not, determine that the object was not recognized and do not output a result. The first preset condition is that the first contour area is greater than 20 and the aspect ratio of the first circumscribed rectangle is between 0.8 and 1.2.

[0040] If the first contour information is not extracted, the process switches to backlighting to extract the preset color area in the image, and performs a second preprocessing on the area extracted with the preset color, wherein the second preprocessing is specifically: performing image binarization processing on the area extracted with the preset color, and performing corrosion and expansion processing on the image after the image binarization processing in sequence; if the preset color area is not extracted in the backlighting case, it is determined that the target is not recognized.

[0041] Perform a second contour detection on the second preprocessed image to determine whether the second contour information is extracted: if the second contour information is extracted, calculate the second contour area and the aspect ratio of the second circumscribed rectangle; if the second contour information is not extracted, determine that the target is not recognized and no output result is given.

[0042] Determine whether the second contour area and the aspect ratio of the second circumscribed rectangle meet a second preset condition. If so, output a recognition result. If not, determine that the object was not recognized and do not output a result. The second preset condition is that the second contour area is greater than 20 and the aspect ratio of the second circumscribed rectangle is between 0.8 and 1.2.

[0043] The required recognition result is selected according to the recognition result switching condition to control the flight of the drone; wherein the recognition result switching condition is: comparing the distance between the drone and the target with the preset distance and selecting the corresponding recognition result.

[0044] The method provided in this embodiment, particularly for the problem of identifying fixed targets, utilizes multi-process calls to multiple visible light cameras for target recognition and information extraction. By switching the corresponding recognition results based on the distance between the camera and the target under specific conditions, it overcomes the limitation of a single focal length camera that cannot adapt to target detection tasks at different distances, thereby improving the recognition distance of the target detection task.

[0045] In a specific example, Figure 1 This is the flowchart of the target recognition algorithm in this example. Two processes are used to call two cameras with different focal lengths, one long and one short, for target recognition and information extraction. The distance between the drone and the target is obtained based on GPS positioning, compared with a preset distance, and the corresponding recognition result is selected to control the drone's flight. The cameras are the drone's onboard cameras. Figure 2 The flowchart for two processes calling two cameras respectively. In this embodiment, process 1 corresponds to the telephoto camera and process 2 corresponds to the short-focus camera. The target recognition algorithm condition coefficients are adjusted according to the internal and external parameters of the cameras to obtain the corresponding target recognition results. During the process of target recognition and information extraction, the telephoto camera and the short-focus camera respectively collect the original image of the specific target. Based on the image characteristics of the specific target, the digital image processing algorithm is used to extract the color and morphological characteristics of the target and lock the pixel coordinate position and posture of the target. The specific process is as follows: Figure 1 shown.

[0046] Taking the acquisition of original images by a telephoto camera as an example, the specific method for target recognition of the acquired original images is as follows:

[0047] The collected original image is subjected to a first preprocessing, which includes filtering the original image by a Gaussian filtering method. After filtering, a preset color area in the image is extracted under direct lighting conditions. In this embodiment, the preset color area is a red area. If a red area in the image is extracted, the extracted red area is subjected to an image binarization process to facilitate the extraction of image information; the binarized image is subjected to erosion and dilation processes in sequence; if no red area is extracted, it is preliminarily determined that the target is not recognized.

[0048] Perform first contour detection on the first preprocessed image to determine whether the first contour information is extracted: If the first contour information is extracted, calculate the first contour area and the aspect ratio of the first circumscribed rectangle; determine whether the first contour area is greater than 20 and whether the aspect ratio of the first circumscribed rectangle is between 0.8 and 1.2: If the first contour area is greater than 20 and the aspect ratio of the first circumscribed rectangle is between 0.8 and 1.2, output the recognized target position information and posture information; If the first contour area is not greater than 20 or the aspect ratio of the first circumscribed rectangle is not between 0.8 and 1.2, it is determined that the target is not recognized and no recognition information is output. It should be noted that the first contour area must be greater than 20 and the aspect ratio of the first circumscribed rectangle must be between 0.8 and 1.2 to obtain the target's recognition information; if one or both conditions are not met, the target cannot be recognized and no recognition information is output.

[0049] If the first contour information is not extracted, the image is adjusted to a backlight condition and the red area is extracted again.

[0050] If a red area in the image is extracted under backlight conditions, a second preprocessing is performed on the extracted red area. The second preprocessing is to perform image binarization on the extracted red area to extract image information, and then perform erosion and dilation on the binarized image in sequence. If no red area is extracted under backlight conditions, it means that the target is not recognized.

[0051] Perform a second contour detection on the second preprocessed image to determine whether the second contour information is extracted under backlight conditions: If the second contour information is extracted, calculate the second contour area and the aspect ratio of the second circumscribed rectangle; determine whether the second contour area is greater than 20 and whether the aspect ratio of the second circumscribed rectangle is between 0.8 and 1.2: If the second contour area is greater than 20 and the aspect ratio of the second circumscribed rectangle is between 0.8 and 1.2, output the target position information and posture information; if the second contour area is not greater than 20 or the aspect ratio of the second circumscribed rectangle is not between 0.8 and 1.2, it is determined that the target is not recognized and no recognition information is output. It should be noted that the second contour area must be greater than 20 and the aspect ratio of the second circumscribed rectangle must be between 0.8 and 1.2 to obtain the target's recognition information; if one or both conditions are not met, the target cannot be recognized and no recognition information is output.

[0052] If the second outline information is not extracted, it is determined that the target is not recognized and no recognition information is output.

[0053] The recognition process for a short-focus camera is the same as that for a long-focus camera, with only some parameters in the algorithm being different.

[0054] The specific method for selecting the corresponding recognition result based on the recognition result switching condition is as follows: based on the GPS location information, the distance between the drone and the target is compared with the preset distance and the corresponding recognition result is selected. In this embodiment, the preset distance is 200 meters. If the distance between the drone and the target is greater than 200 meters, the recognition result output by the long-focus camera is selected; if the distance between the drone and the target is less than 200 meters, the recognition result output by the short-focus camera is selected.

[0055] In this embodiment, the recognition results of the telephoto camera and the short-focus camera are obtained respectively according to the target recognition algorithm, and the distance between the drone and the target is compared with the preset distance as the switching condition of the recognition result. The corresponding recognition result is selected as the input of the drone flight control to control the flight of the drone.

[0056] It should also be noted that if multiple processes are used to call multiple cameras with different focal lengths, the original images are collected by cameras with different focal lengths, and ... Figure 1 The target recognition algorithm in the target recognition algorithm is used to identify the target and obtain multiple recognition results; the distance between the drone's onboard camera and the target is obtained according to the GPS position information, and compared with the preset distance. Based on the comparison result, the corresponding recognition result is selected as the input information of the drone flight control to control the flight of the drone.

[0057] Due to hardware limitations, the detection distance of the recognition algorithm is certain. However, by using cameras with different focal lengths, increasing the number of cameras, and adopting multi-process programming, the recognition distance can be expanded.

[0058] The present invention adopts a multi-process method to call long-focus and short-focus visible light cameras. By switching the position information of the long-focus and short-focus cameras under specific conditions, it overcomes the limitation that a single focal length camera cannot adapt to target detection tasks at different distances, and improves the recognition distance of the target detection task.

[0059] Figure 3 The logic diagram for multiple processes to call multiple cameras for data sharing. Data sharing between multiple processes is the main content of multi-process programming. Figure 3 Different processes are used to process different camera data streams for information collection.

[0060] In this embodiment, two processes are connected to two cameras with different focal lengths, sending dual-camera data. If one of the cameras is detected to be disconnected, only single-camera data is sent. If the other camera is also disconnected, no camera data is sent. The connection status of the cameras is restored in sequence through fault diagnosis and fault reprocessing methods. First, the connection status of one camera is restored, and single-camera data is sent. Then, the connection status of the other camera is restored, thereby sending dual-camera data. Through fault diagnosis and fault reprocessing methods, the camera hardware connection is disconnected and reconnected. Influenced by the connection status between the detection hardware device and the processing board, a multi-process call dual-camera data sharing logic flow is designed at the software level, ensuring that the program's running status is not affected by the hardware connection status.

[0061] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0062] It should also be noted that, in the description of the present invention, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0063] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A UAV flight control method based on target recognition, characterized in that: The method includes utilizing multiple processes to call multiple cameras to collect original images; Performing first contour detection on each of the multiple images collected to determine whether the first contour information is extracted. If the first contour information is extracted, calculating the first contour area and the aspect ratio of the first circumscribed rectangle; if the first contour information is not extracted, preliminarily determining that the target is not recognized; determining whether the aspect ratio of the first contour area and the first circumscribed rectangle meets a first preset condition, and outputting a recognition result if the aspect ratio of the first contour area and the first circumscribed rectangle meets the first preset condition; If the aspect ratio of the first contour area and the first circumscribed rectangle does not meet the first preset condition, it is determined that the target is not recognized; Select the recognition result based on the recognition result switching condition to control the drone flight.

2. The method according to claim 1, characterized in that The method further includes Before performing the first contour detection on the multiple collected images, the collected original images are respectively subjected to the first preprocessing. The first preprocessing includes performing Gaussian filtering on the multiple collected original images, extracting preset color areas in the images under direct lighting conditions after Gaussian filtering, performing image binarization processing on the extracted areas, and performing erosion and dilation processing on the images after the image binarization processing in sequence.

3. The method according to claim 2, characterized in that The method further includes If the preset color area in the image is not extracted under direct lighting conditions, it is determined that the target is not recognized.

4. The method according to claim 1, wherein The initial determination that the target is not identified includes If the first contour information is not extracted, the image is adjusted to a backlight state, preset color areas in the image are extracted in the backlight state, and the extracted areas are subjected to a second preprocessing.

5. The method according to claim 4, characterized in that The second preprocessing of the extracted area includes: The extracted areas are subjected to image binarization processing respectively, and the binarized images are subjected to erosion and dilation processing in sequence.

6. The method according to claim 4, characterized in that The method further includes If the preset color area is not extracted in the backlight state, it is determined that the target is not recognized.

7. The method according to claim 4, characterized in that The method further includes Perform second contour detection on the images after the second preprocessing to determine whether the second contour information is extracted.

8. The method according to claim 7, characterized in that The step of determining whether the second contour information is extracted includes: If the second contour information is extracted, the second contour area and the aspect ratio of the second circumscribed rectangle are calculated; if the second contour information is not extracted, it is determined that the target is not recognized and no result is output.

9. The method according to claim 7, characterized in that The method further includes determining whether the aspect ratio of the second contour area and the second circumscribed rectangle meets a second preset condition, and outputting a recognition result if the aspect ratio of the second contour area and the second circumscribed rectangle meets the second preset condition; If the second contour area and the aspect ratio of the second circumscribed rectangle do not meet the second preset condition, it is determined that the target is not recognized and no result is output.

10. The method according to claim 1, characterized in that The recognition result switching conditions include Compare the distance between the drone and the target with the preset distance and select the corresponding recognition result.

Images