Ornithopter target detection method and system based on long-focus and short-focus cameras

Abstract

The invention provides a flapping-wing aircraft target detection method and system based on a long-focus camera and a short-focus camera. The method comprises the following steps: acquiring a long-focus video stream and a short-focus video stream through a two-axis binocular holder; acquiring a detection target in the long-focus and short-focus image by adopting a target detection algorithm; performing correlation matching on the detection targets in the long and short focus images by adopting a matching algorithm; target information fusion is carried out on the targets which are successfully associated and matched; performing target data fusion on the fused target and the target which is not successfully associated and matched, and integrating the target data into a short-focus image; performing trajectory prediction and tracking on the target by adopting a target tracking algorithm; and selecting a tracking target to control a pan-tilt motor to realize target tracking. The advantages of large visual field of the short-focus camera and good small target detection effect of the long-focus camera are considered, the detection distance and the detection tracking performance of the ornithopter can be effectively improved, and the actual use requirements of the ornithopter are fully met.

Description

Technical Field

[0001] The present invention relates to the technical field of flapping-wing aircraft, and in particular to a target detection method and system for a flapping-wing aircraft based on long-focus and short-focus cameras. Background Art

[0002] In recent years, unmanned aircraft can be mainly divided into three types: fixed-wing aircraft, rotary-wing aircraft, and flapping-wing aircraft. Among them, the flapping-wing aircraft is a new type of unmanned aircraft designed by imitating the body structure and flight mode of birds or insects. Here, the flapping-wing aircraft imitating birds is mainly referred to. It is different from the flight modes of traditional fixed-wing aircraft and rotary-wing aircraft. The flapping-wing aircraft uses a pair of wings to provide lift and thrust for flight through regular flapping, and adjusts the attitude of the tail wing in a way similar to that of birds to change the flight height and direction. Due to its bionic, low-noise, and high-flexibility characteristics, the flapping-wing aircraft has strong concealment and safety, and has great development potential in the field of military reconnaissance, including but not limited to camouflage reconnaissance, intelligence collection, and environmental survey.

[0003] As the "eyes" of unmanned aircraft, cameras are an indispensable part of realizing the reconnaissance function. The combination of cameras and computer vision technology enables unmanned aircraft to have the capabilities of target detection and target tracking, and has better application effects compared with unmanned aircraft that can only transmit images. In actual application scenarios, the smaller the focal length of the camera, the wider the field of view, and the larger the scene that can be photographed. However, relatively, the imaging of distant targets is smaller, and the target detection difficulty is higher; the larger the focal length of the camera, the clearer the imaging of distant targets, and the lower the target detection difficulty. However, the long-focus camera has a narrow field of view and a small imaging range, and it is impossible to photograph the reconnaissance environment well in actual use. Therefore, it is difficult for a single camera to simultaneously take into account a large field of view and good detection effects for small distant targets. Integrating the image information of long-focus and short-focus cameras can simultaneously take into account the advantages of both and make up for the deficiencies of a single camera.

[0004] Due to its own structural characteristics and unique flight mode, the flapping-wing aircraft has certain advantages, but at the same time, it also brings certain difficulties to the design of the vision system. In the case of the same volume, the flapping-wing aircraft is lighter than the traditional rotary-wing aircraft, but relatively, the load of the flapping-wing aircraft is smaller, and the load capacity is poor. The load that can be used to carry the vision system part is limited, and it is impossible to carry large-weight sensors or mechanical structures. At present, mature drone gimbals on the market (such as DJI Zenmuse H20, with a weight of 678 g), or cameras with high clarity, all have more or less problems of being too heavy and too large in volume, and their structures and functions are not suitable for flapping-wing aircraft. Therefore, it is necessary to conduct targeted gimbal design and research according to the requirements of flapping-wing aircraft.

[0005] At present, a large number of scholars have conducted research on both target detection algorithms and target tracking algorithms, but most of these algorithms are currently applied to ground imaging equipment, or some rotor and fixed-wing aircraft. The research on target detection and tracking algorithms for flapping-wing aircraft is still in its infancy. However, due to the high flight speed, high flapping frequency, and inability to hover, flapping-wing aircraft cause large changes in image height and complex image jitter during flight, resulting in scale changes, rotation changes, and blurring of the captured images. At the same time, due to the small load, flapping-wing aircraft have certain weight restrictions on cameras, gimbals, motors, and processing equipment, which will have a certain impact on the performance of the equipment. The above problems make the existing detection and tracking algorithms not very effective when used on flapping-wing aircraft, and it is necessary to design algorithms and detection models in a targeted manner. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a flapping-wing aircraft target detection method and system based on long-focus and short-focus cameras. By designing a two-axis binocular gimbal adapted to flapping-wing aircraft and carrying long-focus and short-focus cameras, the advantages of the short-focus camera's large field of view and the long-focus camera's good small target detection effect are taken into account, and a long-focus and short-focus fusion target detection and tracking algorithm is designed, which effectively improves the detection distance and detection and tracking performance of the flapping-wing aircraft, and fully meets the actual use needs of the flapping-wing aircraft.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] On the one hand, a method for target detection of a flapping-wing aircraft based on a long- and short-focus camera is provided, wherein the method is implemented by a system including a flapping-wing aircraft, a two-axis binocular gimbal, a gimbal processing module, a ground processing terminal, and a communication module;

[0009] The method comprises the following steps:

[0010] S1. Obtaining long- and short-focus video streams of a flapping-wing aircraft during flight through a two-axis binocular gimbal, and transmitting the long- and short-focus video streams to a ground processing terminal for processing through a communication module;

[0011] S2. Detect the long- and short-focus video streams using a target detection algorithm to obtain detection targets in the long- and short-focus images;

[0012] S3, using a long-short focus matching algorithm to perform association matching on the detection targets in the long-short focus images, to obtain targets with successful association matching, targets in the long-focus image without successful association matching, and targets in the short-focus image without successful association matching;

[0013] S4. For the targets with successful associated matching, use the long - short focal fusion formula to calculate the position, size, and confidence of the target bounding box, obtaining the fused targets, and do not process the targets with unsuccessful associated matching;

[0014] S5. Fuse the target data of the fused targets, the targets in the long - focal - length images with unsuccessful associated matching, and the targets in the short - focal - length images with unsuccessful associated matching, and integrate all the fused target data into the short - focal - length image as the final detected image for output;

[0015] S6. Use the target tracking algorithm to process the final detected image, and perform trajectory prediction and tracking on all targets;

[0016] S7. Select the tracking targets, transmit error information to the pan - tilt processing module through the communication module according to the coordinate positions of the tracking targets, and the pan - tilt processing module generates control information based on the error information to control the rotation of the pitch motor and yaw motor of the two - axis binocular pan - tilt, so that the tracking targets move towards the center of the screen to achieve the target tracking function.

[0017] Optionally, in step S1, the two - axis binocular pan - tilt is equipped with long - and short - focal cameras, namely a long - focal camera and a short - focal camera; among them, the long - focal camera includes a long - focal digital camera for obtaining a long - focal video stream; the short - focal camera includes a short - focal digital camera for obtaining a short - focal video stream; the ground processing terminal is paired with the communication module to receive, process, and display the long - and short - focal video streams.

[0018] Optionally, in step S2, use the target detection algorithm to independently detect the long - focal video stream and the short - focal video stream in the long - and short - focal video streams respectively. The two can use the same or different target detection algorithms, but the relevant parameters of the detected targets finally output after the two are processed are kept consistent to provide a standardized data basis for subsequent operations.

[0019] Optionally, step S3 specifically includes:

[0020] S31. Calculate the scaling ratio K between the long - and short - focal images. The calculation formula is as shown in (1):

[0021]

[0022] where f L is the focal length of the long - focal camera, and f S is the focal length of the short - focal camera;

[0023] S32. Reduce the image collected by the long - focal camera according to the scaling ratio K. The reduced image is denoted as G L , and the image size is a×b; use G L (x, y) to represent the image GL The pixel value at coordinates (x, y); denote the short - focus image as G S , with the image size being A×B; use to represent the sub - block in the short - focus image that has the same size as the down - scaled long - focus image, and the coordinates of the upper - left corner point of the sub - block are (m, n);

[0024] S33. Use the NCC (Normalized Cross - Correlation) algorithm to calculate the correlation coefficients between the down - scaled long - focus image G L and all sub - blocks with the same size in the short - focus image G S . The calculation formula is as shown in (2):

[0025]

[0026] In the formula, ρ(m, n) represents the correlation coefficient between G L and . The closer the value of ρ(m, n) is to 1, the higher the matching degree between G L and . In the formula, represents the pixel value of the sub - block at coordinates (x, y), represents the average value of the pixel values at all positions in the sub - block , represents the average value of the pixel values at all positions in the image G L ;

[0027] Traverse the entire short - focus image, calculate and obtain the sub - block in the long - focus image G L with the highest matching degree with the short - focus image G S . This sub - block is the best - matching sub - block, and obtain the coordinates (m * , n * ) of the upper - left corner point of the corresponding sub - block;

[0028] S34. Calculate the coordinates of the center point of the best - matching sub - block according to the coordinates (m * , n * ) of the upper - left corner point of the best - matching sub - block. The calculation formula is as shown in (3):

[0029]

[0030] Map the detection target on the long - focus image to the short - focus image with this center point as the standard. The mapping formula is as shown in (4):

[0031]

[0032] where represents the coordinates of the center point of the long - focus image,​​ represents the center point coordinates of the i-th detection target, represents the height and width of the bounding box of the i-th detection target; K is the scaling ratio of the long and short focal length images; the left side of the equation is the coordinate information and bounding box information of the i-th detection target on the long focal length image after being mapped to the short focal length image;

[0033] S35. Calculate and obtain the deviation matrix between the detection targets in the long and short focal length images; the detection targets in the short focal length image outside the optimal matching sub-block are not considered;

[0034] Use the target association matching algorithm to calculate the deviation value between the detection targets in the short focal length image and the detection targets in the long focal length image. The calculation formula is as shown in (5):

[0035] e ij = 1 - DIOU(i, j) + Class(i, j) (5)

[0036] where e ij represents the deviation value between the i-th detection target in the short focal length image and the j-th detection target in the long focal length image. The smaller the deviation value, the more similar the two detection targets are and the stronger the correlation;

[0037] DIoU represents the similarity between the bounding box of the i-th detection target in the short focal length image and the bounding box of the j-th detection target in the long focal length image; Class(i, j) represents the detection category relationship between the i-th detection target in the short focal length image and the j-th detection target in the long focal length image. If the categories are the same, it is 0; if the categories are different, it is 1;

[0038] By calculating all the deviation values between the detection targets in the long focal length image and the selectable detection targets in the short focal length image, construct the deviation matrix E, and e ij is an element in the deviation matrix;

[0039] S36. Set the deviation threshold e θ , and based on the deviation matrix E, perform association matching on the detection targets in the long and short focal length images;

[0040] Since e ij represents the deviation value between the i-th detection target in the short focal length image and the j-th detection target in the long focal length image, so when e ij ≥ e θ , it means that the deviation value is too large, and no matching is performed between the two targets. In the deviation matrix E, change e ij to e θ, indicating that this element is invalid and does not participate in the matching; when e ij <e θ , retain the e ij value for subsequent matching;

[0041] The deviation matrix after the above steps is denoted as E * , select the e * with the smallest value in the matrix E ij for matching, indicating that the i-th detection target in the short-focus image and the j-th detection target in the long-focus image are successfully matched, denoted as a pair of associated and successfully matched targets; then change all elements in the i-th row and the j-th column of the matrix E * to e θ , indicating that these two targets do not participate in subsequent matching; continuously select the smallest e ij in the matrix for matching according to the above steps until the smallest e ij = e θ when the matching stops;

[0042] S37. After completing the steps in S36, classify the targets in the short- and long-focus images into three categories: associated and successfully matched targets, targets in the long-focus image that are not associated and successfully matched, and targets in the short-focus image that are not associated and successfully matched.

[0043] Optionally, in step S4, the short- and long-focus fusion formula is as shown in Equation (6):

[0044]

[0045] where (x s , y s , w s , h s ) respectively represent the center coordinates, width, and height of the target box of the target to be matched in the short-focus image; (x l , y l , w l , h l ) respectively represent the center coordinates, width, and height of the target box of the target to be matched in the long-focus image; (c s , c l ) respectively represent the confidence levels of the targets to be matched in the short- and long-focus images, and (x, y, w, h) respectively represent the center coordinates, width, and height of the target box of the fused target.

[0046] Optionally, in step S5, for the targets in the long-focus image that are not successfully matched, according to the short- and long-focus image mapping relationship, use the mapping formula in Equation (4) to map the target coordinate information to the short-focus image.

[0047] Optionally, in step S6, the target tracking algorithm adopts the DeepSort tracking algorithm to predict and track the trajectories of all targets.

[0048] Optionally, in step S7, after selecting the tracking target, the coordinates of the center point of the target box of the tracking target are (O x , O y ), and the coordinates of the center point of the short-focus image are (O sx , O sy ). This point is the expected coordinate of the tracking target, that is, to keep the tracking target in the center of the screen;

[0049] The error between the two is calculated using Equation (7):

[0050]

[0051] The error diff is fed back to the controller of the pan-tilt processing module. The controller adopts an incremental PID controller to generate control quantities for the pitch motor and yaw motor of the two-axis binocular pan-tilt according to the magnitude of the error, and controls the rotation of the pitch motor and yaw motor to move the target to the center of the screen to achieve target tracking.

[0052] On the other hand, a target detection system for a flapping-wing aircraft based on long and short focus cameras is provided to implement the method described in any one of the above. The system includes a flapping-wing aircraft, a two-axis binocular pan-tilt, a pan-tilt processing module, a ground processing terminal, and a communication module;

[0053] The two-axis binocular pan-tilt is equipped with long and short focus cameras for obtaining long and short focus video streams during the flight of the flapping-wing aircraft;

[0054] The ground processing terminal is equipped with a long and short focus fusion target detection algorithm module and a target tracking algorithm module for processing the long and short focus video streams;

[0055] Among them, the long and short focus fusion target detection algorithm module is used to process the long and short focus video stream using a target detection algorithm to obtain detection targets in the long and short focus images; use a long and short focus matching algorithm to perform correlation matching on the detection targets in the long and short focus images to obtain successfully correlated targets, targets in the long focus image that have not been successfully correlated, and targets in the short focus image that have not been successfully correlated; calculate the position, size, and confidence of the target box for the successfully correlated targets using the long and short focus fusion formula to obtain the fused targets, and do not process the targets that have not been successfully correlated; perform target data fusion on the fused targets, the targets in the long focus image that have not been successfully correlated, and the targets in the short focus image that have not been successfully correlated, and integrate all the fused target data into the short focus image as the final detection image output;

[0056] The target tracking algorithm module is used to process the final detected image by using a target tracking algorithm, and predict and track the trajectories of all targets.

[0057] The ground processing terminal is also used to select a tracking target, transmit error information to the pan-tilt processing module through the communication module according to the coordinate position of the tracking target, and the pan-tilt processing module generates a motor control quantity according to the error information to control the pitch motor and yaw motor of the two-axis binocular pan-tilt to rotate, so that the tracking target moves to the center of the screen, realizing the target tracking function.

[0058] The communication module is used for information transmission between the pan-tilt processing module and the ground processing terminal, including transmitting long- and short-focus video streams and error information calculated when tracking a target.

[0059] On the other hand, an electronic device is provided, and the electronic device includes:

[0060] A processor;

[0061] A memory, on which computer-readable instructions are stored, and when the computer-readable instructions are loaded and executed by the processor, the steps of the above-mentioned target detection method for a flapping-wing aircraft based on long- and short-focus cameras are implemented.

[0062] On the other hand, a computer-readable storage medium is provided, in which program code is stored, and the program code can be called by a processor to execute the steps of the above-mentioned target detection method for a flapping-wing aircraft based on long- and short-focus cameras.

[0063] The beneficial effects brought by the technical solution provided by the present invention at least include:

[0064] The present invention provides a method and system for target detection of a flapping-wing aircraft based on long and short focal length cameras. Through a two-axis binocular pan-tilt, the long and short focal length cameras are carried to capture long and short focal length video streams during the flight of the flapping-wing aircraft, ensuring that the captured video streams contain both a short focal length video stream with a large field of view and a long focal length video stream with a clearer central area. At the same time, a long and short focal length fusion target detection algorithm is adopted to construct a mapping relationship between the long focal length camera image and the short focal length camera image. Then, the detection targets in the long focal length camera image are associated and matched with the detection targets in the short focal length camera image, and the successfully matched targets are fused with target information. Finally, all target data are integrated into the short focal length image, so that the output detection image contains both the large field of view of the short focal length image and the effective detection of small targets at a long distance by the long focal length image. At the same time, for the targets detected by both the long and short focal lengths, information fusion is performed, which can improve the detection accuracy of the targets and ultimately increase the detection distance of the flapping-wing aircraft. Subsequently, the target can be selected for tracking, and according to the center point coordinates of the target box and the center point coordinates of the image, the error between the two is calculated, and an incremental PID controller is used to control the smooth rotation of the pan-tilt motor to improve the tracking effect. The present invention is easy to implement and convenient to deploy, suitable for lightweight pan-tilts. Its algorithm is deployed on the ground end, which can fully utilize the performance of the ground processing terminal and meet the actual needs of the flapping-wing aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0066] Figure 1 is a flowchart of a method for target detection of a flapping-wing aircraft based on long and short focal length cameras provided by an embodiment of the present invention;

[0067] Figure 2 is a schematic diagram of matching mapping between a long focal length image and a short focal length image provided by an embodiment of the present invention;

[0068] Figure 3 is a schematic diagram of target association and matching of long and short focal length images provided by an embodiment of the present invention;

[0069] Figure 4 is a flowchart of a long and short focal length fusion matching algorithm provided by an embodiment of the present invention;

[0070] Figures 5a-5b is a schematic structural diagram of a two-axis binocular pan-tilt from two perspectives provided by an embodiment of the present invention;

[0071] Figure 6It is a schematic diagram of the target tracking process provided by an embodiment of the present invention. Detailed implementation manners

[0072] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0073] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of the word "example" is intended to present concepts in a specific manner.

[0074] An embodiment of the present invention provides a method for detecting a target of a flapping-wing aircraft based on a long-focus and short-focus camera. The method is implemented by a system including a flapping-wing aircraft, a two-axis binocular pan-tilt, a pan-tilt processing module, a ground processing terminal, and a communication module. As Figure 1 shown, the method includes the following steps:

[0075] S1. Obtain the long-focus and short-focus video streams during the flight of the flapping-wing aircraft through the two-axis binocular pan-tilt, and transmit the long-focus and short-focus video streams to the ground processing terminal through the communication module for processing.

[0076] The two-axis binocular pan-tilt is equipped with a long-focus camera and a short-focus camera, that is, a long-focus camera and a short-focus camera. Among them, the long-focus camera includes a long-focus high-definition small digital camera for obtaining a long-focus video stream; the short-focus camera includes a short-focus high-definition small digital camera for obtaining a short-focus video stream; the ground processing terminal is configured with the communication module to receive, process and display the long-focus and short-focus video streams.

[0077] S2. Detect the long-focus and short-focus video streams by using a target detection algorithm to obtain the detection targets in the long-focus and short-focus images.

[0078] Among them, the long-focus video stream and the short-focus video stream in the long-focus and short-focus video streams are independently detected by using a target detection algorithm. The two can use the same or different target detection algorithms, but the relevant parameters of the detection targets finally output after the two are processed are kept consistent, providing a standardized data basis for subsequent operations.

[0079] Specifically, the target detection algorithm can adopt the YOLOVX series of algorithms as the main target detection algorithm. At the same time, since the flapping-wing aircraft cannot hover, there is a high requirement for the real-time performance of the algorithm. Therefore, it is necessary to prune and lightweight the detection model to meet the actual use requirements of the flapping-wing aircraft. The models and algorithms used for the long-focus and short-focus video streams are allowed to be different. The flapping-wing aerial photography dataset can be captured by two different cameras respectively, and targeted model training can be carried out to improve the detection effect. However, it should be noted that the relevant parameters of the detection targets finally output after the two are processed need to be consistent for subsequent operation and processing.

[0080] S3. Use the long-focus and short-focus matching algorithm to perform correlation matching on the detection targets in the long-focus and short-focus images, and obtain the targets with successful correlation matching, the targets in the long-focus images that have not been successfully correlated, and the targets in the short-focus images that have not been successfully correlated.

[0081] First, it is necessary to perform matching processing on the long-focus and short-focus images. As Figure 2 shown, the main purpose is to find the best mapping position of the long-focus image in the short-focus image. The specific steps include:

[0082] S31. Calculate the scaling ratio K between the long-focus and short-focus images. The calculation formula is as shown in (1):

[0083]

[0084] where f L is the focal length of the long-focus camera, and f S is the focal length of the short-focus camera.

[0085] S32. Reduce the image collected by the long-focus camera according to the scaling ratio K. The reduced image is denoted as G L , and the image size is a×b; use G L (x,y) to represent the pixel value of the image G L at the coordinate (x,y); denote the short-focus image as G S , and the image size is A×B; use to represent the sub-block in the short-focus image with the same size as the reduced long-focus image, and the coordinates of the upper left corner point of the sub-block are (m,n).

[0086] S33. Use the NCC normalized cross-correlation algorithm to calculate the correlation coefficients of all sub-blocks L with the same size in the reduced long-focus image G S and the short-focus image G . The calculation formula is as shown in (2):

[0087]

[0088] In the formula, ρ(m,n) represents G LThe correlation coefficient with , the closer the value of ρ(m,n) is to 1, the higher the matching degree between G L and ; where represents the pixel value of the sub-block at the coordinate (x,y), represents the sub-block the average value of the pixel values at all positions in represents the average value of the pixel values at all positions in the image G L .

[0089] Traverse the entire short-focus image, calculate and obtain the long-focus image G L and the sub-block with the highest matching degree in the short-focus image G S This sub-block is the best matching sub-block, and the coordinates (m of the upper left corner point of the corresponding sub-block are obtained * ,n * ).

[0090] S34. Calculate the center point coordinates of the best matching sub-block according to the coordinates (m * ,n * ) of the upper left corner point of the best matching sub-block. The calculation formula is as shown in (3): The calculation formula is as shown in (3):

[0091]

[0092] Map the detection target on the long-focus image to the short-focus image with this center point as the standard, and the mapping formula is as shown in (4):

[0093]

[0094] Where represents the center point coordinates of the long-focus image, represents the center point coordinates of the i-th detection target, represents the height and width of the target box of the i-th detection target; K is the scaling ratio of the long and short focus images; the on the left side of the equation is the coordinate information and target box information of the i-th detection target on the long-focus image mapped to the short-focus image.

[0095] Furthermore, as Figure 3 shown, perform correlation matching on the targets in the long and short focus images, mainly including detection target 1 in the long-focus image, detection target 2 in the short-focus image, and the best matching mapping area 3.

[0096] S35. Calculate and obtain the deviation matrix between the detection targets in the long and short focus images. The detection target in the short-focus image is in the best matching sub-block Those outside are not considered because the long - focal - length camera's field of view cannot capture the areas where these detection targets are located.

[0097] The target - association matching algorithm is used to calculate the deviation value between the detection target in the short - focal - length image and the detection target in the long - focal - length image. The calculation formula is as shown in (5):

[0098] e ij = 1 - DIOU(i,j)+Class(i,j) (5)

[0099] where e ij represents the deviation value between the i - th detection target in the short - focal - length image and the j - th detection target in the long - focal - length image. The smaller the deviation value, the more similar the two detection targets are and the stronger the correlation.

[0100] DIoU is a loss function in object detection, which is used to calculate the similarity between the predicted bounding box and the ground - truth bounding box. In the above formula, DIoU represents the similarity between the bounding box of the i - th detection target in the short - focal - length image and the bounding box of the j - th detection target in the long - focal - length image.

[0101] Class(i,j) represents the detection - class relationship between the i - th detection target in the short - focal - length image and the j - th detection target in the long - focal - length image. If the classes are the same, it is 0; if the classes are different, it is 1.

[0102] By calculating all the deviation values between the detection targets in the long - focal - length image and the selectable detection targets in the short - focal - length image, a deviation matrix E is constructed, and e ij is an element in the deviation matrix.

[0103] S36. Set the deviation threshold e θ , and based on the deviation matrix E, perform association matching on the detection targets in the long - and short - focal - length images.

[0104] Since e ij represents the deviation value between the i - th detection target in the short - focal - length image and the j - th detection target in the long - focal - length image, when e ij ≥e θ , it means that the deviation value is too large, and no matching is performed between the two targets. In the deviation matrix E, change e ij to e θ , indicating that this element is invalid and does not participate in the matching; when e ij <e θ , retain the e ij value for subsequent matching.

[0105] The deviation matrix after the above - mentioned steps is denoted as E * , in the matrix E* Select e from ij with the smallest value for matching, indicating that the i-th detected target in the short-focus image is successfully matched with the j-th detected target in the long-focus image, denoted as a pair of successfully associated and matched targets; then change all elements in the i-th row and j-th column of matrix E * to e, indicating that these two targets do not participate in subsequent matching; continuously select the smallest e in the matrix according to the above steps θ for matching until the smallest e ij equals e ij and stop the matching. θ

[0106] S37. After completing according to the step S36, the targets in the short- and long-focus images can be divided into three categories, as Figure 3 shown, the successfully associated and matched target 4, the targets 5 in the short-focus image that are not successfully associated and matched, and the targets 6 in the long-focus image that are not successfully associated and matched.

[0107] S4. Calculate the position, size, and confidence of the target box for the successfully associated and matched targets using the short- and long-focus fusion formula to obtain the fused target, and do not process the targets that are not successfully associated and matched.

[0108] In this step, the short- and long-focus fusion formula is as shown in Equation (6):

[0109]

[0110] where (x s , y s , w s , h s ) respectively represent the center coordinates, width, and height of the target box of the target to be matched in the short-focus image; (x l , y l , w l , h l ) respectively represent the center coordinates, width, and height of the target box of the target to be matched in the long-focus image; (c s , c l ) respectively represent the confidence levels of the targets to be matched in the short- and long-focus images, and (x, y, w, h) respectively represent the center coordinates, width, and height of the target box of the fused target.

[0111] S5. Perform fusion of the target data for the fused targets, the targets in the long-focus image that are not successfully associated and matched, and the targets in the short-focus image that are not successfully associated and matched, and integrate all the fused target data into the short-focus image as the final detected image output.

[0112] As Figure 4 ​As shown in the figure, it is the flowchart of the long and short focal length image fusion matching algorithm. Starting from the input of long and short focal length images, through target detection, target association and matching, the fusion of target information with successful matching, and then to the final step of target data fusion in the algorithm. Among them, for the targets in the long focal length image that are not successfully matched, according to the mapping relationship between the long and short focal length images, the mapping formula as shown in Equation (4) is used to map the target coordinate information into the short focal length image, and finally the image after successful detection is output.

[0113] S6. Use the target tracking algorithm to process the final detected image and predict and track all targets.

[0114] Among them, the target tracking algorithm mainly uses the DeepSort tracking algorithm to predict and track all targets. At the same time, due to the characteristics of the flapping-wing aircraft, the video stream captured by the camera has problems such as complex jitter and large scale changes. Therefore, the methods of uniformly accelerated Kalman filtering and DIOU association matching can be used to specifically optimize the DeepSort tracking algorithm to meet the actual needs of the flapping-wing aircraft.

[0115] S7. Select the tracking target, and transmit the error information to the pan-tilt processing module through the communication module according to the coordinate position of the tracking target. The pan-tilt processing module generates control information according to the error information and controls the pitching motor and yaw motor of the two-axis binocular pan-tilt to rotate, so that the tracking target moves to the center of the screen and realizes the target tracking function.

[0116] Among them, the process of selecting the tracking target can be carried out through the upper computer. First, during the target trajectory prediction and tracking process, the DeepSort tracking algorithm will label each target. After that, when selecting the tracking target, the function of selecting the tracking target can be realized by inputting the number command or by designing the upper computer to click on the screen with the mouse to select the nearest target as the tracking object.

[0117] Further, after selecting the tracking target, the center point coordinates of the target box of the tracking target are (O x , O y ), and the center point coordinates of the short focal length image are (O sx , O sy ). This point is the expected coordinate of the tracking target, that is, to keep the tracking target in the center of the screen.

[0118] Calculate the error between the two using Equation (7):

[0119]

[0120] The error diff is fed back to the controller of the two-axis binocular pan-tilt processing module. The controller adopts an incremental PID controller, and its output is a change amount. According to the magnitude of the error, control amounts acting on the pitch motor and yaw motor of the two-axis binocular pan-tilt are continuously generated to control the rotation of the pitch motor and yaw motor, so that the target moves towards the center of the picture, realizing target tracking.

[0121] The present invention takes into account the advantages of a short-focus camera having a large field of view and a long-focus camera having good small-target detection effect, can effectively improve the detection distance and detection and tracking performance of the flapping-wing aircraft, solves the problem of the lack of a small-target detection method for flapping-wing aircraft in practical applications, and fully meets the actual use requirements of flapping-wing aircraft.

[0122] Correspondingly, an embodiment of the present invention further provides a flapping-wing aircraft target detection system based on long and short focus cameras. The system includes a flapping-wing aircraft, a two-axis binocular pan-tilt, a pan-tilt processing module, a ground processing terminal, and a communication module.

[0123] The two-axis binocular pan-tilt is equipped with long and short focus cameras for obtaining long and short focus video streams when the flapping-wing aircraft is flying.

[0124] The ground processing terminal is equipped with a long and short focus fusion target detection algorithm module and a target tracking algorithm module for processing the long and short focus video streams.

[0125] Among them, the long and short focus fusion target detection algorithm module is used to process the long and short focus video streams by using a target detection algorithm to obtain detection targets in the long and short focus images; use a long and short focus matching algorithm to perform correlation matching on the detection targets in the long and short focus images to obtain successfully correlated and matched targets, targets in the long focus images that are not successfully correlated and matched, and targets in the short focus images that are not successfully correlated and matched; calculate the position, size, and confidence of the target box for the successfully correlated and matched targets by using a long and short focus fusion formula to obtain the fused targets, and do not process the targets that are not successfully correlated and matched; perform target data fusion on the fused targets, the targets in the long focus images that are not successfully correlated and matched, and the targets in the short focus images that are not successfully correlated and matched, and integrate all the fused target data into the short focus images as the final detection images for output.

[0126] The target tracking algorithm module is used to process the final detection images by using a target tracking algorithm to perform trajectory prediction and tracking on all targets.

[0127] The ground processing terminal is further configured to select a tracking target, and transmit error information to the pan-tilt processing module through the communication module according to the coordinate position of the tracking target. The pan-tilt processing module generates control information based on the error information, and controls the pitch motor and yaw motor of the two-axis binocular pan-tilt to rotate, so that the tracking target moves towards the center of the screen, realizing the target tracking function;

[0128] The communication module is used for information transmission between the pan-tilt processing module and the ground processing terminal, including transmitting long and short focal length video streams and error information calculated during target tracking.

[0129] The system of this embodiment can be used to execute Figure 1 the technical solutions of the method embodiments shown. The implementation principles and technical effects are similar, and will not be elaborated here.

[0130] Furthermore, the two-axis binocular pan-tilt, to meet the payload requirements of most medium and small flapping-wing aircraft, weighs only 50 g. Most of its mechanical structure adopts a hollow design. While ensuring meeting the strength requirements, the weight is reduced as much as possible. At the same time, the structure is made by 3D printing with 7500 high-performance nylon material. While being light, it also has a certain high-temperature resistance to withstand the temperature rise during the use of the motor. Its specific structure is as Figure 5a 、 Figure 5b shown. Figure 5a 、 Figure 5b are schematic structural diagrams of the two-axis binocular pan-tilt from two perspectives provided by the embodiments of the present invention.

[0131] As Figure 5a 、 Figure 5b shown, the two-axis binocular pan-tilt mainly includes: short focal length camera 1, short focal length camera protective housing 2, long focal length camera 3, long focal length camera protective housing 4, pitch motor bearing fixing part 5, intermediate connecting part 6, yaw motor 7, 5 mm hollow cylinder 8, pan-tilt fixing part 9, pitch motor 10, magnet 11, encoder 12. Among them, the two-axis binocular pan-tilt is installed on the flapping-wing aircraft through the pan-tilt fixing part 9, and the style of the pan-tilt fixing part can be changed according to different flapping-wing aircraft. The pitch motor 10 and the yaw motor 7 are the main power sources of the pan-tilt. To meet the requirements of small size and light weight, both motors are selected as small-size DC brushless motors, and the FOC vector control scheme can be adopted to control the three-phase current output of the motors for corresponding rotation. The motor model can be selected as GB1105, and a single motor is only 7.5 g, with a stator outer diameter of 11 mm and a stator height of 5 mm, and an encoder can be selected and equipped by itself.

[0132] Both the telephoto camera protective case 4 and the short - focal camera protective case 2 adopt a hollow design and have the same size. The difference is that due to the longer telephoto camera lens, in order to fix the lens, a threaded ring is designed in front of the telephoto camera protective case 4 for fixing the telephoto lens. Both the long - and short - focal cameras are small digital cameras with a weight of only 6.8 g. The camera has a wide voltage input of 5 - 40 V, which meets the working voltage range of the flapping - wing aircraft. The lens focal length can be selected according to different flight height requirements.

[0133] Furthermore, since there will be movement in the heading axis direction of the flapping - wing aircraft itself, the two - axis binocular gimbal provided in the embodiment of the present invention only uses two degrees of freedom, which are respectively responsible for pitching - direction rotation and yaw - direction rotation. The motor and the camera are bolt - connected by the pitching - motor bearing fixture 5 and the intermediate connecting piece 6. The angle information of the two motors is read by the magnet 11 and the encoder 12, and the encoder 12 is connected to the gimbal processing module. The encoder 12 uses a magnetic encoder, which records the change in angle by sensing the change in the magnetic field. The magnet 11 is connected to the motor rotor, and the rotation of the motor will drive the magnet to rotate together, thereby changing the magnetic field. The distance between the magnet 11 and the encoder 12 will affect the encoder's sensing of the magnetic - field change to obtain the rotor - position information. The ideal spacing of the magnetic encoder is usually set between 0.1 mm and 0.3 mm. Since the magnet itself has a thickness of 2.5 mm, a 5 - mm hollow cylinder 8 is used to separate the magnet 11 and the encoder 12 to ensure that the spacing is within the ideal range. It should be noted that for the encoder and the magnet of the pitching motor, within the pitching - motor bearing fixture 5, the distance is also ensured to be within the ideal spacing range by a 5 - mm hollow cylinder. The two - axis binocular gimbal obtains the current rotation angle of the motor in real - time through the magnetic encoder and transmits it to the gimbal processing module. A data connection is established between the gimbal processing module and the magnetic encoder to receive the motor - angle change in real - time, and then an incremental PID controller can be used to realize real - time control of the motor to rotate accordingly according to the requirements.

[0134] Such as Figure 6As shown in the figure, the FIGURE shows a schematic diagram of the target tracking process of a flapping-wing aircraft target detection system based on a long-short focus camera provided by the present invention. The specific process is that the two-axis cloud platform captures long-short focus video streams through the long-short focus camera. The cloud platform processing module obtains the long-short focus video streams and constructs a data transmission link with the ground processing terminal through the sky-end communication module and the ground-end communication module, and transmits the long-short focus video streams to the ground processing terminal. The ground processing terminal completes the target detection of the long-short focus images through the long-short focus fusion target detection algorithm module. Then, the ground processing terminal predicts and tracks the trajectory of the detected target in the long-short focus images through the target tracking algorithm module, selects the tracking target, calculates the error diff, and transmits the error information to the cloud platform processing module through the ground-end communication module and the sky-end communication module. The cloud platform processing module generates a motor control amount through an incremental PID controller based on the error diff, and controls the rotation of the pitch motor and the yaw motor of the cloud platform, so that the target moves to the center of the screen and the target tracking is realized.

[0135] In an exemplary embodiment, the present invention further provides an electronic device, which includes:

[0136] a processor;

[0137] a memory, on which computer-readable instructions are stored. When the computer-readable instructions are loaded and executed by the processor, the steps of the above-mentioned flapping-wing aircraft target detection method based on a long-short focus camera are implemented.

[0138] In an exemplary embodiment, the present invention further provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is loaded and executed by the processor to implement the steps of the above-mentioned flapping-wing aircraft target detection method based on a long-short focus camera. For example, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0139] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or terminal device including the element.

[0140] The description mentions "an embodiment", "embodiments", "exemplary embodiments", "some embodiments", etc., indicating that the described embodiments may include specific features, structures, or characteristics, but not necessarily every embodiment includes such specific features, structures, or characteristics. Additionally, when combining specific features, structures, or characteristics with an embodiment, implementing such features, structures, or characteristics in combination with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the relevant art.

[0141] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. Additionally, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0142] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0143] It should be understood that in various embodiments of the present invention, the magnitudes of the serial numbers of the above processes do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0144] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical, or other form.

[0145] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, in each embodiment of the present invention, each functional unit may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit.

[0147] If the described function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs and other various media that can store program codes.

[0148] The present invention covers any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the present invention. To enable the public to have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention. However, those skilled in the art can fully understand the present invention without these detailed descriptions. In addition, to avoid unnecessary confusion to the essence of the present invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0149] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A target detection method for a flapping-wing aircraft based on a long-short focus camera, characterized in that The method is implemented by a system including a flapping-wing aircraft, a two-axis binocular pan-tilt head, a pan-tilt head processing module, a ground processing terminal, and a communication module; The method includes the following steps: S1. Obtain the long-focus and short-focus video streams during the flight of the flapping-wing aircraft through the two-axis binocular pan-tilt head, and transmit the long-focus and short-focus video streams to the ground processing terminal for processing through the communication module; S2. Use an object detection algorithm to detect the long-focus and short-focus video streams, and obtain the detection targets in the long-focus and short-focus images; S3. Use a long-focus and short-focus matching algorithm to perform correlation matching on the detection targets in the long-focus and short-focus images, and obtain the targets with successful correlation matching, the targets in the long-focus images without successful correlation matching, and the targets in the short-focus images without successful correlation matching; S4. Calculate the position, size, and confidence of the target box for the targets with successful correlation matching using the long-focus and short-focus fusion formula to obtain the fused targets, and do not process the targets without successful correlation matching; S5. Perform fusion of the target data for the fused targets, the targets in the long-focus images without successful correlation matching, and the targets in the short-focus images without successful correlation matching, and integrate all the fused target data into the short-focus image as the final detection image for output; S6. Use an object tracking algorithm to process the final detection image and perform trajectory prediction and tracking on all targets; S7. Select a tracking target, transmit error information to the pan-tilt head processing module according to the coordinate position of the tracking target through the communication module, and the pan-tilt head processing module generates control information according to the error information to control the pitching motor and yaw motor of the two-axis binocular pan-tilt head to rotate, so that the tracking target moves to the center of the screen to achieve the target tracking function.

2. The method for detecting a target of a flapping-wing aircraft based on a long-short focus camera according to claim 1, wherein In step S1, the two-axis binocular pan-tilt head is equipped with a long-focus camera and a short-focus camera, that is, a long-focus camera and a short-focus camera; among them, the long-focus camera includes a long-focus digital camera for obtaining a long-focus video stream; the short-focus camera includes a short-focus digital camera for obtaining a short-focus video stream; the ground processing terminal is paired with the communication module to receive, process, and display the long-focus and short-focus video streams.

3. The target detection method for a flapping-wing aircraft based on a long-short focus camera according to claim 1, characterized in that, In step S2, an object detection algorithm is used to independently detect the long-focus video stream and the short-focus video stream in the long-focus and short-focus video streams. The same or different object detection algorithms can be used for the two, but the relevant parameters of the detection targets finally output after the two are processed are kept consistent to provide a standardized data basis for subsequent operations.

4. The method for detecting a target of a flapping-wing aircraft based on a long and short focal length camera according to claim 1, wherein Step S3 specifically includes: S31. Calculate the scaling ratio K between the long-focus and short-focus images, and the calculation formula is as shown in (1): where f L is the focal length of the long - focal - length camera, and f S is the focal length of the short - focal - length camera; S32. Scale down the image captured by the telephoto camera by a scaling ratio K, and denote the scaled-down image as G L , with the image size being a×b; use G L (x,y) to represent the pixel value of the image G L at the coordinate (x,y); denote the short-focus image as G S , with the image size being A×B; use to represent the sub-block in the short-focus image that has the same size as the scaled-down telephoto image, and the coordinates of the upper-left corner point of the sub-block are (m,n); S33. Calculate the correlation coefficient between all sub - blocks with the same size in the down - scaled long - focal image G L and the short - focal image G S using the NCC (Normalized Cross - Correlation) algorithm. The calculation formula is as shown in (2): ​ where ρ(m,n) represents G L and the correlation coefficient of, the closer the value of ρ(m,n) is to 1, the higher the matching degree of G L and ; in the formula represents the pixel value at the coordinate (x,y) of the sub-block, represents the sub-block the average value of the pixel values at all positions in, represents the average value of the pixel values at all positions in the image G L ; Traverse the entire short - focus image, calculate and obtain the long - focus image G L The sub - block with the highest matching degree with the short - focus image G S in it This sub - block is the best - matching sub - block, and obtain the coordinates (m * , n * ) of the upper - left corner point of the corresponding sub - block; S34. Calculate the center point coordinates of the best matching sub-block based on the upper left corner point coordinates (m * , n * ) of the best matching sub-block. The calculation formula is as shown in (3): ​ Map the detection targets on the long-focus image to the short-focus image with this center point as the standard, and the mapping formula is as shown in (4): Among them represents the center point coordinates of the long - focal - length image, represents the center point coordinates of the i - th detected target, represents the height and width of the target box of the i - th detected target; K is the scaling ratio of the long - and short - focal - length images; the on the left side of the equation is the coordinate information and target box information of the i - th detected target on the long - focal - length image after being mapped to the short - focal - length image; S35. Calculate and obtain the deviation matrix between the detected targets in the long and short focus images; the detected targets in the short focus image outside the optimal matching sub-block are not considered; Use an object correlation matching algorithm to calculate the deviation value between the detection targets in the short-focus image and the detection targets in the long-focus image, and the calculation formula is as shown in (5): e ij = 1 - DIOU(i, j)+Class(i, j) (5) where e ij represents the deviation value between the i-th detection target in the short-focus image and the j-th detection target in the long-focus image. The smaller the deviation value, the more similar the two detection targets are and the stronger the correlation is; DIoU represents calculating the similarity between the bounding box of the i-th detected object in the short-focus image and the bounding box of the j-th detected object in the long-focus image; Class(i, j) represents the detection category relationship between the i-th detected object in the short-focus image and the j-th detected object in the long-focus image, which is 0 if the categories are the same and 1 if the categories are different; All deviation values between the detected target in the telephoto image and the selectable detected targets in the short-focus image are calculated to construct a deviation matrix E, where e ij is an element in the deviation matrix; S36. Set the deviation threshold e θ , and perform associated matching on the detection targets in the long and short focus images according to the deviation matrix E; Since e ij represents the deviation value between the i-th detected target in the short-focus image and the j-th detected target in the long-focus image, so when e ij ≥ e θ , it indicates that the deviation value is too large, and the matching between the two targets is not performed. In the deviation matrix E, change e ij to e θ , indicating that this element is invalid and does not participate in the matching; when e ij <e θ , keep the e ij value for subsequent matching; The deviation matrix after the above steps is denoted as E * , in the matrix E * , select the one with the minimum e ij value for matching, indicating that the i-th detected target in the short-focus image and the j-th detected target in the long-focus image are successfully matched, denoted as a pair of associated and successfully matched targets; then change all elements in the i-th row and the j-th column of the matrix E * to e θ , indicating that these two targets do not participate in subsequent matching; continuously select the minimum e ij in the matrix for matching according to the above steps until the minimum e ij = e θ and stop matching S37. After completing the step S36, classify the objects in the long-short focus images into three categories: objects with successful associated matching, objects in the long-focus image without successful associated matching, and objects in the short-focus image without successful associated matching.

5. The method for detecting a target of a flapping-wing aircraft based on a long-short focus camera according to claim 1, wherein In the step S4, the long-short focus fusion formula is shown in Equation (6): where (x s , y s , w s , h s ) respectively represent the center point coordinates, width, and height of the target box of the target to be matched in the short - focal - length image; (x l , y l , w l , h l ) respectively represent the center coordinates, width, and height of the target box of the target to be matched in the long - focal - length image; (c s , c l ) respectively represent the confidence levels of the targets to be matched in the short - and long - focal - length images, and (x, y, w, h) respectively represent the center coordinates, width, and height of the target box of the fused target.

6. The method for detecting a target of a flapping-wing aircraft based on a long and short focal length camera according to claim 1, wherein In the step S5, for the objects in the long-focus image that are not successfully matched, according to the long-short focus image mapping relationship, use the mapping formula as shown in Equation (4) to map the object coordinate information to the short-focus image.

7. The method for detecting a target of a flapping-wing aircraft based on a long-short focus camera according to claim 1, wherein In the step S6, the object tracking algorithm uses the DeepSort tracking algorithm to perform trajectory prediction and tracking on all objects.

8. The method for detecting a target of a flapping-wing aircraft based on a long-short focus camera according to claim 1, wherein In the step S7, after selecting the tracking target, the coordinates of the center point of the target box of the tracking target are (O x , O y ), and the coordinates of the center point of the short-focus image are (O sx , O sy ). This point is the expected coordinate of the tracking target, that is, to keep the tracking target in the center of the screen; Use Equation (7) to calculate the error between the two: Feed the error diff back to the controller of the pan-tilt processing module. The controller uses an incremental PID controller to generate control quantities for the pitch motor and yaw motor of the two-axis binocular pan-tilt according to the magnitude of the error, and controls the pitch motor and yaw motor to rotate, so that the object moves towards the center of the screen to achieve object tracking.

9. A target detection system for a flapping-wing aircraft based on a long-short focal length camera, the system being used to implement the method according to any one of claims 1 to 8, characterized in that, The system includes a flapping-wing aircraft, a two-axis binocular pan-tilt, a pan-tilt processing module, a ground processing terminal, and a communication module; The two-axis binocular pan-tilt is equipped with long-short focus cameras for obtaining long-short focus video streams during the flight of the flapping-wing aircraft; The ground processing terminal is equipped with a long-short focus fusion object detection algorithm module and an object tracking algorithm module for processing the long-short focus video stream; Among them, the long-short focus fusion object detection algorithm module is used to process the long-short focus video stream using an object detection algorithm to obtain the detected objects in the long-short focus images; use a long-short focus matching algorithm to perform associated matching on the detected objects in the long-short focus images to obtain objects with successful associated matching, objects in the long-focus image without successful associated matching, and objects in the short-focus image without successful associated matching; calculate the position, size, and confidence of the bounding box of the object using the long-short focus fusion formula for the objects with successful associated matching to obtain the fused objects, and do not process the objects without successful associated matching; perform fusion of the object data of the fused objects, the objects in the long-focus image without successful associated matching, and the objects in the short-focus image without successful associated matching, and integrate all the fused object data into the short-focus image as the final detected image for output; The object tracking algorithm module is used to process the final detected image using an object tracking algorithm to perform trajectory prediction and tracking on all objects; The ground processing terminal is further configured to select a tracking target, and transmit error information to the pan-tilt processing module through the communication module according to the coordinate position of the tracking target. The pan-tilt processing module generates a motor control amount according to the error information, and controls the pitching motor and the yaw motor of the two-axis binocular pan-tilt to rotate, so that the tracking target moves towards the center of the screen, realizing the target tracking function; The communication module is used for information transmission between the pan-tilt processing module and the ground processing terminal, including transmitting long and short focal length video streams and error information calculated during target tracking.

10. An electronic device, characterized in that, The electronic device includes: a processor; a memory, on which computer-readable instructions are stored. When the computer-readable instructions are loaded and executed by the processor, the method according to any one of claims 1 to 8 is implemented.

Images