Unmanned aerial vehicle pan-tilt camera target identification and tracking method based on artificial intelligence

Abstract

The invention discloses an unmanned aerial vehicle pan-tilt camera target identification and tracking method based on artificial intelligence, and relates to the technical field of target identification and tracking, and the method comprises the steps: equipment connection and data setting, data acquisition, tracking data analysis, pod pan-tilt camera data analysis, and early warning prompt. Uploading of target person information and setting of a target tracking mode are completed, when it is detected that a target person exists in all pedestrians, current unmanned aerial vehicle data and the shooting direction of a pod pan-tilt camera are obtained, and a target tracking flight point of an unmanned aerial vehicle is obtained; the problem of limitation existing in the current unmanned aerial vehicle pan-tilt camera target recognition tracking development feasibility analysis process is solved, the missing degree and definition of images in all directions at all time points are collected and analyzed, the shooting angle of a pod pan-tilt camera is adjusted, early warning prompt is provided, and the target recognition tracking development feasibility of the pod pan-tilt camera is improved. And the effect in the target identification and tracking process of the unmanned aerial vehicle pan-tilt camera is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of target recognition and tracking, and in particular to a method for target recognition and tracking of a drone gimbal camera based on artificial intelligence. Background Art

[0002] With the development of science and technology, drone technology has gradually matured. Now target recognition and tracking is not only limited to roadside monitoring, but also combines drones with gimbal cameras to complete target recognition and tracking.

[0003] Prior art, such as the invention application patent with announcement number: CN105959625A, discloses a method and device for controlling drone tracking and shooting, including: receiving a reference frame image of a target sent by the drone, and calculating pixel information of the target in the reference frame image; receiving a current frame image of the target sent by the drone, and calculating pixel information of the target in the current frame image; determining a direction to be adjusted based on the pixel information of the target in the reference frame image and the pixel information of the target in the current frame image; and adjusting the shooting posture of the drone according to the direction to be adjusted.

[0004] This application first creates a target tracking task, completes the uploading of the target person information and the setting of the target tracking method. When the target person is detected among the pedestrians, the current UAV data and the shooting direction of the pod gimbal camera are obtained, and the target tracking flight point of the UAV is obtained. This solves the limitations of the current feasibility analysis process of target recognition and tracking by UAV gimbal cameras, collects and analyzes the missing degree and clarity of images in all directions at each time point, adjusts the shooting angle of the pod gimbal camera, and provides early warning prompts, ensuring the effect of the UAV gimbal camera in the target recognition and tracking process.

[0005] With respect to the above scheme, the inventors of the present application have at least discovered that the above technology has at least the following technical problems: 1. The current prior art only analyzes the image, determines the direction to be adjusted based on the pixel information of the target in the reference frame image and the pixel information of the target in the current frame image, and then adjusts the shooting posture of the drone according to the direction to be adjusted. There is a lack of motion prediction of the target person, and no predicted flight point for the next flight of the drone is provided. Changing the drone posture alone makes it impossible to track the moving target person, and thus it is impossible to continuously track the target person, which reduces the effect of drone target tracking to a certain extent.

[0006] 2. There is a lack of convenient user channels, and no simple creation of tracking target tasks is provided, so it is impossible to set the tracking method of the target person; at the same time, there is a lack of pod gimbal camera to connect to the drone, and there is no main control center for two-way control. When the image orientation is incorrect, only the drone posture is adjusted instead of adjusting the camera angle. There is no guarantee that the drone can still maintain stable flight when the camera angle is adjusted. Summary of the invention

[0007] In view of the above-mentioned technical deficiencies, the purpose of this application is to provide an artificial intelligence-based drone gimbal camera target recognition and tracking method.

[0008] To solve the above technical problems, the present application adopts the following technical solutions: The present application provides an artificial intelligence-based UAV gimbal camera target recognition and tracking method, comprising the following steps: Step 1, equipment connection and data setting: install the pod gimbal camera to the bottom of the UAV, and connect the UAV, pod gimbal camera and main control center; create a tracking target task in the main control center, and complete the upload of target person information and the setting of target tracking method.

[0009] Step 2: Data acquisition: Obtain the image set corresponding to each pedestrian at the current flight point of the drone and transmit it to the visual analysis module. When the target person is detected among the pedestrians, output the basic data corresponding to the target person; at the same time, obtain the current drone data and the shooting direction of the pod gimbal camera.

[0010] Step 3: Tracking data analysis: Obtain the target tracking flight point of the drone and the shooting direction of the pod gimbal camera. When the drone flies to the target tracking flight point, obtain the basic data corresponding to the target person again, and make the drone perform the next flight tracking at the predicted movement speed of the target person.

[0011] Step 4: Pod gimbal camera data analysis: Set each time point during the time period when the UAV flies to the target tracking flight point, collect the image loss and clarity of each direction at each time point, and adjust the shooting angle of the pod gimbal camera according to the image loss in each direction. And according to the clarity analysis of each time point, the clarity assessment coefficient is obtained to determine whether the shooting of the pod gimbal camera is abnormal.

[0012] Step 5: Early warning: When the pod gimbal camera's shooting is abnormal, an early warning will be issued.

[0013] Preferably, the basic data of the target person includes predicted movement speed and predicted movement direction.

[0014] Preferably, the current drone data includes the drone flight speed, the aerial parallel distance between the drone and the target person, and the direction of the target person in the drone.

[0015] Preferably, the acquisition of the drone target tracking flight point and the pod gimbal camera shooting direction, the specific acquisition process is as follows: the basic data of the target person, the target tracking method, the current drone data and the pod gimbal camera shooting direction, the standard aerial shooting distance between the drone and the target person are transmitted to the tracking analysis model, and then the drone target tracking flight point and the pod gimbal camera shooting direction are output.

[0016] Preferably, the specific training process of the tracking training model is as follows: extracting a tracking analysis data set from a database, and cleaning, annotating, expanding and preprocessing the tracking analysis data set, and dividing the tracking analysis data set into a tracking analysis training set and a tracking analysis verification set, and then using the tracking analysis training set and the tracking analysis verification set to train the model, thereby verifying and adjusting the training model, and thereby obtaining a tracking analysis model.

[0017] The beneficial effects of the present application are: 1. The artificial intelligence-based drone gimbal camera target recognition and tracking method provided by the present application first creates a tracking target task, completes the uploading of the target person information and the setting of the target tracking method, and when the target person is detected among the pedestrians, obtains the current drone data and the shooting direction of the pod gimbal camera, and obtains the target tracking flight point of the drone, which solves the limitations existing in the feasibility analysis process of the current drone gimbal camera target recognition and tracking development, collects and analyzes the image loss and clarity of each orientation at each time point, adjusts the shooting angle of the pod gimbal camera, and provides early warning prompts, thereby ensuring the effect of the drone gimbal camera target recognition and tracking process.

[0018] 2. This application combines a drone with a pod gimbal camera, and provides a more convenient target person setting and target tracking task setting. It only requires uploading relevant information of the target person to be tracked, and the tracking method of the target person can be freely selected. The target tracking task creation method is simple, making the user application range more extensive.

[0019] 3. This application sets up fixed time intervals for collecting clarity and image loss in all directions at each time point, changes the shooting direction of the pod gimbal camera, and analyzes the clarity. When the shooting of the pod gimbal camera is abnormal, an early warning prompt is issued to provide users with a better drone shooting experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 The figure is a flowchart of the implementation steps of the present application method. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] See also Figure 1 As shown, the present application provides an artificial intelligence-based drone gimbal camera target recognition and tracking method, including the following steps: Step 1, equipment connection and data setting: install the pod gimbal camera to the bottom of the drone, and connect the drone, pod gimbal camera and main control center; create a tracking target task in the main control center, and complete the upload of target person information and the setting of target tracking method.

[0024] In a specific example, the pod gimbal camera is installed at the bottom of the drone, and the drone, the pod gimbal camera and the main control center are connected. The specific installation and connection process is as follows: the four installation points on the top of the pod gimbal camera are installed corresponding to the installation points on the bottom of the drone, and the drone and the pod gimbal camera are turned on, and the drone, the pod gimbal camera and the main control center are respectively connected through Bluetooth technology; the main control center sends a low-altitude flight instruction to the drone, checks the flight status of the drone, and when the drone flight remains stable, the main control center determines that the drone flight is normal; when the drone flight cannot remain stable, the main control center determines that the drone flight is abnormal, and sends a recall instruction to the drone, and checks and stabilizes the installation of the drone and the pod gimbal camera.

[0025] It should be noted that the specific process of checking the flight status of the drone is as follows: gravity sensors are installed at the four corners of the drone, and the gravity at the four corners of the drone is obtained through the gravity sensors at the four corners of the drone body. The stability of the drone flight is judged based on the gravity change value and gravity difference at the four corners of the drone. When the gravity change amplitude at the four corners of the drone is too large or the gravity difference is too large, it is judged that the drone flight cannot remain stable. When the gravity change amplitude at the four corners of the drone is normal and the gravity difference is normal, it is judged that the drone flight remains stable.

[0026] It should be noted that the flight status of the drone is checked because a pod gimbal camera is installed at the bottom of the drone to prevent the drone from becoming unstable or leaning during flight due to the installation of the pod gimbal camera while the drone is performing its mission.

[0027] In a specific example, the target tracking task is created in the main control center, and the target person information is uploaded and the target tracking method is set. The specific creation process is as follows: the target person information includes the target person's gender, height, weight and facial images in all directions; select the Create Tracking Target Task option in the main control center, jump to the Create Tracking Target Task page, fill in the target person's gender, height and weight in the target person information according to the text prompts in the target person information column, and upload the target person's facial images in all directions in the upload box of the target person's facial image; the tracking method includes in front of the target person, behind, to the left and to the right, and select one of the tracking methods; select the Complete Creation option at the bottom of the Create Tracking Target Task page to complete the creation of the tracking target task.

[0028] It should be noted that the facial images of the target person in all directions will be transmitted to the visual analysis module.

[0029] This application combines a drone with a pod gimbal camera, and provides a more convenient target person setting and target tracking task setting. It only requires uploading relevant information of the target person to be tracked, and the tracking method of the target person can be freely selected. The target tracking task creation method is simple, making it more applicable to users.

[0030] Step 2: Data acquisition: Obtain the image set corresponding to each pedestrian at the current flight point of the drone and transmit it to the visual analysis module. When the target person is detected among the pedestrians, output the basic data corresponding to the target person; at the same time, obtain the current drone data and the shooting direction of the pod gimbal camera.

[0031] In a specific example, the basic data of the target person includes a predicted movement speed and a predicted movement direction.

[0032] It should be noted that the predicted movement direction is the movement direction of the target person predicted by the visual analysis module according to the body movements of the target person.

[0033] In a specific example, the current drone data includes the drone's flight speed, the aerial parallel distance between the drone and the target person, and the direction of the target person in the drone.

[0034] Step 3: Tracking data analysis: Obtain the target tracking flight point of the drone and the shooting direction of the pod gimbal camera. When the drone flies to the target tracking flight point, obtain the basic data corresponding to the target person again, and make the drone perform the next flight tracking at the predicted movement speed of the target person.

[0035] In a specific example, the UAV target tracking flight point and the pod gimbal camera shooting direction are obtained, and the specific acquisition process is as follows: the basic data of the target person, the target tracking method, the current UAV data and the pod gimbal camera shooting direction, and the standard aerial shooting distance between the UAV and the target person are transmitted to the tracking analysis model, and then the UAV target tracking flight point and the pod gimbal camera shooting direction are output.

[0036] In a specific example, the tracking training model has a specific training process as follows: extracting a tracking analysis data set from a database, and cleaning, annotating, expanding and preprocessing the tracking analysis data set, and dividing the tracking analysis data set into a tracking analysis training set and a tracking analysis verification set, and then using the tracking analysis training set and the tracking analysis verification set to train the model, thereby verifying and adjusting the training model, and thereby obtaining a tracking analysis model.

[0037] It should be noted that data cleaning is to remove abnormal, duplicate, incomplete or useless data in the data set; data labeling is to label each sample in the data set for use in model training; data expansion is to expand the data set through operations such as rotation, translation, and scaling to improve the generalization ability of the model; data preprocessing is to normalize and denoise the data so that the data meets the requirements of the model input.

[0038] It should be noted that the data set includes the tracking analysis data set, which includes the basic data of the target person, the target tracking method, the current UAV data and the shooting direction of the pod gimbal camera, the standard aerial shooting distance between the UAV and the target person, the UAV target tracking flight point and the shooting direction of the pod gimbal camera.

[0039] Step 4: Pod gimbal camera data analysis: Set each time point during the time period when the UAV flies to the target tracking flight point, collect the image loss and clarity of each direction at each time point, and adjust the shooting angle of the pod gimbal camera according to the image loss in each direction. And according to the clarity analysis of each time point, the clarity assessment coefficient is obtained to determine whether the shooting of the pod gimbal camera is abnormal.

[0040] It should be noted that the missing degree of each azimuth image is set according to the target tracking method. For example, when the target tracking method is frontal, the azimuth images are represented by dividing the image into various azimuth images with the center point of the front of the target tracking person as the segmentation center point. The size of the missing target person in a certain azimuth image is the missing degree of the target person in the azimuth image.

[0041] In a specific example, the shooting angle of the pod gimbal camera is adjusted according to the degree of image loss in each orientation. The specific adjustment process is as follows: the degree of image loss in each orientation is compared with the orientation image loss threshold in the database. When the degree of image loss in a certain orientation is greater than the orientation image loss threshold in the database and no obstruction is detected, the rotation instruction of the pod gimbal camera corresponding to the orientation image loss is obtained in the database, and the rotation instruction is sent to the pod gimbal camera in the main control center, thereby completing the adjustment of the shooting angle of the pod gimbal camera.

[0042] In a specific example, the clarity evaluation coefficient is obtained based on the clarity analysis at each time point. The specific analysis process is as follows: the clarity at each time point is recorded as Q i , where i represents the number corresponding to each time point, i = 1, 2, ..., n, n represents the total number of time points, and n is a natural integer greater than or equal to 2; Substitute into the calculation formula A clarity evaluation coefficient λ is obtained, where e is represented by a natural integer, Q′ is represented by a reference value of the clarity in the database, and σ is represented by a compensation factor corresponding to the clarity in the database.

[0043] It should be noted that 0<σ<1.

[0044] It should be noted that the compensation factor corresponding to the clarity is a digital factor that is multiplied by the uncorrected measurement result to account for the error in the clarity calculation process.

[0045] In a specific example, the specific judgment process for judging whether the shooting of the pod gimbal camera is abnormal is as follows: comparing the clarity assessment coefficient with the clarity assessment coefficient threshold in the database, when the clarity assessment coefficient is less than or equal to the clarity assessment coefficient threshold in the database, it is judged that the shooting of the pod gimbal camera is normal, otherwise it is judged that the shooting of the pod gimbal camera is abnormal.

[0046] This application sets up fixed time intervals for collecting clarity and image loss in all directions at each time point, changes the shooting direction of the pod gimbal camera, and analyzes the clarity. When the shooting of the pod gimbal camera is abnormal, an early warning prompt is issued to provide users with a better drone shooting experience.

[0047] Step 5: Early warning: When the pod gimbal camera's shooting is abnormal, an early warning will be issued.

[0048] It also includes a database for storing tracking analysis data sets, azimuth image loss thresholds, clarity reference values, clarity corresponding compensation factors, and clarity assessment coefficient thresholds.

[0049] The artificial intelligence-based drone gimbal camera target recognition and tracking method provided in the present application first creates a tracking target task, completes the uploading of the target person information and the setting of the target tracking method, and when the target person is detected among the pedestrians, obtains the current drone data and the shooting direction of the pod gimbal camera, and obtains the target tracking flight point of the drone, which solves the limitations existing in the current feasibility analysis process of drone gimbal camera target recognition and tracking development, collects and analyzes the image loss and clarity of each orientation at each time point, adjusts the shooting angle of the pod gimbal camera, and provides early warning prompts, thereby ensuring the effect of the drone gimbal camera target recognition and tracking process.

[0050] The above contents are merely examples and explanations of the concept of the present application. The technical personnel in the relevant technical field may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined in this specification, they should all fall within the scope of protection of the present application.

Claims

1. A target recognition and tracking method for a UAV gimbal camera based on artificial intelligence, characterized in that: include: Step 1: Device connection and data setting: Install the pod gimbal camera to the bottom of the drone, and connect the drone, pod gimbal camera and main control center; create a target tracking task in the main control center, and complete the upload of target person information and the setting of target tracking method; Step 2: Data acquisition: Obtain the image set corresponding to each pedestrian at the current flight point of the drone and transmit it to the visual analysis module. When the target person is detected among the pedestrians, output the basic data corresponding to the target person; at the same time, obtain the current drone data and the shooting direction of the pod gimbal camera; Step 3: Tracking data analysis: Obtain the target tracking flight point of the drone and the shooting direction of the pod gimbal camera. When the drone flies to the target tracking flight point, obtain the basic data corresponding to the target person again, and make the drone perform the next flight tracking at the predicted movement speed of the target person; Step 4: Pod gimbal camera data analysis: Set each time point during the time period when the UAV flies to the target tracking flight point, collect the image loss and clarity of each direction at each time point, and adjust the shooting angle of the pod gimbal camera according to the image loss of each direction. And according to the clarity analysis of each time point, obtain the clarity assessment coefficient, and then judge whether the shooting of the pod gimbal camera is abnormal; Step 5: Early warning: When the pod gimbal camera's shooting is abnormal, an early warning will be issued.

2. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 1, characterized in that: The pod gimbal camera is installed on the bottom of the drone, and the drone, pod gimbal camera and main control center are connected. The specific installation and connection process is as follows: Install the four mounting points on the top of the pod gimbal camera corresponding to the mounting points on the bottom of the drone, turn on the drone and the pod gimbal camera, and connect the drone, pod gimbal camera and the main control center respectively through Bluetooth technology; send a low-altitude flight command to the drone at the main control center to check the flight status of the drone. When the drone flight remains stable, the main control center determines that the drone is flying normally; when the drone flight cannot remain stable, the main control center determines that the drone flight is abnormal, sends a recall command to the drone, and checks and stabilizes the installation of the drone and the pod gimbal camera.

3. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 2, characterized in that: The target tracking task is created in the main control center, and the target person information is uploaded and the target tracking method is set. The specific creation process is as follows: The target person information includes the target person's gender, height, weight and facial images in all directions; select the Create Tracking Target Task option in the main control center, jump to the Create Tracking Target Task page, fill in the target person's gender, height and weight in the target person information according to the text prompts in the target person information column, and upload the target person's facial images in all directions in the upload box of the target person's facial image; tracking methods include in front of the target person, behind, to the left and to the right, and select one of the tracking methods; select the Complete Creation option at the bottom of the Create Tracking Target Task page to complete the creation of the tracking target task.

4. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 3 is characterized in that: The basic data of the target person includes a predicted movement speed and a predicted movement direction.

5. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 4, characterized in that: The current drone data includes the drone flight speed, the aerial parallel distance between the drone and the target person, and the direction of the target person in the drone.

6. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 5, characterized in that: The specific acquisition process of obtaining the target tracking flight point of the drone and the shooting direction of the pod gimbal camera is as follows: The basic data of the target person, the target tracking method, the current UAV data and the shooting direction of the pod gimbal camera, and the standard aerial shooting distance between the UAV and the target person are transmitted to the tracking analysis model, and then the UAV target tracking flight point and the shooting direction of the pod gimbal camera are output.

7. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 6, characterized in that: The specific training process of the tracking training model is as follows: A tracking analysis dataset is extracted from the database, and the tracking analysis dataset is cleaned, annotated, expanded and preprocessed, and the tracking analysis dataset is divided into a tracking analysis training set and a tracking analysis verification set. The tracking analysis training set and the tracking analysis verification set are then used to train the model, thereby verifying and adjusting the training model, and thereby obtaining the tracking analysis model.

8. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 7, characterized in that: The shooting angle of the pod gimbal camera is adjusted according to the image loss degree in each direction. The specific adjustment process is as follows: The image missing degree of each orientation is compared with the orientation image missing degree threshold in the database. When the image missing degree of a certain orientation is greater than the orientation image missing degree threshold in the database and no obstruction is detected, the pod gimbal camera rotation instruction corresponding to the orientation image missing degree is obtained in the database, and the rotation instruction is sent to the pod gimbal camera in the main control center to complete the adjustment of the shooting angle of the pod gimbal camera.

9. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 8, characterized in that: The clarity evaluation coefficient is obtained based on the clarity analysis at each time point, and the specific analysis process is as follows: The clarity at each time point is recorded as Q i , where i represents the number corresponding to each time point, i = 1, 2, ..., n, n represents the total number of time points, and n is a natural integer greater than or equal to 2; Substitute into the calculation formula A clarity evaluation coefficient λ is obtained, where e is represented by a natural integer, Q′ is represented by a reference value of the clarity in the database, and σ is represented by a compensation factor corresponding to the clarity in the database.

10. The method for target recognition and tracking of a drone gimbal camera based on artificial intelligence according to claim 9, characterized in that: The specific process of judging whether the shooting of the pod gimbal camera is abnormal is as follows: The clarity assessment coefficient is compared with the clarity assessment coefficient threshold in the database. When the clarity assessment coefficient is less than or equal to the clarity assessment coefficient threshold in the database, it is judged that there is no abnormality in the shooting of the pod gimbal camera. Otherwise, it is judged that there is an abnormality in the shooting of the pod gimbal camera.

Images