A gimbal-based method and device for pedestrian tracking by unmanned aerial vehicles (UAVs).

Abstract

This invention relates to the field of image processing technology and provides a gimbal-based drone pedestrian following method and apparatus. The gimbal-based drone pedestrian following method includes: extracting a first human image region based on an initial image region selected by the user in the current real-time image; extracting a first distribution feature vector corresponding to the first human image region; acquiring subsequent real-time images and extracting a second human image region from the subsequent real-time images; extracting a second distribution feature vector corresponding to the second human image region; when the vector distance between the first and second distribution feature vectors is less than a first threshold, calculating the actual position difference between the first and second human image regions, and controlling the drone to perform a following action based on the actual position difference. This solution avoids heavy image processing and large-scale data transmission, improves the overall efficiency of the system, and reduces the requirements for hardware performance.

Description

Technical Field

[0001] This invention belongs to the technical field of image processing, and particularly relates to a method and device for pedestrian following by a drone based on a gimbal. Background Technology

[0002] In recent years, with the continuous development and popularization of drone technology, drones have been increasingly used in various fields. Among them, drone-based automatic tracking technology has shown great potential and broad application prospects in fields such as photography, security, and logistics. The core of automatic tracking technology lies in how to accurately identify and track targets, especially in dynamic environments where the accuracy and stability of target tracking are particularly important.

[0003] Traditional drone tracking technology primarily relies on GPS signals and inertial navigation for target localization and tracking. However, these methods may not meet accuracy requirements in certain scenarios, especially in complex environments where GPS signals are easily interfered with, leading to decreased tracking accuracy. Furthermore, while inertial navigation systems can compensate for the shortcomings of GPS signals to some extent, the problem of accumulated errors is difficult to completely avoid.

[0004] To overcome the aforementioned technical bottlenecks, vision-based target tracking technology has emerged. Through real-time image processing, drones can identify and lock onto targets, achieving high-precision automatic following. Existing visual tracking technologies mainly include feature-point-based tracking, template-matching-based tracking, and deep learning-based tracking methods. Among these, deep learning-based tracking methods have made significant progress in recent years, but their high requirements for computing resources and data volume make them difficult to widely apply in resource-constrained drone systems. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for pedestrian following by a drone based on a gimbal, in order to solve the technical problem that deep learning tracking methods have high requirements for computing resources and data volume.

[0006] A first aspect of this invention provides a gimbal-based drone pedestrian following method, the gimbal-based drone pedestrian following method comprising:

[0007] Extract the first human image region based on the initial image region selected by the user in the current real-time image;

[0008] Extract the first distribution feature vector corresponding to the first human image region; wherein, the first distribution feature vector is used to describe the distribution relationship between contour and color;

[0009] Acquire subsequent real-time images, and extract the second human image region from the subsequent real-time images;

[0010] Extract the second distribution feature vector corresponding to the second human image region;

[0011] When the vector distance between the first distribution feature vector and the second distribution feature vector is less than a first threshold, the actual position difference between the first human image region and the second human image region is calculated, and the drone is controlled to perform a following action based on the actual position difference.

[0012] Furthermore, the step of extracting the first human image region based on the initial image region selected by the user in the current real-time image includes:

[0013] Extract the background fusion matrix between the current real-time image and the initial image region;

[0014] Extract the central rectangular region from the initial image region; wherein the central rectangular region is a rectangular region of a first preset size;

[0015] In the initial image region, extract multiple adjacent rectangular regions surrounding each first pixel;

[0016] Calculate the first similarity between the adjacent rectangular regions and the background fusion matrix;

[0017] Calculate the second similarity between the adjacent rectangular regions and the central rectangular region;

[0018] If the first similarity is greater than or equal to the second similarity, then the adjacent rectangular region is taken as the background region;

[0019] If the first similarity is less than the second similarity, then the adjacent rectangular region is taken as the foreground region;

[0020] Count the number of foreground regions in multiple adjacent rectangular regions corresponding to the first pixel;

[0021] Calculate the percentage of the foreground region in the total number of multiple adjacent rectangular regions;

[0022] If the percentage value is within a preset range, the first pixel is taken as an edge pixel; wherein the preset range includes 0.39 to 0.64.

[0023] In the initial image region, one or more image regions composed of consecutive edge pixels are extracted;

[0024] The largest image region is taken as the first human body image region.

[0025] Further, the step of extracting the background fusion matrix between the current real-time image and the initial image region includes:

[0026] The current real-time image is converted to grayscale to obtain a first grayscale image;

[0027] The initial image region is converted to grayscale to obtain a second grayscale image;

[0028] Extract the first grayscale regions at the four diagonal positions in the first grayscale image;

[0029] Extract the second grayscale regions at the four diagonal positions of the second grayscale image; wherein the first grayscale region and the second grayscale region are rectangular regions of a first preset size;

[0030] The average value between the pixel matrices corresponding to the first grayscale regions and the pixel matrices corresponding to the second grayscale regions is used as the background fusion matrix.

[0031] Further, the step of extracting multiple adjacent rectangular regions surrounding each first pixel in the initial image region includes:

[0032] Centered on the first pixel, extract a rectangular image region of a second preset size;

[0033] The rectangular image region is divided into multiple adjacent rectangular regions; wherein, the adjacent rectangular regions are rectangular regions of a first preset size.

[0034] Further, the step of extracting the first distribution feature vector corresponding to the first human image region includes:

[0035] Based on a preset human torso proportion, the first human image region is divided into multiple human torso regions; wherein, the human torso region includes a head region, an upper body region, and a lower body region.

[0036] Extract the pixel value of each second pixel in each of the human torso regions, and calculate the pixel difference between adjacent pixels;

[0037] In each of the human torso regions, adjacent pixels with a pixel difference less than a second threshold are merged to obtain one or more color distribution regions.

[0038] The first distribution feature vector is calculated based on the distribution relationship between the multiple human torso regions and the color distribution regions corresponding to each of the multiple human torso regions.

[0039] Further, the step of calculating the first distribution feature vector based on the distribution relationship between the plurality of human torso regions and the color distribution regions corresponding to each of the plurality of human torso regions includes:

[0040] Calculate the center of the first region among the plurality of said human torso regions;

[0041] Calculate the center of the second region corresponding to the color distribution region of each of the multiple human torso regions;

[0042] Calculate the first distance between the centers of multiple first regions;

[0043] Construct a first distance vector by representing the first distances corresponding to the centers of the multiple first regions;

[0044] Calculate the second distance between the centers of multiple second regions;

[0045] Construct a second distance vector by representing the second distances corresponding to the centers of the multiple second regions;

[0046] Calculate the third distance between the center of the first region and the center of the second region corresponding to each human body trunk region;

[0047] Calculate the third distance corresponding to each of the multiple human torso regions to construct a third distance vector;

[0048] The first distance vector, the second distance vector, and the third distance vector are combined to obtain the first distribution feature vector.

[0049] Furthermore, the step of acquiring subsequent real-time images and extracting the second human image region from the subsequent real-time images includes:

[0050] The number of pixels, region width, and region height corresponding to the first human image region are counted, and the number of pixels is multiplied by the diffusion coefficient to obtain the number of diffused pixels;

[0051] Based on the number of diffused pixels, the region width, and the region height, a diffused image region is generated in the real-time image; wherein the center of the diffused image region coincides with the center of the first human body image region.

[0052] Extract the second human image region from the diffused image region of the subsequent real-time image.

[0053] Further, the step of extracting the first human image region composed of continuous edge pixels in the initial image region includes:

[0054] In the initial image region, extract the edge pixels at each horizontal coordinate;

[0055] If there is a single edge pixel at the current horizontal coordinate, then that edge pixel is taken as the first human body edge pixel.

[0056] If there are multiple edge pixels at the current horizontal coordinate, calculate the pixel distance between the multiple edge pixels;

[0057] Extract multiple edge pixels whose pixel distance is less than the second threshold;

[0058] The average position among multiple edge pixels whose pixel distance is less than the second threshold is taken as the second human body edge pixel.

[0059] The image region formed by the first human body edge pixels and the second human body edge pixels is taken as the first human body image region.

[0060] A second aspect of the present invention provides a gimbal-based unmanned aerial vehicle (UAV) pedestrian following device, comprising:

[0061] The first extraction unit is used to extract the first human image region based on the initial image region selected by the user in the current real-time image;

[0062] The second extraction unit is used to extract the first distribution feature vector corresponding to the first human image region; wherein, the first distribution feature vector is used to describe the distribution relationship between contour and color;

[0063] The acquisition unit is used to acquire subsequent real-time images and extract the second human image region from the subsequent real-time images;

[0064] The third extraction unit is used to extract the second distribution feature vector corresponding to the second human image region;

[0065] The control unit is configured to calculate the actual position difference between the first human image region and the second human image region when the vector distance between the first distribution feature vector and the second distribution feature vector is less than a first threshold, and control the UAV to perform a following action based on the actual position difference.

[0066] A third aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the gimbal-based unmanned aerial vehicle (UAV) pedestrian following method described in the first aspect.

[0067] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the gimbal-based unmanned aerial vehicle (UAV) pedestrian following method described in the first aspect.

[0068] The beneficial effects of this invention compared to existing technologies are as follows: By having the user select an initial image region in the current real-time image, a first human image region is extracted, and the corresponding first distribution feature vector is further extracted. This distribution feature vector can effectively describe the distribution relationship between human body contours and colors, enabling more accurate identification and tracking of target individuals in subsequent image processing. By comparing the vector distance between the first and second distribution feature vectors, this invention achieves target matching based on feature vectors, improving the accuracy and robustness of target recognition, and enabling stable target tracking even under environmental changes or poor lighting conditions. By acquiring subsequent images in real time and extracting the current target human image region in each frame, the UAV can quickly respond to target movement. The actual position difference between the first and second human image regions is calculated, and the UAV is controlled to perform a following action based on this position difference, enabling the UAV to closely follow the target and ensuring the real-time performance and smoothness of the following process. Compared to deep learning-based methods, this invention uses a feature vector-based tracking method, which has lower computational complexity and can achieve efficient target tracking with limited computing resources. By matching feature vectors and calculating distances, the heavy image processing and large amount of data transmission are avoided, improving the overall efficiency of the system and reducing the requirements for hardware performance, making it suitable for resource-constrained UAV platforms. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 A schematic flowchart of a gimbal-based unmanned aerial vehicle (UAV) pedestrian following method provided by the present invention is shown.

[0071] Figure 2 A schematic diagram of a gimbal-based unmanned aerial vehicle (UAV) pedestrian following device according to an embodiment of the present invention is shown.

[0072] Figure 3 A schematic diagram of a terminal device provided in an embodiment of the present invention is shown. Detailed Implementation

[0073] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0074] This invention provides a gimbal-based method and apparatus for pedestrian following by unmanned aerial vehicles (UAVs), which addresses the technical problem that deep learning-based tracking methods have high requirements for computing resources and data volume.

[0075] First, this invention provides a gimbal-based method for pedestrian following using a drone. Please see [link / reference]. Figure 1 , Figure 1 A schematic flowchart of a gimbal-based drone pedestrian following method provided by the present invention is shown. Figure 1 As shown, the gimbal-based drone pedestrian following method may include the following steps:

[0076] Step 101: Extract the first human image region based on the initial image region selected by the user in the current real-time image;

[0077] The user selects an initial image region containing the pedestrian from the real-time image transmitted by the drone's camera, using an interactive method (such as selecting a box on a mobile device or controller screen). This selected region contains the pedestrian's image information. Based on this initial image region, the drone extracts the first human image region, as follows:

[0078] Specifically, step 101 includes steps A1 to A11:

[0079] Step A1: Extract the background fusion matrix between the current real-time image and the initial image region;

[0080] The background fusion matrix is ​​used to describe the background information in an image.

[0081] Specifically, step A1 includes steps A101 to A105:

[0082] Step A101: Convert the current real-time image to grayscale to obtain a first grayscale image;

[0083] Grayscale conversion is the process of converting a color image into a grayscale image. A grayscale image contains only brightness information; the value of each pixel represents its brightness, but not color information. Grayscale conversion simplifies subsequent calculations and processing.

[0084] Step A102: Convert the initial image region to grayscale to obtain a second grayscale image;

[0085] Similarly, the initial image region is converted to grayscale to obtain a grayscale image containing only brightness information. The purpose of this step is to allow the initial image region to be compared with the current real-time image on the same processing dimension.

[0086] Step A103: Extract the first grayscale regions at the four diagonal positions in the first grayscale image;

[0087] In the grayscale image of the current real-time image, select grayscale regions at four diagonal positions. These regions typically represent background information because the diagonal positions usually do not contain the target pedestrian.

[0088] Step A104: Extract the second grayscale regions at the four diagonal positions in the second grayscale image; wherein the first grayscale region and the second grayscale region are rectangular regions of a first preset size;

[0089] Similarly, in the grayscale image of the initial image region, four diagonally opposite grayscale regions are selected. It is assumed that these regions also primarily contain background information.

[0090] Each grayscale region is a rectangular area of ​​a preset size. The preset size is selected based on the specific application scenario to balance computational complexity and background information accuracy.

[0091] Step A105: The average value between the pixel matrices corresponding to the multiple first grayscale regions and the pixel matrices corresponding to the multiple second grayscale regions is used as the background fusion matrix.

[0092] The pixels in each of the first and second grayscale regions are averaged (averaging the position of each element) to obtain a background fusion matrix representing the background brightness, which represents the background brightness features of the current real-time image and the initial image region.

[0093] In the embodiments corresponding to steps A101 to A105, grayscale processing simplifies the information in the image, and background features are captured by extracting grayscale regions at diagonal positions. The selection of diagonal positions is based on the assumption that these regions mainly contain background information, rather than the target pedestrian. By calculating the average value of these regions, a fusion matrix representing the background brightness can be formed. The role of the background fusion matrix is ​​to help distinguish between the foreground and background in the image. Because the background fusion matrix captures the brightness features of the background, the foreground region (i.e., the target pedestrian) can be effectively identified by comparing it with the real-time image. This method has practical application value in image processing and computer vision, especially in resource-constrained environments, because grayscale processing and matrix averaging calculation are relatively simple and efficient.

[0094] Step A2: Extract the central rectangular region from the initial image region; wherein the central rectangular region is a rectangular region of a first preset size;

[0095] Within the initial image area selected by the user, a central rectangular region of a preset size is defined. This region, located at the center of the selected area, primarily contains the features of the target pedestrian.

[0096] Step A3: In the initial image region, extract multiple adjacent rectangular regions surrounding each first pixel;

[0097] For each pixel in the initial image region, several adjacent rectangular regions are selected. These regions are used for subsequent similarity calculations.

[0098] Specifically, step A3 includes steps A31 to A32:

[0099] Step A31: Using the first pixel as the center, extract a rectangular image region of a second preset size;

[0100] Identify a first pixel in the initial image region and use it as the center point. Around this center point, extract a rectangular image region of a preset size. For example, if the second preset size is 20×20 pixels, this means the rectangular region around that pixel will contain 400 pixels (20 rows and 20 columns).

[0101] Select a center pixel and extract a large rectangular area (second preset size) around it. The purpose of this step is to ensure that contextual information around the center pixel is captured, not just information about a single pixel.

[0102] Step A32: Divide the rectangular image region into multiple adjacent rectangular regions; wherein the adjacent rectangular regions are rectangular regions of a first preset size.

[0103] The extracted rectangular image region of the second preset size is further divided into multiple adjacent smaller rectangular regions. For example, if each smaller rectangular region is 5×5 pixels in size, then a large rectangular region of 20×20 can be divided into 16 smaller rectangular regions of 5×5 pixels each.

[0104] Each adjacent small rectangular area is the same size, determined by a first preset size. For example, if the first preset size is 5×5 pixels, this means that each adjacent rectangular area contains 25 pixels.

[0105] This large rectangular region is divided into multiple adjacent smaller rectangular regions (of a first preset size). This division allows for a more detailed analysis of the local features surrounding the central pixel.

[0106] Each small rectangular region has a fixed size, which ensures standardization and consistency in the analysis. By extracting information from these adjacent rectangular regions, fine features around the central pixel can be obtained, which is crucial for background modeling and foreground separation in pedestrian following algorithms.

[0107] In the embodiments corresponding to steps A31 and A32, specific steps are taken to extract and analyze multiple adjacent rectangular regions around the center pixel in the initial image region. These steps, by refining feature extraction and background modeling, help improve the accuracy and efficiency of the UAV pedestrian following method.

[0108] Step A4: Calculate the first similarity between the adjacent rectangular regions and the background fusion matrix;

[0109] Each adjacent rectangular region is compared to the background fusion matrix, and their similarity is calculated. Similarity can be measured by the difference in pixel values, for example, based on mean squared error (MSE) or correlation coefficient.

[0110] Step A5: Calculate the second similarity between the adjacent rectangular regions and the central rectangular region;

[0111] Each adjacent rectangular region is compared with the central rectangular region, and their similarity is calculated. The purpose of this step is to determine whether these adjacent regions share the same characteristics as the central region (assuming it is the human body region).

[0112] Step A6: If the first similarity is greater than or equal to the second similarity, then the adjacent rectangular region is taken as the background region;

[0113] If the similarity between an adjacent rectangular region and the background is greater than or equal to its similarity with the central region, then the adjacent region is considered to belong to the background.

[0114] Step A7: If the first similarity is less than the second similarity, then the adjacent rectangular region is taken as the foreground region;

[0115] Conversely, if the adjacent region is more similar to the central region, it is considered to belong to the foreground region, that is, it may be part of the human body.

[0116] Step A8: Count the number of foreground regions in the multiple adjacent rectangular regions corresponding to the first pixel;

[0117] For each pixel, count the number of adjacent rectangular regions that are identified as foreground regions. Step A9: Calculate the proportion of the number of foreground regions in the total number of adjacent rectangular regions;

[0118] Calculate the proportion of the foreground region to the total number of all adjacent rectangular regions.

[0119] Step A10: If the percentage value is within a preset range, then the first pixel is taken as an edge pixel; wherein the preset range includes 0.39 to 0.64;

[0120] If the proportion of the foreground region is within a preset range (0.39 to 0.64), the pixel is considered to be located at the edge of the human body. This range was determined experimentally to indicate an appropriate foreground region proportion.

[0121] Step A11: In the initial image region, extract the first human image region composed of continuous edge pixels;

[0122] Specifically, step A11 includes steps A111 to A116:

[0123] Step A111: In the initial image region, extract the edge pixels at each horizontal coordinate;

[0124] In the initial image region, for each x-coordinate (i.e., each column in the image), the edge pixels in that column are extracted.

[0125] Step A112: If there is a single edge pixel at the current horizontal coordinate, then take that edge pixel as the first human body edge pixel;

[0126] If there is only one edge pixel at the current horizontal coordinate, then that edge pixel is directly regarded as the first human body edge pixel. This step is relatively simple because there are no other edge pixels that need further processing.

[0127] Step A113: If there are multiple edge pixels at the current horizontal coordinate, calculate the pixel distance between the multiple edge pixels;

[0128] If there are multiple edge pixels at the current horizontal coordinate, it is necessary to further calculate the distance between these edge pixels. The purpose of this is to identify which edge pixels belong to the same human body contour.

[0129] Step A114: Extract multiple edge pixels whose pixel distance is less than the second threshold;

[0130] For multiple edge pixels, calculate the pixel distance between them. If the pixel distance is less than a predetermined threshold (second threshold), these edge pixels are considered to be part of the same human body contour. Only multiple edge pixels with a pixel distance less than the second threshold are extracted to filter out edge pixels that may be noise or other objects.

[0131] Step A115: Take the average position among multiple edge pixels whose pixel distance is less than the second threshold as the second human body edge pixel;

[0132] For edge pixels whose pixel distance is less than the second threshold, calculate their average position. This average position can more accurately represent the human body edge position at that horizontal coordinate. Use the calculated average position as the second human body edge pixel.

[0133] Step A116: The image region formed by the first human body edge pixels and the second human body edge pixels is taken as the first human body image region.

[0134] All the edge pixels of the first and second human bodies are combined to form a complete first human body image region. This region is composed of continuous edge pixels, representing the outline of the human body.

[0135] In the embodiments corresponding to steps A111 to A116, these steps allow for the extraction of a precise first human image region composed of continuous edge pixels from the initial image region. This is crucial for subsequent pedestrian tracking and image processing, as it ensures the accuracy and integrity of the target human body region.

[0136] In the embodiments corresponding to steps A1 to A11, the human body portion in the initial image region is separated by the difference between the background and foreground. A background fusion matrix can be used to identify and exclude background portions in the initial image. Similarly, features of the central rectangular region help determine which regions belong to the foreground. The foreground region is evaluated based on similarity calculations, and edge pixels are obtained through statistical evaluation. Finally, the largest connected region is selected as the human body region. This method effectively utilizes the contrast between the background and foreground and ensures accurate extraction of the pedestrian region through pixel-by-pixel analysis. This method has practical application value in image processing and computer vision, enabling accurate pedestrian tracking without relying on complex deep learning models.

[0137] Step 102: Extract the first distribution feature vector corresponding to the first human image region; wherein, the first distribution feature vector is used to describe the distribution relationship between contour and color;

[0138] Image features of pedestrians are extracted from the user-selected image region. These features are represented by feature vectors. The feature vectors characterize the distribution of various attributes (such as contour and color) in the image. The feature vectors contain the distributional relationship between contour and color, and can be understood as a comprehensive description of the pedestrian's shape and color features.

[0139] Specifically, step 102 includes steps 1021 to 1024:

[0140] Step 1021: Based on a preset human torso proportion, the first human image region is divided into multiple human torso regions; wherein, the human torso region includes a head region, an upper body region, and a lower body region.

[0141] Based on a pre-defined human torso proportion, the first human image region (i.e., the image region containing pedestrians) is divided into several specific areas. These regions typically include a head region, an upper body region, and a lower body region. The specific division proportions may be based on specific proportions in human anatomy, such as the head accounting for 1 / 7 of the total height and the upper body accounting for 4 / 7 of the total height.

[0142] Based on a preset ratio, the human body image is divided into different torso regions (head, upper body, and lower body). This segmentation helps to extract features from each part separately, improving the precision and accuracy of feature description.

[0143] Step 1022: Extract the pixel value of each second pixel in each of the human torso regions, and calculate the pixel difference between adjacent pixels;

[0144] Within each defined human torso region, the pixel value (RGB or grayscale value, etc.) of each second pixel is extracted. Then, the pixel differences between these pixels and their neighboring pixels are calculated. These differences can reflect local variations and texture features of the image.

[0145] Within each torso region, pixel values ​​are extracted and the differences between adjacent pixels are calculated. Pixel differences are an important indicator of local texture features, reflecting subtle image changes.

[0146] Step 1023: Merge adjacent pixels with a pixel difference less than the second threshold in each of the human torso regions to obtain one or more color distribution regions;

[0147] For each pixel within the torso region, if the pixel difference between adjacent pixels is less than a preset second threshold, they are considered to belong to the same color distribution region, and these adjacent pixels are merged. This allows pixels of similar colors to be grouped into one or more color distribution regions.

[0148] By setting a threshold, pixels whose difference between adjacent pixels is less than the threshold are merged. This step helps to eliminate minor noise and details, and aggregates regions of similar color to form color distribution areas.

[0149] Step 1024: Calculate the first distribution feature vector based on the distribution relationship between the multiple human torso regions and the color distribution regions corresponding to each of the multiple human torso regions.

[0150] Finally, based on the color distribution areas within each torso region and their interrelationships, a first overall distribution feature vector is calculated. This feature vector integrates the color and texture information of the pedestrian's head, upper body, and lower body, and can be used for subsequent pedestrian recognition and tracking.

[0151] Based on the color distribution areas of each torso region and their interrelationships, a comprehensive distribution feature vector is calculated. This feature vector integrates the color and texture features of different torso parts and is an important feature for describing the appearance of pedestrians.

[0152] In the embodiments corresponding to steps 1021 to 1024, segmenting the torso region and extracting features separately allows for a more refined description of the pedestrian's appearance, improving the accuracy of recognition and tracking. Merging pixel differences and color distribution regions effectively reduces noise interference and enhances the robustness of the features. When calculating the distribution feature vector, the feature information from each torso region is integrated, providing a comprehensive description of the pedestrian's appearance, which helps improve the stability and reliability of the tracking algorithm. These steps, through refined feature extraction and comprehensive representation, contribute to improving the accuracy and robustness of the gimbal-based UAV pedestrian following method.

[0153] Specifically, step 1024 includes steps 10241 to 10249:

[0154] Step 10241: Calculate the first region center of the plurality of human torso regions;

[0155] Step 10242: Calculate the center of the second region corresponding to the color distribution region of each of the multiple human torso regions;

[0156] For each torso region, it is also necessary to calculate the center point of its color distribution area.

[0157] Step 10243: Calculate the first distance between the centers of the plurality of first regions;

[0158] Calculate the distances between the center points of these first regions. In practice, calculate the Euclidean distance between any two center points of the first regions.

[0159] Step 10244: Construct a first distance vector from the first distances corresponding to the centers of the multiple first regions;

[0160] Organize all the distance values ​​between each other obtained from the above calculation into a vector, which is called the "first distance vector".

[0161] Step 10245: Calculate the second distance between the centers of the multiple second regions;

[0162] Similarly, calculate the distance between the centers of every two second regions.

[0163] Step 10246: Construct a second distance vector from the second distances corresponding to the centers of the multiple second regions;

[0164] The distance values ​​between all the centers of the second region obtained above are organized into a vector, which is called the "second distance vector".

[0165] Step 10247: Calculate the third distance between the center of the first region and the center of the second region corresponding to each human torso region;

[0166] For each human torso region, calculate the distance between the center of its first region and the center of its corresponding second region.

[0167] Step 10248: Calculate the third distance corresponding to each of the multiple human torso regions and construct a third distance vector;

[0168] The distance values ​​between the first and second region centers of each torso region are organized into a vector, which is called the "third distance vector".

[0169] Step 10249: Merge the first distance vector, the second distance vector, and the third distance vector to obtain the first distribution feature vector.

[0170] Finally, the first distance vector, the second distance vector, and the third distance vector are combined to form a comprehensive vector, which is the "first distribution feature vector".

[0171] In the embodiments corresponding to steps 10241 to 10249, these steps yield a comprehensive distribution feature vector that integrates the geometric and color distribution relationships between the torso region and its corresponding color distribution region. This feature vector can be used as a feature description in subsequent algorithms, and thus for applications such as pedestrian tracking or recognition.

[0172] Step 103: Acquire subsequent real-time images and extract the second human image region from the subsequent real-time images;

[0173] As the drone flies, the camera continuously captures new real-time images. In these new real-time images, the system locates the area where the pedestrian is located and extracts the new pedestrian image area.

[0174] Specifically, step 103 includes steps 1031 to 1033:

[0175] Step 1031: Count the number of pixels, region width and region height corresponding to the first human image region, and multiply the number of pixels by the diffusion coefficient to obtain the number of diffused pixels;

[0176] The number of pixels, width, and height of the first human body image region are counted. These parameters allow us to understand the size and shape of the first human body image region.

[0177] The number of pixels refers to the total number of pixels contained in the first human body image region.

[0178] The region width and region height are the width and height of the first human body image region, respectively.

[0179] The number of pixels counted is multiplied by a predetermined diffusion coefficient to obtain the number of diffused pixels. The diffusion coefficient is a parameter used to control the degree of diffusion; by adjusting it, the size of the diffused area can be controlled.

[0180] Step 1032: Based on the number of diffused pixels, the region width, and the region height, generate a diffused image region in the real-time image; wherein the center of the diffused image region coincides with the center of the first human body image region;

[0181] Based on the calculated number of diffused pixels, and the width and height of the first human image region, a diffused image region is generated in the real-time image. This diffused image region is an expanded search area used to more easily find the second human image region in subsequent real-time images.

[0182] The key is to ensure that the center of the diffused image region coincides with the center of the first human body image region. This means that the diffused image region expands with the first human body image region as the center, thus enabling better target tracking.

[0183] Step 1033: Extract the second human image region from the diffused image region of the subsequent real-time image.

[0184] In subsequent real-time images, the focus is on searching and extracting the second human image region within the diffused image region. Because the diffused image region already includes the center point of the first human image region, searching and extracting the target within this extended region will be more accurate and effective.

[0185] In the embodiments corresponding to steps 1031 to 1033, these steps allow for dynamic adjustment of the search area in real-time images, ensuring continuous tracking of the target pedestrian. This method can improve the robustness and accuracy of UAV pedestrian following in complex environments.

[0186] Step 104: Extract the second distribution feature vector corresponding to the second human image region;

[0187] A new feature vector (second distribution feature vector) of the pedestrian is extracted from subsequent images. This vector also describes the pedestrian's contour and color distribution. The extraction logic of the second distribution feature vector and the first distribution feature vector will not be elaborated here.

[0188] Step 105: When the vector distance between the first distribution feature vector and the second distribution feature vector is less than the first threshold, calculate the actual position difference between the first human image region and the second human image region, and control the UAV to perform a following action based on the actual position difference.

[0189] Compare the distance between the feature vector of the initial image region (first distribution feature vector) and the feature vector of the current image region (second distribution feature vector). If the distance between these two vectors is less than a preset threshold (meaning the pedestrian in the current image region is the originally selected pedestrian), then calculate the actual position difference between the initial pedestrian position and the current pedestrian position.

[0190] Based on this positional difference, the drone's position and attitude are adjusted to maintain its tracking of the pedestrian.

[0191] As an optional embodiment of this application, the bounding box ratio of the human body will change when the drone's viewing angle or altitude changes, leading to ranging deviation. To solve this problem, this application adjusts the camera angle in real time using sensor data (such as IMU, GPS, altimeter, etc.) to keep the target object stable in the field of view. The specific implementation logic is as follows:

[0192] During flight, the drone's IMU, GPS, and altimeter collect attitude, position, and altitude data in real time; the camera captures a real-time video stream. The gimbal controller adjusts the camera angle in real time based on IMU sensor data to reduce the impact of flight sway on video capture and keep the target object centered in the field of view. The IMU sensor acquires the drone's pitch, roll, and yaw angles. The altimeter or laser rangefinder obtains the drone's current altitude.

[0193] Obtain the camera intrinsic parameter matrix (K): containing information about the focal length and optical center coordinates. Obtain the camera extrinsic parameter matrix (R,T): containing the rotation matrix (R) and translation vector (T), describing the camera's position and orientation in the world coordinate system.

[0194] Suppose the four vertices of the bounding box are (x1, y1), (x2, y2), (x3, y3), and (x4, y4).

[0195] Convert image coordinates to camera coordinates:

[0196] Transform the camera coordinate system to the world coordinate system:

[0197] Obtain the rotation matrices corresponding to pitch, roll, and yaw angles. Multiplying by the rotation matrix yields Z k Adding the height difference to obtain

[0198] Reproject the world coordinate system onto the image plane:

[0199] Using known camera parameters and altitude information, the actual distance to a target object can be calculated using triangulation. f represents the focal length.

[0200] In the embodiments corresponding to steps 101 to 105, the user selects an initial image region in the current real-time image to extract the first human image region, and further extracts the corresponding first distribution feature vector. This distribution feature vector can effectively describe the distribution relationship between human body contours and colors, enabling more accurate identification and tracking of the target person in subsequent image processing. By comparing the vector distance between the first and second distribution feature vectors, this invention achieves target matching based on feature vectors, improving the accuracy and robustness of target recognition, and enabling stable target tracking even under environmental changes or poor lighting conditions. By acquiring subsequent images in real time and extracting the current target human image region in each frame, the UAV can quickly respond to the target's movement. The actual position difference between the first and second human image regions is calculated, and the UAV is controlled to perform a following action based on this position difference, enabling the UAV to closely follow the target and ensuring the real-time performance and smoothness of the following process. Compared to deep learning-based methods, this invention uses a feature vector-based tracking method, which has lower computational complexity and can achieve efficient target tracking with limited computing resources. By matching feature vectors and calculating distances, the heavy image processing and large amount of data transmission are avoided, improving the overall efficiency of the system and reducing the requirements for hardware performance, making it suitable for resource-constrained UAV platforms.

[0201] like Figure 2 This invention provides a gimbal-based drone pedestrian following device; please refer to [link / reference]. Figure 2 , Figure 2 The diagram shows a schematic of a gimbal-based unmanned aerial vehicle (UAV) pedestrian following device provided by the present invention, as shown below. Figure 2 The gimbal-based drone pedestrian following device shown includes:

[0202] The first extraction unit 21 is used to extract a first human image region based on an initial image region selected by the user in the current real-time image.

[0203] The second extraction unit 22 is used to extract the first distribution feature vector corresponding to the first human image region; wherein, the first distribution feature vector is used to describe the distribution relationship between contour and color;

[0204] Acquisition unit 23 is used to acquire subsequent real-time images and extract the second human image region from the subsequent real-time images;

[0205] The third extraction unit 24 is used to extract the second distribution feature vector corresponding to the second human image region;

[0206] The control unit 25 is used to calculate the actual position difference between the first human image region and the second human image region when the vector distance between the first distribution feature vector and the second distribution feature vector is less than a first threshold, and to control the drone to perform a following action based on the actual position difference.

[0207] This invention provides a gimbal-based drone pedestrian following device. The user selects an initial image region within the current real-time image, extracts a first human image region, and further extracts the corresponding first distribution feature vector. This distribution feature vector effectively describes the distribution relationship between human contours and colors, enabling more accurate identification and tracking of the target person in subsequent image processing. By comparing the vector distance between the first and second distribution feature vectors, this invention achieves feature vector-based target matching, improving the accuracy and robustness of target recognition, and ensuring stable target tracking even under changing environmental conditions or poor lighting. Real-time acquisition of subsequent images and extraction of the current target human image region in each frame ensures the drone can quickly respond to target movement. The actual positional difference between the first and second human image regions is calculated, and the drone is controlled to perform following actions based on this difference, enabling the drone to closely follow the target and ensuring the real-time and smoothness of the following process. Compared to deep learning-based methods, this invention employs a feature vector-based tracking method, which has lower computational complexity and can achieve efficient target tracking with limited computing resources. By matching feature vectors and calculating distances, the heavy image processing and large amount of data transmission are avoided, improving the overall efficiency of the system and reducing the requirements for hardware performance, making it suitable for resource-constrained UAV platforms.

[0208] Figure 3 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. Figure 3 As shown, a terminal device 3 in this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a gimbal-based drone pedestrian following method program. When the processor 30 executes the computer program 32, it implements the steps described in the various embodiments of the gimbal-based drone pedestrian following method, for example... Figure 1 Steps 101 to 105 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each unit in the above-described device embodiments, for example... Figure 2 The function of the unit shown.

[0209] For example, the computer program 32 can be divided into one or more units, which are stored in the memory 31 and executed by the processor 30 to complete the present invention. The one or more units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 32 in the terminal device 3. For example, the specific functions of each unit of the computer program 32 can be divided as follows:

[0210] The first extraction unit is used to extract the first human image region based on the initial image region selected by the user in the current real-time image;

[0211] The second extraction unit is used to extract the first distribution feature vector corresponding to the first human image region; wherein, the first distribution feature vector is used to describe the distribution relationship between contour and color;

[0212] The acquisition unit is used to acquire subsequent real-time images and extract the second human image region from the subsequent real-time images;

[0213] The third extraction unit is used to extract the second distribution feature vector corresponding to the second human image region;

[0214] The control unit is configured to calculate the actual position difference between the first human image region and the second human image region when the vector distance between the first distribution feature vector and the second distribution feature vector is less than a first threshold, and control the UAV to perform a following action based on the actual position difference.

[0215] The terminal device includes, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of a terminal device 3 and does not constitute a limitation on a terminal device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0216] The processor 30 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0217] The memory 31 can be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. The memory 31 can also be an external storage device of the terminal device 3, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 3. Furthermore, the memory 31 can include both internal and external storage units of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0218] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0219] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0220] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0221] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0222] This invention provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0223] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0224] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0225] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0226] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0227] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units.

[0228] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0229] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0230] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0231] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0232] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0233] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for pedestrian following using a gimbal-based drone, characterized in that, The gimbal-based drone pedestrian following method includes: Based on the initial image region selected by the user in the current real-time image, extract the first human image region; Extract the first distribution feature vector corresponding to the first human image region; wherein, the first distribution feature vector is used to describe the distribution relationship between contour and color, and the first distribution feature vector integrates the color and texture information of the pedestrian's head, upper body and lower body; Acquire subsequent real-time images, and extract the second human image region from the subsequent real-time images; Extract the second distribution feature vector corresponding to the second human image region; When the vector distance between the first distribution feature vector and the second distribution feature vector is less than a first threshold, the actual position difference between the first human image region and the second human image region is calculated, and the drone is controlled to perform a following action based on the actual position difference. The step of extracting the first human image region based on the initial image region selected by the user in the current real-time image includes: Extract the background fusion matrix between the current real-time image and the initial image region; Extract the central rectangular region from the initial image region; wherein the central rectangular region is a rectangular region of a first preset size; In the initial image region, extract multiple adjacent rectangular regions surrounding each first pixel; Calculate the first similarity between the adjacent rectangular regions and the background fusion matrix; Calculate the second similarity between the adjacent rectangular regions and the central rectangular region; If the first similarity is greater than or equal to the second similarity, then the adjacent rectangular region is taken as the background region; If the first similarity is less than the second similarity, then the adjacent rectangular region is taken as the foreground region; Count the number of foreground regions in multiple adjacent rectangular regions corresponding to the first pixel; Calculate the percentage of the foreground region in the total number of multiple adjacent rectangular regions; If the percentage value is within a preset range, then the first pixel is taken as an edge pixel; wherein the preset range is 0.39 to 0.

64. In the initial image region, the first human image region composed of continuous edge pixels is extracted; The step of extracting the background fusion matrix between the current real-time image and the initial image region includes: The current real-time image is converted to grayscale to obtain a first grayscale image; The initial image region is converted to grayscale to obtain a second grayscale image; Extract the first grayscale regions at the four diagonal positions in the first grayscale image; Extract the second grayscale regions at the four diagonal positions of the second grayscale image; wherein the first grayscale region and the second grayscale region are rectangular regions of a first preset size; The average value between the pixel matrices corresponding to the first grayscale regions and the pixel matrices corresponding to the second grayscale regions is used as the background fusion matrix.

2. The gimbal-based UAV pedestrian following method as described in claim 1, characterized in that, The step of extracting multiple adjacent rectangular regions around each first pixel in the initial image region includes: Centered on the first pixel, extract a rectangular image region of a second preset size; The rectangular image region is divided into multiple adjacent rectangular regions; wherein, the adjacent rectangular regions are rectangular regions of a first preset size.

3. The gimbal-based UAV pedestrian following method as described in claim 1, characterized in that, The step of extracting the first distribution feature vector corresponding to the first human image region includes: Based on a preset human torso proportion, the first human image region is divided into multiple human torso regions; wherein, the human torso region includes a head region, an upper body region, and a lower body region. Extract the pixel value of each second pixel in each of the human torso regions, and calculate the pixel difference between adjacent pixels; In each of the human torso regions, adjacent pixels with a pixel difference less than a second threshold are merged to obtain one or more color distribution regions. The first distribution feature vector is calculated based on the distribution relationship between the multiple human torso regions and the color distribution regions corresponding to each of the multiple human torso regions.

4. The gimbal-based UAV pedestrian following method as described in claim 3, characterized in that, The step of calculating the first distribution feature vector based on the distribution relationship between the multiple human torso regions and the color distribution regions corresponding to each of the multiple human torso regions includes: Calculate the center of the first region among the plurality of said human torso regions; Calculate the center of the second region corresponding to the color distribution region of each of the multiple human torso regions; Calculate the first distance between the centers of multiple first regions; Construct a first distance vector by representing the first distances corresponding to the centers of the multiple first regions; Calculate the second distance between the centers of multiple second regions; Construct a second distance vector by representing the second distances corresponding to the centers of the multiple second regions; Calculate the third distance between the center of the first region and the center of the second region corresponding to each human body trunk region; Calculate the third distance corresponding to each of the multiple human torso regions to construct a third distance vector; The first distance vector, the second distance vector, and the third distance vector are combined to obtain the first distribution feature vector.

5. The gimbal-based UAV pedestrian following method as described in claim 1, characterized in that, The step of acquiring subsequent real-time images and extracting the second human image region from the subsequent real-time images includes: The number of pixels, region width, and region height corresponding to the first human image region are counted, and the number of pixels is multiplied by the diffusion coefficient to obtain the number of diffused pixels; Based on the number of diffused pixels, the region width, and the region height, a diffused image region is generated in the real-time image; wherein the center of the diffused image region coincides with the center of the first human body image region. Extract the second human image region from the diffused image region of the subsequent real-time image.

6. The gimbal-based UAV pedestrian following method as described in claim 1, characterized in that, The step of extracting the first human image region composed of continuous edge pixels in the initial image region includes: In the initial image region, extract the edge pixels at each horizontal coordinate; If there is a single edge pixel at the current horizontal coordinate, then that edge pixel is taken as the first human body edge pixel. If there are multiple edge pixels at the current horizontal coordinate, calculate the pixel distance between the multiple edge pixels; Extract multiple edge pixels whose pixel distance is less than the second threshold; The average position among multiple edge pixels whose pixel distance is less than the second threshold is taken as the second human body edge pixel. The image region formed by the first human body edge pixels and the second human body edge pixels is taken as the first human body image region.

7. A gimbal-based unmanned aerial vehicle (UAV) pedestrian following device, characterized in that, The gimbal-based unmanned aerial vehicle (UAV) pedestrian following device includes: The first extraction unit is used to extract the first human image region based on the initial image region selected by the user in the current real-time image; The second extraction unit is used to extract the first distribution feature vector corresponding to the first human image region; wherein, the first distribution feature vector is used to describe the distribution relationship between contour and color, and the first distribution feature vector integrates the color and texture information of the pedestrian's head, upper body and lower body; The acquisition unit is used to acquire subsequent real-time images and extract the second human image region from the subsequent real-time images; The third extraction unit is used to extract the second distribution feature vector corresponding to the second human image region; The control unit is configured to calculate the actual position difference between the first human image region and the second human image region when the vector distance between the first distribution feature vector and the second distribution feature vector is less than a first threshold, and control the drone to perform a following action based on the actual position difference. The step of extracting the first human image region based on the initial image region selected by the user in the current real-time image includes: Extract the background fusion matrix between the current real-time image and the initial image region; Extract the central rectangular region from the initial image region; wherein the central rectangular region is a rectangular region of a first preset size; In the initial image region, extract multiple adjacent rectangular regions surrounding each first pixel; Calculate the first similarity between the adjacent rectangular regions and the background fusion matrix; Calculate the second similarity between the adjacent rectangular regions and the central rectangular region; If the first similarity is greater than or equal to the second similarity, then the adjacent rectangular region is taken as the background region; If the first similarity is less than the second similarity, then the adjacent rectangular region is taken as the foreground region; Count the number of foreground regions in multiple adjacent rectangular regions corresponding to the first pixel; Calculate the percentage of the foreground region in the total number of multiple adjacent rectangular regions; If the percentage value is within a preset range, then the first pixel is taken as an edge pixel; wherein the preset range is 0.39 to 0.

64. In the initial image region, the first human image region composed of continuous edge pixels is extracted; The step of extracting the background fusion matrix between the current real-time image and the initial image region includes: The current real-time image is converted to grayscale to obtain a first grayscale image; The initial image region is converted to grayscale to obtain a second grayscale image; Extract the first grayscale regions at the four diagonal positions in the first grayscale image; Extract the second grayscale regions at the four diagonal positions of the second grayscale image; wherein the first grayscale region and the second grayscale region are rectangular regions of a first preset size; The average value between the pixel matrices corresponding to the first grayscale regions and the pixel matrices corresponding to the second grayscale regions is used as the background fusion matrix.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps in the gimbal-based unmanned aerial vehicle pedestrian following method as described in any one of claims 1 to 6.

Images