A neural network-based visual tracking method and system for unmanned aerial vehicles

Abstract

The application provides a kind of unmanned aerial vehicle vision tracking method and system based on neural network, real-time collection dynamic image sequence containing target object motion trajectory information is carried out by unmanned aerial vehicle camera device, then the sequence is extracted multi-scale space-time feature, obtain the global motion feature of the three-dimensional space displacement change of the target and the local texture feature of the surface detail space-time continuity, the global motion feature and local texture feature are input into the pre-training neural network model fusion, generate real-time motion prediction parameter, adjust the flight attitude parameter of unmanned aerial vehicle accordingly, generate dynamic tracking instruction, control unmanned aerial vehicle and target keep preset relative motion relationship, the dynamic tracking instruction drives unmanned aerial vehicle to execute tracking task, simultaneously real-time update image sequence and feedback, form closed loop tracking control, realize efficient, stable and accurate vision tracking to target object.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for visual tracking of unmanned aerial vehicles (UAVs) based on neural networks. Background Technology

[0002] With the rapid development of technology, drones have been widely used in many fields. Among them, drone visual tracking technology has become one of the research hotspots, playing an important role in many scenarios such as security monitoring, logistics distribution, agricultural plant protection, and film and television shooting.

[0003] Early drone tracking technologies relied primarily on pre-set fixed flight paths or simple sensor feedback. For example, some traditional methods used GPS positioning systems to obtain the approximate location information of the target, and the drone followed a pre-programmed flight to track it. However, this approach has significant limitations. GPS positioning accuracy is limited, and signals are easily interfered with in complex environments (such as urban areas with tall buildings or dense forests), leading to a significant decrease in tracking accuracy and making it difficult to meet the needs for precise target tracking.

[0004] Later, vision-based drone tracking technology gradually emerged. Some existing technologies use simple image matching algorithms to determine the target position by comparing the similarity between the current image and an initial template image, and then adjust the drone's flight accordingly. However, this method is very sensitive to factors such as changes in lighting and target posture. When the target's appearance changes slightly or the ambient lighting conditions change, tracking failures are likely to occur. Moreover, such methods can only handle relatively simple target motion patterns and cannot effectively track complex three-dimensional spatial motions and rapidly changing target trajectories.

[0005] Furthermore, some tracking algorithms based on single features, such as those using only the target's color or edge features, are too one-sided in feature extraction and cannot fully reflect the target's motion characteristics. For example, relying solely on color features can lead to tracking errors or even target loss if the target's color is similar to the surrounding environment or changes under different lighting conditions; similarly, relying solely on edge features makes it difficult to handle situations where the target is partially occluded, resulting in unstable tracking. Summary of the Invention

[0006] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a neural network-based UAV visual tracking method, the method comprising:

[0007] The drone uses a camera device to capture a sequence of dynamic images of the target object in real time. The sequence of dynamic images contains the motion trajectory information of the target object at continuous moments.

[0008] Multi-scale spatiotemporal feature extraction is performed on the dynamic image sequence to obtain global motion features and local texture features of the target image sequence. The global motion features are used to characterize the displacement changes of the target object in three-dimensional space, and the local texture features are used to characterize the spatiotemporal continuity of the surface details of the target object.

[0009] The global motion features and the local texture features are input into a pre-trained neural network model for feature fusion to generate real-time motion prediction parameters for the target object;

[0010] Based on the real-time motion prediction parameters, the flight attitude parameters of the UAV are adjusted to generate corresponding dynamic tracking commands. The dynamic tracking commands are used to control the UAV to maintain a preset relative motion relationship with the target object.

[0011] The UAV is driven to perform visual tracking tasks according to the dynamic tracking command, while the dynamic image sequence is updated in real time and fed back to the multi-scale spatiotemporal feature extraction step to form a closed-loop tracking control.

[0012] In another aspect, embodiments of the present invention also provide a neural network-based UAV visual tracking system, including a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0013] Based on the above, this application embodiment uses a drone equipped with a camera device to collect dynamic image sequences containing the motion trajectory information of the target object in real time. On this basis, multi-scale spatiotemporal feature extraction is performed, simultaneously acquiring global motion features and local texture features. Global motion features accurately grasp the displacement changes of the target object in three-dimensional space, allowing the drone to understand the target's motion trend from a macroscopic perspective; while local texture features capture the spatiotemporal continuity of the target object's surface details, supplementing information at the microscopic level. Next, the global motion features and local texture features are input into a pre-trained neural network model for feature fusion and to generate real-time motion prediction parameters. Utilizing the powerful learning and analysis capabilities of the neural network, different types of features are effectively integrated to predict the real-time motion of the target object, greatly improving the accuracy and reliability of motion prediction. Based on the real-time motion prediction parameters, the drone's flight attitude parameters are adjusted and dynamic tracking commands are generated, ensuring that the drone can respond in real time to the target's motion and maintain a preset relative motion relationship with the target object. This allows the drone to no longer simply fly along a preset path but intelligently adapt to the target's dynamic changes, achieving highly flexible tracking. Finally, the UAV is driven to perform visual tracking tasks according to dynamic tracking commands, and the dynamic image sequence is updated in real time to form a closed-loop tracking control. This allows for continuous adjustment and optimization based on the target's new state, greatly improving the stability and robustness of the tracking. Even when the target's trajectory is complex and variable, the UAV can still continuously and accurately track the target, reducing the risk of tracking loss and improving the overall success rate of visual tracking tasks. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the execution flow of the UAV visual tracking method based on neural networks provided in an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of exemplary hardware and software components of a neural network-based UAV visual tracking system provided in an embodiment of the present invention. Detailed Implementation

[0016] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating a neural network-based UAV visual tracking method according to an embodiment of the present invention. The following is a detailed description of the neural network-based UAV visual tracking method.

[0017] Step S110: The camera device mounted on the drone collects a dynamic image sequence of the target object in real time. The dynamic image sequence contains the motion trajectory information of the target object at continuous moments.

[0018] In this embodiment, taking a rescue scenario as an example, such as a rescue operation after an earthquake in a mountainous area, the target is a survivor trapped in a landslide area. After the rescue drone takes off, its onboard camera has a set resolution and frame rate, for example, a resolution of 1920×1080 pixels and a frame rate of 30 frames per second. As the drone approaches the target area, the camera continuously captures images including the survivor. For example, at the initial time t1, the survivor is captured in a certain position in the image. As time progresses to consecutive times such as t2 and t3, the survivor may be constantly moving, such as trying to move to a safer area or being forced to change position due to further landslides. The changes in the survivor's position in the image at these different times constitute the motion trajectory information. The frames of images captured by the camera are arranged in chronological order to form a dynamic image sequence. This dynamic image sequence completely records the survivor's activities at the rescue site, including preliminary information about their direction of movement and speed.

[0019] Step S120: Multi-scale spatiotemporal feature extraction is performed on the dynamic image sequence to obtain global motion features and local texture features of the target image sequence. The global motion features are used to characterize the displacement changes of the target object in three-dimensional space, and the local texture features are used to characterize the spatiotemporal continuity of the surface details of the target object.

[0020] In this embodiment, continuing with the example of mountain rescue, after the rescue drone collects a sequence of dynamic images of survivors, multi-scale spatiotemporal feature extraction can be performed. For example, the dynamic image sequence can be decomposed into multiple consecutive time segments, assuming each time segment is 1 second (containing 30 frames). For each time segment, inter-frame alignment is performed on the image frames. During this process, some feature points in each frame need to be matched and corrected to eliminate the jitter caused by drone flight swaying or slight changes in shooting angle, resulting in a stable image sequence after jitter elimination. For example, when processing a 1-second time segment, feature point detection is performed on frames 1 to 30 to find feature points such as a marker on the survivor's clothing or a fixed rock in the surrounding environment. Alignment correction is then performed based on the positional changes of these feature points in different frames.

[0021] Next, the stable image sequence undergoes frame-by-frame grayscale normalization, mapping the pixel grayscale values ​​of each frame to a specific range, such as 0-255. Then, an optical flow field matrix is ​​constructed based on the pixel differences between adjacent frames. Each element in the optical flow field matrix represents the instantaneous motion vector of the target object (survivor) in the image plane. For example, by calculating the positional change of a pixel in adjacent frames, its velocity components in the x and y directions are obtained, thereby determining the motion vector of that pixel.

[0022] Then, a multi-resolution pyramid algorithm is used to perform hierarchical analysis of the optical flow field matrix, extracting histograms of motion direction distribution at different scales. Assuming the highest resolution layer of the Gaussian pyramid structure is the original optical flow field matrix with a resolution of 1920×1080, each lower layer resolution is generated through stepwise downsampling based on a Gaussian kernel (e.g., a 5×5 kernel size), with a downsampling ratio of 0.5. Starting from the highest resolution layer, the optical flow field matrix of each level is divided into uniformly distributed square grid cells, assuming each square grid cell is 10×10 pixels in size, and each square grid cell covers a set number of optical flow vectors. Within each square grid cell, the direction angles of all optical flow vectors are counted, and the direction angles are divided into multiple direction intervals according to a preset 10° interval, generating a histogram of the direction angle distribution for each square grid cell. For example, if there are 50 optical flow vectors within a square grid cell, after counting the direction angles of these vectors, the cell is divided into 10° intervals to obtain its direction angle distribution histogram. The orientation angle distribution histograms of all square grid cells within each level are summed across cells to obtain the motion orientation distribution histogram for the current level. Maximum value normalization is performed on the motion orientation distribution histogram for each level. The normalized histograms are then concatenated as one-dimensional vectors in descending order of resolution to form a cross-scale fused orientation distribution feature vector. Finally, principal component analysis is performed on the orientation distribution feature vector for dimensionality reduction. The dimensionality-reduced orientation distribution feature vector is then fused with the average optical flow amplitude of each level of the Gaussian pyramid structure using a layer-by-layer weighted method to generate the final vector representation of the global motion features. This global motion feature accurately characterizes the survivor's displacement changes in three-dimensional space, such as whether they are moving to the upper left or lower right, and the magnitude of their movement.

[0023] Simultaneously, a local region centered on the target object (survivor) is extracted from the stable image sequence, assuming the extracted local region is 300×300 pixels in size. This local region is then subjected to spatial domain convolutional kernel filtering, with a kernel size of 3×3, to extract a multi-channel texture response map. The multi-channel texture response map is then aggregated along the time axis using a sliding window of 10 frames. The mean and variance of the response of each channel within the sliding window are calculated to generate local texture features. These local texture features can characterize the spatiotemporal continuity of surface details on the survivor, such as the changes in the texture and color of the survivor's clothing at different times, which helps to more accurately identify and track targets in complex environments.

[0024] Step S130: Input the global motion features and the local texture features into a pre-trained neural network model for feature fusion to generate real-time motion prediction parameters for the target object.

[0025] In this embodiment, the previously obtained global motion features and local texture features can be input into a pre-trained neural network model. This neural network model may be pre-trained using a large amount of rescue scenario data, such as image data of rescue scenarios collected in multiple different mountainous rescue simulation scenarios. The global motion features and local texture features are respectively mapped to the high-dimensional latent space of the pre-trained neural network model, assuming the high-dimensional latent space has 1024 dimensions. After mapping, the global motion features yield the corresponding first latent vector, and the local texture features yield the second latent vector.

[0026] A cross-attention mechanism is used to calculate the association weight matrix between the first and second latent vectors. This mechanism combines the relationships between global motion features and local texture features; for example, a survivor's movement direction might be related to the relative positional change of a distinctive texture on their clothing. The second latent vector is then reconstructed using the association weight matrix to obtain the enhanced third latent vector. Finally, the first and third latent vectors are added element-wise to obtain the fused fourth latent vector.

[0027] Then, the fourth hidden vector is input into a multilayer perceptron for nonlinear transformation. The first hidden layer of the multilayer perceptron has 512 neurons. The fourth hidden vector is input into the first hidden layer for linear transformation to obtain the first intermediate feature vector. A nonlinear activation function, such as ReLU, is applied to the first intermediate feature vector to generate the activated first nonlinear feature vector. The activated first nonlinear feature vector is then input into the second hidden layer of the multilayer perceptron, which has 256 neurons. A linear transformation is performed to obtain the second intermediate feature vector, which is then processed by a nonlinear activation function to generate the activated second nonlinear feature vector. The activated second nonlinear feature vector is then input into the output layer of the multilayer perceptron for linear transformation to generate the unnormalized original prediction vector.

[0028] The unnormalized original prediction vector is segmented into position and velocity sub-vectors. The position sub-vector contains the original data of the three-dimensional spatial coordinates, and the velocity sub-vector contains the original data of the three-dimensional velocity vector. The position sub-vector is normalized and scaled to match its numerical range with the UAV camera's field-of-view coordinate system. For example, the x-coordinate range from -1 to 1 is mapped to the coordinate range corresponding to the left and right boundaries of the camera's field of view. The velocity sub-vector is also normalized and scaled to match its numerical range with the UAV's maximum flight speed threshold. For example, if the UAV's maximum horizontal flight speed is 20 m / s, the velocity values ​​in the velocity sub-vector are mapped to the range of 0 to 20. The normalized position and velocity vectors are concatenated to form the final prediction vector. The first three elements are extracted from the final prediction vector as the survivor's predicted position coordinates at the next moment, and the last three elements are extracted as the survivor's predicted velocity vector at the next moment. These predicted position coordinates and velocity vectors will provide crucial information for subsequent adjustments to the UAV's flight attitude.

[0029] Step S140: Adjust the flight attitude parameters of the UAV based on the real-time motion prediction parameters, and generate a corresponding dynamic tracking command. The dynamic tracking command is used to control the UAV to maintain a preset relative motion relationship with the target object.

[0030] In this embodiment, based on the previously generated real-time motion prediction parameters, the rescue drone begins to adjust its flight attitude parameters. For example, it first acquires the drone's current flight status data, assuming the drone's current pitch angle is 10°, yaw angle is 30°, roll angle is 5°, altitude is 500 meters, and horizontal speed is 10 meters per second. Then, based on the Euclidean distance between the predicted position coordinates and the drone's current position, it calculates the heading deviation angle in the horizontal plane and the vertical altitude compensation. For example, if the predicted survivor's position is deviated to the left by a set distance relative to the drone's current position in the horizontal direction, calculations show that the yaw angle needs to be increased by 15° to align with the survivor's new position, and the altitude needs to be decreased by 50 meters in the vertical direction to maintain a suitable shooting angle, thus obtaining the vertical altitude compensation.

[0031] Next, based on the vector difference between the velocity vector and the UAV's current horizontal velocity, the torque adjustment coefficient of the propulsion motor and the propeller speed increment are calculated. Assuming the survivor is predicted to move left at 5 m / s, while the UAV's current horizontal velocity is 10 m / s and not in the same direction, the horizontal velocity error vector is calculated. The magnitude of this error vector is multiplied by a preset propulsion response coefficient (e.g., 0.5) to obtain the propulsion motor torque adjustment coefficient. Simultaneously, if the survivor is predicted to have an upward or downward velocity in the vertical direction, a scalar subtraction is performed with the UAV's current vertical velocity component to obtain a vertical velocity error scalar. This error scalar is multiplied by the propeller lift conversion coefficient (e.g., 1.2) to obtain the propeller speed increment.

[0032] The adjusted yaw angle target setpoint (45°), altitude target setpoint (450 meters), torque adjustment coefficient, and propeller speed increment are packaged into a data packet format for dynamic tracking commands, generating corresponding dynamic tracking commands. These commands guide the UAV's flight control system to perform corresponding actions to maintain the preset relative motion relationship with the survivor, ensuring continuous tracking of the survivor's position and acquisition of clear image information during the rescue process.

[0033] Step S150: Drive the UAV to perform a visual tracking task according to the dynamic tracking command, while updating the dynamic image sequence in real time and feeding it back to the multi-scale spatiotemporal feature extraction step to form a closed-loop tracking control.

[0034] In detail, after receiving the dynamic tracking command, the rescue drone transmits the command to the flight control system via a wireless communication link. Based on the dynamic tracking command, the flight control system can synchronously adjust the pitch angle, yaw angle, roll angle, altitude, and propeller speed. During the adjustment process, the drone continuously captures updated dynamic image sequences through its camera system. For example, when the drone adjusts its yaw angle and altitude according to the command, the camera system still captures images containing survivors at a frame rate of 30 frames per second.

[0035] In each updated frame of the dynamic image, a search area can be expanded by a preset proportion (e.g., 20%) centered on the predicted location coordinates. Assuming the predicted location coordinates are pixels (500, 300) in the image, the expanded search area would be a rectangular region centered at (500, 300). A color histogram backprojection is performed on the pixels within the search area to generate a target probability distribution map. The connected region with the highest probability density is located in the target probability distribution map; this connected region is likely the location of the survivor. The minimum bounding rectangle of this connected region is then calculated. The position parameters of the latest bounding box are determined based on the vertex coordinates of the minimum bounding rectangle. These position parameters are then smoothed using a Kalman filter compared to the position parameters of the historical bounding boxes to eliminate jitter noise caused by image noise or minor movements of the survivor.

[0036] The consistency between the latest bounding box position and the predicted position coordinates is checked. The pixel distance between the center point coordinates of the latest bounding box and the predicted position coordinates is calculated, assuming this pixel distance is 50 pixels. Perspective projection inversion is performed using the current altitude of the drone (500 meters) and the focal length parameters of the camera device (e.g., 50 mm) to convert the pixel distance into the actual physical distance. If this actual physical distance is greater than a preset threshold (e.g., 10 meters), it is determined that the survivor may have experienced violent movement or been obstructed by a new landslide, and the execution of the current tracking command is paused. At this time, the historical feature data cached in the neural network model is cleared, and the multi-scale spatiotemporal feature extraction step is re-executed from the current frame. During the re-execution, the position parameters of the latest bounding box are used as the initial input first, skipping the dependency on historical feature data.

[0037] The cropping range of the local region is updated based on the latest verified bounding box position. Centered on the latest verified bounding box position, the cropping range is expanded by a preset scaling factor (e.g., 1.5) to form a new local region. Gaussian pyramid downsampling is performed on the new local region to generate multi-resolution image patches. The histogram of oriented gradients (HARG) algorithm is applied to each of the multi-resolution image patches to extract spatial gradient distribution features at different resolutions. These spatial gradient distribution features at different resolutions are then weighted and superimposed to generate updated local texture features. Simultaneously, the optical flow field matrix is ​​recalculated within the expanded cropping range, and the histogram of oriented gradients for the global motion features is updated.

[0038] The updated global motion features and local texture features are input into the neural network model to iteratively generate new real-time motion prediction parameters. During each iteration, the changing trend of the actual physical distance is recorded. If the rate of decrease of the actual physical distance is less than a convergence threshold (e.g., 0.5 meters / iteration) for a preset number of consecutive iterations (e.g., 5), the system is considered to have entered a steady-state tracking phase. In this steady-state, the sampling frequency of the camera is reduced, for example, from 30 frames / second to 15 frames / second, and the computational layers for multi-scale spatiotemporal feature extraction are reduced to minimize computational resource consumption. When the actual physical distance remains within a preset threshold for a predetermined duration (e.g., 10 seconds), a tracking completion flag is activated, and feature update iterations cease based on the flag, maintaining the current flight attitude parameters until a new visual tracking task is received. In this way, the entire system forms a closed-loop tracking control, enabling continuous and stable tracking of the target object (survivor) in complex rescue scenarios.

[0039] Based on the above steps, this embodiment of the application uses a drone equipped with a camera device to collect dynamic image sequences containing the motion trajectory information of the target object in real time. On this basis, multi-scale spatiotemporal feature extraction is performed, simultaneously acquiring global motion features and local texture features. Global motion features accurately grasp the displacement changes of the target object in three-dimensional space, allowing the drone to understand the target's motion trend from a macroscopic perspective; while local texture features capture the spatiotemporal continuity of the target object's surface details, supplementing information at the microscopic level. Next, the global motion features and local texture features are input into a pre-trained neural network model for feature fusion and to generate real-time motion prediction parameters. Utilizing the powerful learning and analysis capabilities of the neural network, different types of features are effectively integrated to predict the real-time motion of the target object, greatly improving the accuracy and reliability of motion prediction. Based on the real-time motion prediction parameters, the drone's flight attitude parameters are adjusted and dynamic tracking commands are generated, ensuring that the drone can respond in real time to the target's motion and maintain a preset relative motion relationship with the target object. This allows the drone to no longer simply fly along a preset path but to intelligently adapt to the target's dynamic changes, achieving highly flexible tracking. Finally, the UAV is driven to perform visual tracking tasks according to dynamic tracking commands, and the dynamic image sequence is updated in real time to form a closed-loop tracking control. This allows for continuous adjustment and optimization based on the target's new state, greatly improving the stability and robustness of the tracking. Even when the target's trajectory is complex and variable, the UAV can still continuously and accurately track the target, reducing the risk of tracking loss and improving the overall success rate of visual tracking tasks.

[0040] In one possible implementation, step S120 includes:

[0041] Step S121: Decompose the dynamic image sequence into multiple consecutive time segments, and perform inter-frame alignment processing on the image frames in each time segment to obtain a stable image sequence after eliminating jitter.

[0042] For example, each time segment can be set to 5 seconds. Since the camera's frame rate is 30 frames per second, each time segment contains 150 frames. These 150 frames can be aligned between frames using any image registration algorithm from related technologies. This image registration algorithm can detect significant feature points in each frame, such as unique patterns on a survivor's clothing or the outlines of fixed rocks in the surrounding environment, as reference points. For example, after determining the positions of these reference points in the first frame, the corresponding positions of these reference points can be found in subsequent frames. Then, based on the deviations of these corresponding positions, the images can be transformed by translation, rotation, etc., thereby eliminating image jitter caused by slight shaking or airflow during drone flight and obtaining a stable image sequence.

[0043] Step S122: Perform frame-by-frame grayscale normalization processing on the stable image sequence, and construct an optical flow field matrix based on the pixel differences between adjacent frames. The optical flow field matrix is ​​used to quantize the instantaneous motion vector of the target object in the image plane.

[0044] In detail, frame-by-frame grayscale normalization adjusts the pixel grayscale values ​​of each frame of the image to a uniform range. For example, the arbitrary distribution of pixel grayscale values ​​in each frame, originally between 0 and 255, is normalized to between 0 and 1 through linear transformations or other methods. Then, an optical flow field matrix is ​​constructed based on the pixel differences between adjacent frames. Taking a certain pixel as an example, the positional changes of that pixel in the horizontal and vertical directions are calculated in two adjacent frames. Assuming that the coordinates of a pixel in the nth frame are (x1, y1), and the coordinates of that pixel in the (n+1)th frame become (x2, y2), then the motion component in the horizontal direction is x2-x1, and the motion component in the vertical direction is y2-y1. Performing this calculation for each pixel in each frame of the image constructs the optical flow field matrix, which is used to quantify the instantaneous motion vector of the survivor in the image plane.

[0045] Step S123: The optical flow field matrix is ​​analyzed hierarchically using the multi-resolution pyramid algorithm to extract motion direction distribution histograms at different scales, and the motion direction distribution histogram features of each layer are concatenated to form the global motion features.

[0046] Step S123 includes:

[0047] Step S1231: Perform a Gaussian pyramid generation operation on the optical flow field matrix to obtain a Gaussian pyramid structure containing multiple resolution levels, wherein the highest resolution layer corresponds to the original optical flow field matrix, and each lower resolution layer is generated by step-by-step downsampling based on Gaussian kernel.

[0048] Step S1232: Starting from the highest resolution layer of the Gaussian pyramid structure, the optical flow field matrix of each layer is divided into uniformly distributed square grid cells, and each square grid cell covers a preset number of optical flow vectors.

[0049] In this embodiment, the highest resolution layer corresponds to the original optical flow field matrix, with a resolution of 1920×1080 (consistent with the image resolution acquired by the camera device). Each lower layer resolution is generated through stepwise downsampling based on a Gaussian kernel. For example, the Gaussian kernel size is set to 5×5, and the downsampling ratio is 0.5. Starting from the highest resolution layer, the optical flow field matrix of each layer is divided into uniformly distributed square grid cells. Assuming each square grid cell is 10×10 pixels in size, each square grid cell covers a set number of optical flow vectors. Taking the highest resolution layer as an example, the optical flow field matrix of this layer is 1920×1080 pixels. Therefore, 192 square grid cells can be divided horizontally (1920÷10=192) and 108 square grid cells can be divided vertically (1080÷10=108), for a total of 192×108 square grid cells.

[0050] Step S1233: Calculate the direction angles of all optical flow vectors in each square grid cell, and divide the direction angles into multiple direction intervals according to the preset angle intervals to generate a histogram of the direction angle distribution of each square grid cell.

[0051] For example, setting a preset angle range of 10°, for an optical flow vector within a square grid cell, the angle between each optical flow vector and the horizontal direction is calculated as the orientation angle. Assuming there are 30 optical flow vectors within a square grid cell, their orientation angles are calculated and then divided into 10° intervals. If there are 5 optical flow vectors with orientation angles between 0° and 10°, 8 between 10° and 20°, and so on, a histogram of the orientation angle distribution for that square grid cell is generated.

[0052] Step S1234: Accumulate the direction angle distribution histograms of all square grid cells in each level across cells to obtain the motion direction distribution histogram of the current level.

[0053] For example, in a certain level with 100 square grid cells, a histogram of orientation angle distribution is obtained for each square grid cell. Then, the values ​​of these 100 orientation angle distribution histograms for corresponding intervals are summed. For example, in the interval from 0° to 10°, the value of the first square grid cell is 5, the second is 3, and so on. Summing up these 100 values ​​gives the summed value for that level in the 0° to 10° interval. Performing the above operation on all intervals yields the motion direction distribution histogram for the current level.

[0054] Step S1235: Perform maximum value normalization on the motion direction distribution histogram of each level, and then concatenate the normalized motion direction distribution histograms of each level into one-dimensional vectors in order of resolution from high to low to form a cross-scale fused direction distribution feature vector.

[0055] For example, maximum value normalization involves dividing the value of each interval by the maximum value in the motion direction distribution histogram of that level, ensuring that all interval values ​​are between 0 and 1. For instance, if the value of the motion direction distribution histogram of a certain level in the 0° to 10° interval is 20, and the maximum value for that level is 50, then after normalization, the value for that interval is 20 ÷ 50 = 0.4. The normalized motion direction distribution histograms of each level are then concatenated as one-dimensional vectors in descending order of resolution. Assuming the highest resolution level's motion direction distribution histogram, after normalization, yields a vector of length 10, and the next resolution level yields a vector of length 8, concatenating these vectors sequentially results in a cross-scale fused direction distribution feature vector of length 18.

[0056] Step S1236: Perform principal component analysis dimensionality reduction on the directional distribution feature vector, and then perform layer-by-layer weighted fusion of the dimensionality-reduced directional distribution feature vector with the average optical flow amplitude of each level of the Gaussian pyramid structure to generate the final vector representation of the global motion features.

[0057] In detail, principal component analysis (PCA) dimensionality reduction involves calculating the covariance matrix of the directional distribution eigenvectors, then determining their eigenvalues ​​and eigenvectors, and selecting the top few eigenvectors to reduce the dimensionality of the directional distribution eigenvectors. Assume the length of the directional distribution eigenvectors after dimensionality reduction is 8. For the average optical flow amplitude of each level of the Gaussian pyramid structure, the average amplitude of all optical flow vectors in the optical flow field matrix of each level is calculated. For example, the optical flow field matrix of the highest resolution level contains 192×108×10×10 optical flow vectors (the total number of optical flow vectors is calculated based on the previously divided square grid cells). The amplitudes of these optical flow vectors are summed and divided by the total number of optical flow vectors to obtain the average optical flow amplitude of that level. Then, the dimensionality-reduced directional distribution eigenvectors and the average optical flow amplitudes of each level are weighted and fused according to set weights. For example, the weight of the dimensionality-reduced directional distribution eigenvectors is 0.6, and the weight of the average optical flow amplitudes of each level is 0.4. Multiplying them element-wise and summing them yields the final vector representation of the global motion characteristics.

[0058] Step S124: Extract a local region centered on the target object from the stable image sequence, and perform spatial convolution kernel filtering on the local region to extract a multi-channel texture response map.

[0059] In detail, assuming the extracted local region is 300×300 pixels in size, this local region is centered on the survivor's position in the image. Spatial domain convolutional kernel filtering uses a 3×3 convolutional kernel. For each pixel within the local region, the nine pixel values ​​covered by the kernel are multiplied by the corresponding weights of the kernel, and then summed to obtain the filtered pixel value. For example, for the pixel in the upper left corner of the local region, the nine pixel values ​​covered by the kernel are p1, p2, p3, p4, p5, p6, p7, p8, and p9, and the kernel weights are w1, w2, w3, w4, w5, w6, w7, w8, and w9, respectively. Therefore, the filtered pixel value is p1×w1 + p2×w2 + p3×w3 + p4×w4 + p5×w5 + p6×w6 + p7×w7 + p8×w8 + p9×w9. By performing this operation on each pixel within a local region, a filtered image is obtained, which contains multiple channels (e.g., three RGB channels), thereby extracting a multi-channel texture response map.

[0060] Step S125: Aggregate the multi-channel texture response map along the time axis using a sliding window, and calculate the mean and variance of the response of each channel within the sliding window to generate the local texture features.

[0061] For example, let's set the sliding window size to 10 frames, using frames 1 to 10 of the multi-channel texture response map as the first sliding window. For each channel (taking the R channel as an example), calculate the mean of the R channel value for the corresponding pixel position in these 10 frames. Assume that at a certain pixel position, the R channel value is r1 in frame 1, r2 in frame 2, and so on, up to r10 in frame 10. Then, the mean of the R channel value for that pixel position within the sliding window is (r1 + r2 + ... + r10) ÷ 10. Similarly, calculate the variance of the R channel value for that pixel position within the sliding window. The variance is calculated by first squared the difference between each value and the mean, such as (r1 - mean)^2, (r2 - mean)^2, etc., and then summing these squared values ​​and dividing by 10. By performing the above mean and variance calculations for each channel, we can obtain the mean and variance of the response of each channel within the sliding window. Combining these values ​​generates local texture features.

[0062] In one possible implementation, step S130 includes:

[0063] Step S131: Map the global motion features and the local texture features to the high-dimensional latent space of the pre-trained neural network model to obtain the corresponding first latent vector and second latent vector.

[0064] In this embodiment, it is assumed that the high-dimensional latent space of the pre-trained neural network model is set to 512 dimensions. For global motion features, which contain multiple elements, these elements are mapped to the 512-dimensional high-dimensional latent space using a trained mapping function. This mapping process is performed according to the mapping rules determined during the pre-training of the neural network model, resulting in the first latent vector. Similarly, for local texture features, they are also mapped to the 512-dimensional high-dimensional latent space according to the same mapping rules, thus obtaining the second latent vector.

[0065] Step S132: Calculate the association weight matrix between the first latent vector and the second latent vector using the cross-attention mechanism, and reconstruct the second latent vector by weighting it according to the association weight matrix to obtain the enhanced third latent vector.

[0066] In detail, the cross-attention mechanism comprehensively considers the relationship between each element in the first and second hidden vectors. Taking a certain element in the first hidden vector as an example, it can be calculated with all elements in the second hidden vector, and the calculation process may involve operations such as dot products. Assuming the first element of the first hidden vector is a1, and the second hidden vector has 512 elements b1, b2, ..., b512, the relationship values ​​between a1 and b1, a1 and b2, etc., are calculated. After a series of calculations (based on the cross-attention calculation method predefined by the neural network model), the association weight matrix between the first and second hidden vectors is obtained. Each element in this association weight matrix represents the degree of association between elements in the first and second hidden vectors. Then, the second hidden vector is reconstructed with weights based on this association weight matrix, that is, each element in the association weight matrix is ​​multiplied with the corresponding element in the second hidden vector, and the results of the multiplications are added together to obtain the enhanced third hidden vector. For example, if an element in the association weight matrix is ​​w1 and the corresponding element in the second latent vector is b1, then the reconstructed element is w1×b1. Performing this operation on all elements in the second latent vector yields the enhanced third latent vector.

[0067] Step S133: Add the first hidden vector and the third hidden vector element by element to obtain the fused fourth hidden vector.

[0068] Element-wise addition means adding the first element of the first hidden vector to the first element of the third hidden vector, the second element of the first hidden vector to the second element of the third hidden vector, and so on. For example, if the first hidden vector is [a1, a2, a3, ...] and the third hidden vector is [b1, b2, b3, ...], then the merged fourth hidden vector is [a1+b1, a2+b2, a3+b3, ...].

[0069] Step S134: Input the fourth hidden vector into the multilayer perceptron for nonlinear transformation, and output the predicted position coordinates and velocity vector of the target object at the next moment.

[0070] For example, step S134 includes:

[0071] Step S1341: Input the fourth hidden vector into the first hidden layer of the multilayer perceptron and perform a linear transformation to obtain the first intermediate feature vector.

[0072] In this embodiment, the first hidden layer of the multilayer perceptron has a specific number of neurons, assumed to be 256 neurons. Each element in the fourth hidden vector has a predefined connection weight with a neuron in the first hidden layer; these weights are determined during the neural network pre-training phase. For the first element of the fourth hidden vector, it is multiplied by the connection weight of each neuron in the first hidden layer, and these products are then accumulated on the corresponding neurons. For example, if the fourth hidden vector is [e1, e2, e3, ...], and the connection weights between the first neuron in the first hidden layer and the elements of the fourth hidden vector are [w11, w12, w13, ...], then the accumulated result on the first neuron is e1×w11+e2×w12+e3×w13+..., which is the first element of the first intermediate feature vector. The other elements of the first intermediate feature vector are calculated in the same way to obtain the first intermediate feature vector.

[0073] Step S1342: Perform nonlinear activation function processing on the first intermediate feature vector to generate the activated first nonlinear feature vector.

[0074] For example, the ReLU function is used as the nonlinear activation function. For each element of the first intermediate feature vector, if the value of the element is less than 0, it is set to 0; if the value of the element is greater than or equal to 0, it remains unchanged. For example, if the first intermediate feature vector is [f1, f2, f3, ...], if f1 is less than 0, the first element of the activated first nonlinear feature vector is 0; if f2 is greater than or equal to 0, the second element of the activated first nonlinear feature vector is f2, and so on, thus obtaining the activated first nonlinear feature vector.

[0075] Step S1343: The activated first nonlinear feature vector is input into the second hidden layer of the multilayer perceptron for linear transformation to obtain the second intermediate feature vector.

[0076] For example, the second hidden layer also has a predefined number of neurons, let's say 128. Similar to the calculation method of the first hidden layer, each element of the activated first nonlinear feature vector is multiplied by the connection weights of the neurons in the second hidden layer, and then accumulated across the neurons to obtain the elements of the second intermediate feature vector. For example, if the activated first nonlinear feature vector is [g1, g2, g3, ...], and the connection weights between the first neuron of the second hidden layer and these vector elements are [w21, w22, w23, ...], then the first element of the second intermediate feature vector is g1×w21+g2×w22+g3×w23+..., and all elements of the second intermediate feature vector are calculated in this way.

[0077] Step S1344: Perform nonlinear activation function processing on the second intermediate feature vector to generate the activated second nonlinear feature vector.

[0078] For example, using the ReLU function, for each element of the second intermediate feature vector, if it is less than 0, it becomes 0; if it is greater than or equal to 0, it remains unchanged. For example, if the second intermediate feature vector is [h1, h2, h3, ...], if h1 is less than 0, the first element of the activated second nonlinear feature vector is 0; if h2 is greater than or equal to 0, the second element of the activated second nonlinear feature vector is h2, and so on, to obtain the activated second nonlinear feature vector.

[0079] Step S1345: The activated second nonlinear feature vector is input to the output layer of the multilayer perceptron for linear transformation to generate an unnormalized original prediction vector.

[0080] For example, the output layer has a specific number of neurons, say 6, corresponding to the three dimensions of the predicted position coordinates (x, y, z) and the three dimensions of the velocity vector (vx, vy, vz). Each element of the activated second nonlinear feature vector is multiplied by the connection weights of the output layer neurons and summed to obtain the elements of the unnormalized original prediction vector. For example, if the activated second nonlinear feature vector is [i1, i2, i3, ...], the connection weights between the first neuron of the output layer and these vector elements are [w31, w32, w33, ...], and the first element of the unnormalized original prediction vector is i1×w31+i2×w32+i3×w33+..., all elements of the unnormalized original prediction vector are obtained in this way.

[0081] Step S1346: The unnormalized original prediction vector is divided into position component vectors and velocity component vectors, wherein the position component vectors contain the original data of three-dimensional spatial coordinates, and the velocity component vectors contain the original data of three-dimensional velocity vectors.

[0082] For example, if the unnormalized original prediction vector is [j1, j2, j3, j4, j5, j6], then the position component sub-vector is [j1, j2, j3] and the velocity component sub-vector is [j4, j5, j6].

[0083] Step S1347: Perform normalization scaling on the position component subvector to match its numerical range with the field of view coordinate system of the UAV camera device, and generate a normalized position coordinate vector.

[0084] Assume that in the UAV camera's field-of-view coordinate system, the x-coordinate ranges from -1 to 1, the y-coordinate ranges from -1 to 1, and the z-coordinate ranges from -1 to 1. For the original x-coordinate value in the position component sub-vector, let's say x0, first find the minimum value min_x and the maximum value max_x of the x-coordinate in the position component sub-vector. Then, normalize it by calculating (x0 - min_x) ÷ (max_x - min_x) × 2 - 1. For example, if the minimum value of the x-coordinate in the position component sub-vector is -5, the maximum value is 5, and the original value x0 is 0, then the normalized x-coordinate is (0 - (-5)) ÷ (5 - (-5)) × 2 - 1 = 0. Normalize the y and z coordinates in the same way to obtain the normalized position coordinate vector.

[0085] Step S1348: Perform normalization scaling on the velocity component vector to make its numerical range match the maximum flight speed threshold of the UAV, and generate a normalized velocity vector.

[0086] Assume the drone's maximum horizontal speed is 20 m / s and its vertical speed is 10 m / s. For the original horizontal velocity value in the velocity component vector, let's say vx0, first find the maximum horizontal velocity value max_vx in the velocity component vector. Then, normalize it by calculating vx0 ÷ max_vx × 20. For example, if the maximum horizontal velocity value in the velocity component vector is 50 m / s and the original value vx0 is 25 m / s, then the normalized horizontal velocity is 25 ÷ 50 × 20 = 10 m / s. The vertical velocity is normalized in a similar way to obtain the normalized velocity vector.

[0087] Step S1349: The normalized position coordinate vector and velocity vector are concatenated to form the final prediction vector.

[0088] Step S13410: Extract the first three elements from the final prediction vector as the predicted position coordinates of the target object at the next moment, and extract the last three elements as the predicted velocity vector of the target object at the next moment.

[0089] For example, if the normalized position coordinate vector is [k1, k2, k3] and the normalized velocity vector is [k4, k5, k6], then the final prediction vector is [k1, k2, k3, k4, k5, k6]. The first three elements are extracted from the final prediction vector as the survivor's predicted position coordinates at the next moment, and the last three elements are extracted as the survivor's predicted velocity vector at the next moment. Assuming the final prediction vector is [l1, l2, l3, l4, l5, l6], then the predicted position coordinates are (l1, l2, l3), and the predicted velocity vector is (l4, l5, l6).

[0090] Step S135: Calculate the required heading angle adjustment, flight altitude correction, and thrust parameters for the UAV based on the predicted position coordinates and velocity vector, and generate the real-time motion prediction parameters.

[0091] For example, step S135 includes:

[0092] Step S1351: Obtain the current three-dimensional spatial coordinates and current velocity vector of the UAV, wherein the three-dimensional spatial coordinates include longitude, latitude and altitude components.

[0093] For example, suppose the current three-dimensional spatial coordinates of the drone are (x1, y1, z1), where x1 is longitude, y1 is latitude, z1 is altitude component, and the current velocity vector is (vx1, vy1, vz1).

[0094] Step S1352: Subtract the longitude component of the predicted position coordinates from the current longitude component of the UAV to obtain the first lateral displacement deviation, and subtract the latitude component of the predicted position coordinates from the current latitude component of the UAV to obtain the second lateral displacement deviation.

[0095] For example, assuming the predicted position coordinates are (x2, y2, z2), the first lateral displacement deviation is x2-x1. The second lateral displacement deviation, y2-y1, is obtained by subtracting the latitude component of the predicted position coordinates from the current latitude component of the UAV.

[0096] Step S1353: Calculate the comprehensive displacement vector in the horizontal plane based on the first lateral displacement deviation and the second lateral displacement deviation, and subtract the direction angle of the comprehensive displacement vector from the current yaw angle of the UAV to obtain the initial heading deviation angle.

[0097] For example, first calculate the magnitude of the overall displacement deviation. According to the Pythagorean theorem, the magnitude of the overall displacement deviation is sqrt((x2-x1)^2+(y2-y1)^2). For example, if x2-x1=3 and y2-y1=4, then the magnitude of the overall displacement deviation is sqrt(3^2+4^2)=5. Then calculate the direction angle of the overall displacement vector using trigonometric functions. Assuming the direction angle is θ, tanθ=(y2-y1)÷(x2-x1), and obtain the value of θ using the arctangent function. If x2-x1=3 and y2-y1=4, then tanθ=4÷3, and the value of θ obtained using the arctangent function is approximately 53.13°. Subtract the direction angle of the overall displacement vector from the current yaw angle of the UAV to obtain the initial heading deviation angle. Assuming the current yaw angle of the UAV is α, the initial heading deviation angle is θ-α.

[0098] Step S1354: Subtract the altitude component of the predicted position coordinates from the current altitude component of the UAV to obtain the vertical altitude displacement deviation, and divide the altitude displacement deviation by a preset unit time step to generate the flight altitude correction.

[0099] In detail, the altitude component of the predicted position coordinates is subtracted from the current altitude component of the UAV to obtain the vertical altitude displacement deviation, i.e., z2-z1. This altitude displacement deviation is then divided by a preset unit time step to generate the flight altitude correction. Assuming the preset unit time step is 1 second, if z2-z1 = 10 meters, then the flight altitude correction is 10 ÷ 1 = 10 meters.

[0100] Step S1355: Subtract the horizontal component of the velocity vector from the horizontal component of the current velocity vector of the UAV to obtain the horizontal velocity error vector, and multiply the magnitude of the horizontal velocity error vector by a preset thrust response coefficient to generate the torque adjustment coefficient of the propulsion motor.

[0101] For example, suppose the horizontal component of the velocity vector is (vx2, vy2), and the horizontal velocity error vector is (vx2-vx1, vy2-vy1). For example, if vx2=15 m / s, vx1=10 m / s, vy2=5 m / s, vy1=3 m / s, then the horizontal velocity error vector is (15-10, 5-3)=(5, 2). Multiply the magnitude of the horizontal velocity error vector by the preset propulsion response coefficient to generate the torque adjustment coefficient of the propulsion motor. First, calculate the magnitude of the horizontal velocity error vector. According to the Pythagorean theorem, the magnitude of the horizontal velocity error vector is sqrt((vx2-vx1)^2+(vy2-vy1)^2). If (vx2-vx1)=5, (vy2-vy1)=2, then the magnitude of the horizontal velocity error vector is sqrt(5^2+2^2)=sqrt(29). Assuming the preset propulsion response coefficient is 0.5, then the torque adjustment coefficient of the propulsion motor is sqrt(29)×0.5.

[0102] Step S1356: Subtract the vertical component of the velocity vector from the vertical component of the current velocity vector of the UAV to obtain a vertical velocity error scalar, and multiply the vertical velocity error scalar with the propeller lift conversion coefficient to generate the propeller speed increment.

[0103] For example, the vertical component of the velocity vector can be subtracted from the vertical component of the current velocity vector of the UAV to obtain the vertical velocity error scalar, i.e., vz2 - vz1. This vertical velocity error scalar is then multiplied by the propeller lift conversion factor to generate the propeller speed increment. Assuming vz2 = 8 m / s, vz1 = 6 m / s, and the propeller lift conversion factor is 1.2, then the vertical velocity error scalar is 8 - 6 = 2 m / s, and the propeller speed increment is 2 × 1.2 = 2.4.

[0104] Step S1357: Input the initial heading deviation angle into the angle smoothing filter for noise suppression to obtain the smoothed heading angle adjustment amount.

[0105] For example, the angle smoothing filter processes the initial heading deviation angle according to a set algorithm (such as a weighted average algorithm). Assuming the initial heading deviation angle is β, the angle smoothing filter calculates a weighted average based on the heading deviation angle values ​​from several previous moments. For instance, if the heading deviation angles at the previous three moments are β-1, β-2, and β-3, with corresponding weights w1, w2, and w3, then the smoothed heading angle adjustment is (β-1×w1 + β-2×w2 + β-3×w3) ÷ (w1 + w2 + w3).

[0106] Step S1358: The flight altitude correction, torque adjustment coefficient and propeller speed increment are respectively subjected to interval truncation processing with the corresponding dynamic threshold to generate the limited flight altitude correction, torque adjustment coefficient and propeller speed increment.

[0107] For example, assuming the dynamic threshold for flight altitude correction is -5 to 5 meters, if the calculated flight altitude correction is -8 meters, it is truncated to -5 meters; if the calculated flight altitude correction is 8 meters, it is truncated to 5 meters. The torque adjustment coefficient and propeller speed increment are handled similarly. Assuming the dynamic threshold for torque adjustment coefficient is 0 to 1, if the calculated torque adjustment coefficient is 1.5, it is truncated to 1; assuming the dynamic threshold for propeller speed increment is 0 to 3, if the calculated propeller speed increment is 4, it is truncated to 3.

[0108] Step S1359: The smoothed heading angle adjustment, the limited flight altitude correction, the torque adjustment coefficient, and the propeller speed increment are encapsulated into a structured parameter set according to a preset coding rule to generate real-time motion prediction parameters containing multi-dimensional adjustment commands.

[0109] For example, a preset encoding rule specifies that these parameters are arranged in a specific order and some check bits are added to generate real-time motion prediction parameters, which will be used to adjust the drone's flight attitude to track survivors.

[0110] In one possible implementation, step S140 includes:

[0111] Step S141: Obtain the current flight status data of the UAV, which includes pitch angle, yaw angle, roll angle, altitude and horizontal speed.

[0112] For example, suppose the drone's current pitch angle is 12°, which represents the degree of tilt of the drone's fuselage relative to the horizontal plane in the forward and backward direction; the yaw angle is 35°, which reflects the angle of deviation of the drone's nose from due north; the roll angle is 8°, which reflects the angle of rotation of the drone's fuselage around the flight axis; the altitude is 550 meters, which is the drone's vertical altitude relative to the sea level; and the horizontal speed is 12 meters per second, which is the drone's flight speed in the horizontal direction.

[0113] Step S142: Calculate the heading deviation angle in the horizontal plane and the height compensation in the vertical direction based on the Euclidean distance between the predicted position coordinates and the current position of the UAV.

[0114] In detail, the predicted position coordinates are the predicted position of the survivor at the next moment. Assume the predicted position coordinates are (x2, y2, z2), and the current position coordinates of the UAV are (x1, y1, z1). First, calculate the Euclidean distance on the horizontal plane. According to the principle of the Euclidean distance formula, the distance on the horizontal plane is d = sqrt((x2-x1)^2 + (y2-y1)^2). For example, if x2-x1 = 5 meters and y2-y1 = 3 meters, then d = sqrt(5^2 + 3^2) = sqrt(34) meters. Then, calculate the heading deviation angle θ on the horizontal plane according to the trigonometric function, tanθ = (y2-y1) ÷ (x2-x1). In the above example, tanθ = 3 ÷ 5 = 0.6. The value of θ obtained by the arctangent function is approximately 31°. The vertical height compensation is z2-z1. Assume z2-z1 = -20 meters, which means that the predicted position is 20 meters lower in the vertical direction relative to the current position.

[0115] Step S143: Adjust the target setting value of the yaw angle based on the heading deviation angle, and adjust the target setting value of the altitude based on the altitude compensation amount.

[0116] For example, if the current yaw angle is 35° and the calculated heading deviation angle is 31°, then the adjusted target yaw angle setting is 35° + 31° = 66°. If the current altitude is 550 meters and the altitude compensation is -20 meters, then the adjusted target altitude setting is 550 - 20 = 530 meters.

[0117] Step S144: Calculate the torque adjustment coefficient of the propulsion motor and the propeller speed increment based on the vector difference between the velocity vector and the current horizontal velocity of the UAV.

[0118] Assuming the predicted velocity vector is (vx2, vy2), and the current horizontal velocity of the UAV is (vx1, vy1), the velocity vector difference is (vx2-vx1, vy2-vy1). For example, if vx2=15 m / s, vx1=12 m / s, vy2=4 m / s, and vy1=3 m / s, then the velocity vector difference is (15-12, 4-3)=(3, 1). First, calculate the magnitude of the velocity vector difference. According to the Pythagorean theorem, the magnitude is sqrt((3)^2+(1)^2)=sqrt(10) m / s. The torque adjustment coefficient of the propulsion motor is related to the magnitude of the velocity vector difference. Assuming there is a preset proportional coefficient k1 (which is determined based on the characteristics of the UAV's power system), for example, k1=0.3, then the torque adjustment coefficient of the propulsion motor is sqrt(10)×0.3. The propeller speed increment is related to the change in vertical velocity. Assuming the vertical component of the predicted velocity vector is vz2 = 6 m / s and the current vertical velocity of the UAV is vz1 = 5 m / s, the velocity difference is vz2 - vz1 = 1 m / s. There exists a coefficient k2 related to the propeller characteristics, for example, k2 = 1.5, then the propeller speed increment is 1 × k2 = 1.5.

[0119] Step S145: The adjusted yaw angle target setting value, altitude target setting value, torque adjustment coefficient, and propeller speed increment are encapsulated into a data packet format of the dynamic tracking command, and the corresponding dynamic tracking command is generated.

[0120] In detail, following a specific data packet format, for example, the yaw angle target setting is first converted to a numerical form with a specific precision (e.g., retained to two decimal places), assuming it to be 66.00°; the altitude target setting is 530 meters; the torque adjustment coefficient is retained with appropriate precision according to the previously calculated results, assuming it to be 0.95; and the propeller speed increment is 1.5. These data are arranged in a predetermined order, and some necessary verification information (such as checksums) is added to generate a dynamic tracking command. This dynamic tracking command is sent to the UAV's flight control system to guide the UAV to adjust its flight attitude, thereby continuously tracking the location of survivors in mountainous earthquake rescue scenarios.

[0121] In one possible implementation, step S150 includes:

[0122] Step S151: The dynamic tracking command is transmitted to the flight control system of the UAV via a wireless communication link, triggering the flight control system to synchronously adjust the pitch angle, yaw angle, roll angle, altitude and propeller speed.

[0123] In this embodiment, the wireless communication link employs a specific communication protocol, such as a wireless communication protocol based on the IEEE 802.11 standard, to ensure accurate transmission of commands. Upon receiving the command, the flight control system adjusts the relevant flight attitude parameters according to the target yaw angle, target altitude, torque adjustment coefficient, and propeller speed increments specified in the command. For example, if the target yaw angle is 66.00°, the flight control system adjusts components such as the UAV's rudder to gradually approach this target value. For the target altitude of 530 meters, the system controls the propeller speed to change lift, thereby adjusting the UAV's altitude. The system adjusts the output torque of the propulsion motor according to the torque adjustment coefficient, adjusts the propeller speed according to the propeller speed increments, and simultaneously adjusts the pitch and roll angles accordingly to maintain a stable flight attitude for the UAV.

[0124] Step S152: During the attitude adjustment process of the UAV, the updated dynamic image sequence is continuously acquired through the camera device, and the latest bounding box position of the target object in the dynamic image is marked.

[0125] In detail, the camera device acquires images at a fixed frame rate, such as 30 frames per second. In each frame, an object detection algorithm determines the latest bounding box position of the survivor. This object detection algorithm searches for possible target regions in the image based on previously learned features of the survivor (such as color features, texture features, etc.), and then determines the smallest rectangular region containing the survivor as the bounding box. For example, in one frame, the object detection algorithm detects the survivor's body outline, determining its upper left corner coordinates as (x3, y3) and lower right corner coordinates as (x4, y4). This rectangular region (x3, y3, x4, y4) is the latest bounding box position.

[0126] Step S153: Perform a consistency check between the latest bounding box position and the predicted position coordinates. If the consistency check result indicates that the position deviation exceeds a preset threshold, then re-initialize the multi-scale spatiotemporal feature extraction step.

[0127] In detail, the center coordinates of the latest bounding box position are calculated, assuming they are ((x3+x4)÷2, (y3+y4)÷2), and the deviation is calculated between these and the predicted position coordinates (x2, y2). First, the horizontal deviation is calculated as ((x3+x4)÷2-x2), then the vertical deviation is calculated as ((y3+y4)÷2-y2). Finally, the total positional deviation is calculated using the Pythagorean theorem, i.e., sqrt(((x3+x4)÷2-x2)^2+((y3+y4)÷2-y2)^2). Assuming a preset threshold of 10 meters, if the calculated positional deviation is greater than 10 meters, it may indicate that the survivor has experienced sudden, violent movement or a new occlusion situation. In this case, the multi-scale spatiotemporal feature extraction step needs to be re-initialized. This means clearing all temporary data stored in the previous process, such as the previous optical flow field matrix calculation results and intermediate data from the local texture feature calculation process, and then restarting the multi-scale spatiotemporal feature extraction operation from the current frame image.

[0128] Step S154: Update the truncation range of the local region according to the latest bounding box position after verification, and recalculate the global motion features and local texture features to obtain the updated global motion features and local texture features.

[0129] In detail, the updated bounding box position can be used as the center. For example, if the previous local region was a 300×300 pixel area centered on the predicted position coordinates, adjustments are made based on the updated bounding box position. For instance, if the coordinate change of the updated bounding box position necessitates expanding the local region, following a set rule (e.g., increasing by 100 pixels horizontally and vertically), the new local region becomes 500×500 pixels. The previous operations are then repeated on this new local region to calculate global motion features and local texture features. For global motion features, the optical flow matrix is ​​recalculated, including processing image frames within the new local region to construct the optical flow matrix. Then, a multi-resolution pyramid algorithm is used for hierarchical analysis to extract motion direction distribution histograms at different scales. This series of operations yields the updated global motion features. For local texture features, spatial convolution kernel filtering is applied to the new local region to extract multi-channel texture response maps. Then, a sliding window aggregation is performed along the time axis, and the mean and variance of the responses within the sliding window are calculated to obtain the updated local texture features.

[0130] Step S155: Input the updated global motion features and local texture features into the neural network model to iteratively generate new real-time motion prediction parameters until the relative motion relationship between the UAV and the target object meets the preset tracking accuracy.

[0131] In this embodiment, the neural network model performs feature fusion and nonlinear transformation based on the new input, and outputs new predicted position coordinates and velocity vectors. Then, based on these values, it calculates new real-time motion prediction parameters such as heading angle adjustment, flight altitude correction, and thrust parameters. This process iterates continuously, adjusting the UAV's flight attitude based on the new predicted position and velocity in each iteration. For example, if the new predicted position coordinates are closer to the survivor's actual position in one iteration, the UAV's flight attitude adjustment will be smaller. This process continues until the relative position deviation and velocity matching between the UAV and the survivor reach preset tracking accuracy requirements, such as a position deviation within 5 meters and a velocity matching error within 1 meter / second. At this point, it indicates that the UAV can stably and accurately track the survivor, completing the visual tracking task.

[0132] In one possible implementation, step S152 includes:

[0133] Step S1521: In each frame of the updated dynamic image, expand the search area by a preset ratio with the predicted position coordinates as the center.

[0134] In detail, in each updated frame of the dynamic image, a search region is expanded by a preset ratio centered on the predicted position coordinates. Assuming the predicted position coordinates are (x2, y2), the preset ratio is 20%, and the image resolution is 1920×1080 pixels, then in the horizontal direction, the left and right boundaries of the search region are x2-(1920×0.1) and x2+(1920×0.1), respectively; and in the vertical direction, the upper and lower boundaries of the search region are y2-(1080×0.1) and y2+(1080×0.1), respectively. This defines a search region centered on the predicted position coordinates.

[0135] Step S1522: Perform back projection of the color histogram on the pixels in the search area to generate a target probability distribution map.

[0136] For example, the distribution characteristics of survivor clothing colors have already been analyzed. By comparing the color of each pixel within the search area with these color distribution characteristics, the probability that each pixel belongs to a survivor can be calculated. For each pixel within the search area, assuming its color is c, and its corresponding probability in the color histogram is p(c), then the value of that pixel in the target probability distribution map is p(c). By performing this calculation on all pixels within the search area, the target probability distribution map is generated.

[0137] Step S1523: Locate the connected region with the highest probability density in the target probability distribution map, and calculate the minimum bounding rectangle of the connected region.

[0138] In the target probability distribution map, by traversing all pixels, the pixel with the highest probability density is found as the starting point. Then, starting from this pixel, a search is performed outwards, forming a connected region composed of pixels with a probability density greater than a certain threshold (e.g., 0.5) that are interconnected. This connected region may contain the survivor's complete or partial body. The minimum bounding rectangle of this connected region is then calculated, i.e., the smallest rectangle that completely contains the region. Assume the coordinates of the top-left vertex of this minimum bounding rectangle are (x3, y3), and the coordinates of the bottom-right vertex are (x4, y4).

[0139] Step S1524: Determine the position parameters of the latest bounding box position based on the vertex coordinates of the minimum bounding rectangle.

[0140] For example, the vertex coordinates (x3, y3, x4, y4) of the smallest bounding rectangle are the positional parameters of the latest bounding box position, which determine the approximate location range of the survivor in this frame of the image.

[0141] Step S1525: Perform Kalman filtering smoothing on the position parameters and the position parameters of the historical bounding box to eliminate jitter noise.

[0142] Kalman filtering is an optimal estimation algorithm based on a state-space model of a linear system. Assume the state vector of the historical bounding box position parameters is [x_last, y_last, vx_last, vy_last], where x_last and y_last are the center coordinates of the bounding box in the previous frame, and vx_last and vy_last are the velocity estimates of the center coordinates in the x and y directions, respectively. The bounding box position parameters of the current frame are [x_current, y_current]. First, the state vector of the current frame is predicted according to the state transition equation, which may be [x_predicted=x_last+vx_last, y_predicted=y_last+vy_last, vx_predicted=vx_last, vy_predicted=vy_last]. Then, the error between the measured and predicted values ​​is calculated according to the measurement equation, which is [z=[x_current, y_current]-[x_predicted, y_predicted]]. Finally, the updated state vector is calculated based on the Kalman gain, which is obtained from parameters such as the error covariance matrix. Finally, the smoothed position parameters are obtained. This process effectively eliminates noise caused by factors such as jitter during image acquisition or slight shaking of survivors.

[0143] In one possible implementation, step S153 includes:

[0144] Step S1531: Calculate the pixel distance between the center point coordinates of the latest bounding box position and the predicted position coordinates.

[0145] For example, the center point coordinates of the latest bounding box are ((x3+x4)÷2, (y3+y4)÷2), and the pixel distance between them and the predicted position coordinates (x2, y2) is calculated as follows. First, the horizontal pixel distance is calculated as the absolute value of ((x3+x4)÷2-x2), and the vertical pixel distance is calculated as the absolute value of ((y3+y4)÷2-y2). Then, the total pixel distance is calculated according to the Pythagorean theorem, i.e., sqrt(((x3+x4)÷2-x2)^2+((y3+y4)÷2-y2)^2).

[0146] Step S1532: Convert the pixel distance into the actual physical distance, specifically by performing perspective projection inversion using the current altitude of the UAV and the focal length parameters of the camera device.

[0147] Assume the drone's current altitude is h meters and the camera's focal length is f millimeters. According to the principle of perspective projection, the actual physical distance d = (h × pixel distance × sensor size) ÷ (f × image resolution). Here, sensor size refers to the actual dimensions of the camera's sensor, for example, a sensor width of w millimeters and a height of h_s millimeters. Assuming the image is captured according to the sensor's aspect ratio, then for horizontal calculations, the sensor size is taken as w millimeters. For vertical calculations or comprehensive calculations, the size can be adjusted based on the specific circumstances.

[0148] Step S1533: If the actual physical distance is greater than the preset threshold, it is determined that the target object has undergone violent movement or occlusion, and the execution of the current tracking command is suspended.

[0149] Assuming a preset threshold of 10 meters, if the calculated actual physical distance is greater than 10 meters, it means that the survivor's location deviates significantly from the predicted location. This could be due to a sudden, rapid movement by the survivor (such as running away from the drone) or obstruction by new landslides or other objects. In this case, the execution of the current tracking command should be paused to prevent the drone from adjusting its flight attitude based on erroneous predictions.

[0150] Step S1534: Clear the historical feature data cached in the neural network model and restart the multi-scale spatiotemporal feature extraction step from the current frame.

[0151] Step S1535: After re-executing the multi-scale spatiotemporal feature extraction step, the position parameters of the latest bounding box are used as the initial input first, skipping the dependence on historical feature data.

[0152] In this embodiment, the neural network model caches some historical feature data during previous tracking processes, such as intermediate results of previously calculated global motion features and local texture features. This cached data is cleared, and then multi-scale spatiotemporal feature extraction is re-performed starting from the current frame image. After re-executing the multi-scale spatiotemporal feature extraction step, the position parameters of the latest bounding box are preferentially used as the initial input, skipping the dependency on historical feature data. For example, when recalculating local texture features, the local region is truncated centered on the latest bounding box position, instead of relying on previous local region positions; when calculating global motion features, pixels near the latest bounding box position are used as reference points for calculations such as the optical flow matrix. This allows for faster and more accurate adaptation to sudden changes in survivor positions or re-tracking after occlusion.

[0153] In one possible implementation, step S154 includes:

[0154] Step S1541: Using the latest bounding box position after verification as the center, expand the cropping range according to the preset scaling factor to form a new local region.

[0155] In this embodiment, it is assumed that the coordinates of the top-left vertex of the latest bounding box after verification are (x3, y3), and the coordinates of the bottom-right vertex are (x4, y4), with a preset scaling factor of 1.5. Then, the horizontal boundaries of the new local region are calculated as follows: the new left boundary is x3 - ((x4 - x3) × 0.25), and the new right boundary is x4 + ((x4 - x3) × 0.25); the vertical boundaries are calculated as follows: the new top boundary is y3 - ((y4 - y3) × 0.25), and the new bottom boundary is y4 + ((y4 - y3) × 0.25). This determines a new local region centered on the latest bounding box and with an expanded range.

[0156] Step S1542: Perform Gaussian pyramid downsampling on the new local region to generate multi-resolution image blocks.

[0157] In detail, Gaussian pyramid downsampling is a method of downsampling an image using a Gaussian kernel function. First, for a new local region of the image, starting with the highest resolution layer, a convolution operation is performed using a Gaussian kernel (e.g., a 5×5 kernel size). This convolution operation weights and sums the pixels surrounding each pixel according to the Gaussian kernel function, resulting in a Gaussian-filtered image. Then, the filtered image is downsampled according to a set downsampling ratio (e.g., 0.5), that is, every other pixel is sampled, thus obtaining the next resolution layer of the image. This process is repeated to generate image patches of multiple resolution levels, each image patch being one layer in a multi-resolution image patch system.

[0158] Step S1543: Apply the Histogram of Oriented Gradients algorithm to the multi-resolution image blocks respectively to extract the spatial gradient distribution features at different resolutions.

[0159] For example, for each image patch at a given resolution, the Histogram of Oriented Gradients (HCG) algorithm calculates the gradient direction and magnitude of each pixel in the image. The gradient direction is determined by calculating the rate of change of grayscale at the pixel in the horizontal and vertical directions. For instance, for a pixel (x, y), if its horizontal grayscale change rate is Gx and its vertical grayscale change rate is Gy, then the gradient direction θ = arctan(Gy / Gx). The gradient magnitude is sqrt(Gx^2 + Gy^2). The image is then divided into several small regions (e.g., 8×8 regions). A histogram of the gradient direction is calculated within each small region, further dividing the gradient direction into several intervals (e.g., 0°-20°, 20°-40°, etc.). The cumulative gradient magnitude within each interval is calculated, thus obtaining the histogram of the gradient direction for each small region. The combination of the histograms of the gradient direction for all small regions represents the spatial gradient distribution characteristics of the image patch at that resolution.

[0160] Step S1544: The spatial gradient distribution features of different resolutions are superimposed according to weights to generate updated local texture features. At the same time, the optical flow field matrix is ​​recalculated within the expanded cropping range, and the direction distribution histogram of the global motion features is updated.

[0161] For example, spatial gradient distribution features at different resolutions have varying importance in representing local textures; therefore, different weights are assigned to each resolution's spatial gradient distribution feature. Assume the weight for the highest resolution is 0.5, the next lower resolution is 0.3, and the next lower is 0.2. For each corresponding feature component, the feature components at different resolutions are multiplied by their respective weights and then summed. For example, if a component of the highest resolution spatial gradient distribution feature is f1, the component corresponding to the next lower resolution is f2, and the component corresponding to the next lower resolution is f3, then the updated local texture feature for that component is f1×0.5 + f2×0.3 + f3×0.2. This process is repeated for all feature components to obtain the updated local texture features.

[0162] In one possible implementation, step S155 includes:

[0163] Step S1551: In each iteration, record the changing trend of the actual physical distance.

[0164] Step S1552: If the rate of decrease of the actual physical distance in a consecutive preset number of iterations is less than the convergence threshold, it is determined that the tracking steady state has been entered. In the tracking steady state, the sampling frequency of the camera device is reduced and the computational level of the multi-scale spatiotemporal feature extraction is reduced.

[0165] Step S1553: When the actual physical distance remains within the preset threshold for a predetermined duration, activate the tracking completion flag, and stop feature update iteration according to the tracking completion flag, maintaining the current flight attitude parameters until a new visual tracking task is received.

[0166] For example, within the expanded cropping range, similar to the previous calculation of the optical flow field matrix, the optical flow field matrix is ​​constructed based on the pixel differences between adjacent frames. For each frame image within the new local region, the positional change of each pixel in adjacent frames is calculated to obtain the horizontal and vertical motion components, thereby constructing the optical flow field matrix. Then, a multi-resolution pyramid algorithm is used to perform hierarchical analysis of the optical flow field matrix to update the orientation distribution histogram of global motion features. Specifically, the optical flow field matrix is ​​subjected to Gaussian pyramid generation to obtain a structure with multiple resolution levels. Starting from the highest resolution level, the optical flow field matrix of each level is divided into uniformly distributed square grid cells (e.g., each square grid cell is 10×10 pixels in size). The orientation angles of all optical flow vectors are counted within each square grid cell, and the orientation angles are divided into multiple orientation intervals according to a preset angle interval (e.g., 10°), generating an orientation angle distribution histogram for each square grid cell. The orientation angle distribution histograms of all square grid cells within each level are accumulated across cells to obtain the motion orientation distribution histogram of the current level. Maximum value normalization is performed on the motion direction distribution histogram of each level. The normalized motion direction distribution histograms of each level are then concatenated into one-dimensional vectors in descending order of resolution to form a cross-scale fused direction distribution feature vector. This direction distribution feature vector is part of the updated global motion feature direction distribution histogram. The updated global motion feature direction distribution histogram is obtained through complete calculation.

[0167] Furthermore, during each iteration, the trend of actual physical distance changes is recorded. The actual physical distance is obtained by converting the pixel distance between the latest bounding box position and the predicted position coordinates. This conversion process involves perspective projection inversion based on the UAV's current altitude and the camera's focal length parameters. In each iteration, the current actual physical distance is compared with the actual physical distance of the previous iteration, and the difference is calculated to obtain the trend of actual physical distance change. For example, if the current actual physical distance is 8 meters and the actual physical distance of the previous iteration was 9 meters, then the actual physical distance is decreasing by 1 meter.

[0168] If the rate of decrease of the actual physical distance in a predetermined number of consecutive iterations is less than the convergence threshold, the system is considered to have entered a steady-state tracking phase. Assume the predetermined number of iterations is 5, and the convergence threshold is 0.5 meters per iteration. In these 5 iterations, the decrease in actual physical distance is calculated for each iteration. For example, the first iteration decreases by 1 meter, the second by 0.8 meters, the third by 0.4 meters, the fourth by 0.3 meters, and the fifth by 0.2 meters. The average decrease rate is (1 + 0.8 + 0.4 + 0.3 + 0.2) ÷ 5 = 0.54 meters per iteration. Since 0.54 meters per iteration is less than the convergence threshold of 0.5 meters per iteration, the system is considered to have entered a steady-state tracking phase. In this steady-state, the sampling frequency of the camera is reduced, and the computational layers for multi-scale spatiotemporal feature extraction are decreased. For example, the original sampling frequency of the camera is 30 frames per second, which is reduced to 15 frames per second. For multi-scale spatiotemporal feature extraction, if previously three layers of Gaussian pyramid downsampling were used to calculate global motion features and local texture features, this can now be reduced to two layers. This reduces the consumption of computational resources, while still maintaining effective tracking of the target object (survivor) under relatively stable tracking conditions.

[0169] When the actual physical distance remains within a preset threshold for a predetermined duration, a tracking completion flag is activated. Based on this flag, feature updates cease, and the current flight attitude parameters are maintained until a new visual tracking task is received. Assuming the preset threshold is 5 meters and the predetermined duration is 10 seconds, if the actual physical distance remains within 5 meters for 10 seconds (e.g., 4 meters, 3 meters, 3.5 meters, 4.2 meters, 3.8 meters, 4.5 meters, 4 meters, 3.2 meters, 3.6 meters, and 4 meters respectively within those 10 seconds), satisfying the condition of remaining within 5 meters, the tracking completion flag is activated. Based on this flag, updates to global motion features and local texture features cease. The UAV maintains its current flight attitude parameters, such as pitch angle, yaw angle, roll angle, altitude, and propeller speed, until a new visual tracking task is received, such as the discovery of a new survivor or a request from the command center to readjust the tracking position.

[0170] Figure 2 The diagram illustrates exemplary hardware and software components of a neural network-based drone visual tracking system 100 that can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the neural network-based drone visual tracking system 100 and to perform the functions described in this application.

[0171] The neural network-based UAV visual tracking system 100 can be a general-purpose server or a special-purpose server, both of which can be used to implement the neural network-based UAV visual tracking method of this application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the load.

[0172] For example, a neural network-based UAV visual tracking system 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the neural network-based UAV visual tracking system 100 may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The neural network-based UAV visual tracking system 100 also includes an input / output (I / O) interface 150 between the computer and other input / output devices.

[0173] For ease of explanation, only one processor is described in the neural network-based UAV visual tracking system 100. However, it should be noted that the neural network-based UAV visual tracking system 100 of this application may also include multiple processors, and therefore the steps performed by one processor as described in this application may also be performed jointly or individually by multiple processors. For example, if the processor of the neural network-based UAV visual tracking system 100 performs steps A and B, it should be understood that steps A and B may also be performed jointly by two different processors or individually by one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor jointly perform steps A and B.

[0174] Furthermore, this embodiment of the invention also provides a readable storage medium, wherein computer-executable instructions are preset in the readable storage medium, and when the processor executes the computer-executable instructions, the above-mentioned UAV visual tracking method based on neural networks is implemented.

[0175] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A neural network-based unmanned aerial vehicle visual tracking method, characterized in that, The method comprises: real-time collection of a dynamic image sequence of a target object by a camera carried by a UAV, the dynamic image sequence containing motion trajectory information of the target object at consecutive time points; multi-scale spatio-temporal feature extraction on the dynamic image sequence to obtain global motion features and local texture features of the target image sequence, wherein the global motion features are used to represent displacement changes of the target object in three-dimensional space, and the local texture features are used to represent spatio-temporal continuity of surface details of the target object; input of the global motion features and the local texture features into a pre-trained neural network model for feature fusion to generate real-time motion prediction parameters of the target object; adjustment of flight attitude parameters of the UAV based on the real-time motion prediction parameters to generate corresponding dynamic tracking instructions, the dynamic tracking instructions being used to control the UAV to maintain a preset relative motion relationship with the target object; driving of the UAV to perform a visual tracking task according to the dynamic tracking instructions, while real-time updating of the dynamic image sequence and feedback to the multi-scale spatio-temporal feature extraction step to form a closed-loop tracking control; the driving of the UAV to perform the visual tracking task according to the dynamic tracking instructions, while the real-time updating of the dynamic image sequence and the feedback to the multi-scale spatio-temporal feature extraction step to form the closed-loop tracking control, comprising: transmission of the dynamic tracking instructions to a flight control system of the UAV through a wireless communication link to trigger the flight control system to synchronously adjust a pitch angle, a yaw angle, a roll angle, an altitude and a propeller speed; during the attitude adjustment process of the UAV, continuous collection of an updated dynamic image sequence by the camera and marking of a latest bounding box position of the target object in the dynamic image; consistency checking of the latest bounding box position and a predicted position coordinate of the target object at a next time point, and if a position deviation exceeds a preset threshold, re-initialization of the multi-scale spatio-temporal feature extraction step; updating of a local region range centered on the target object according to the checked latest bounding box position, and re-computation of the global motion features and the local texture features to obtain updated global motion features and local texture features; input of the updated global motion features and the local texture features into the neural network model to iteratively generate new real-time motion prediction parameters until the relative motion relationship between the UAV and the target object meets a preset tracking accuracy. 2.The neural network-based UAV visual tracking method of claim 1, wherein, the multi-scale spatio-temporal feature extraction on the dynamic image sequence to obtain the global motion features and the local texture features of the target image sequence, comprising: decomposition of the dynamic image sequence into a plurality of consecutive time segments, and inter-frame alignment processing of image frames in each time segment to obtain a stable image sequence with elimination of jitter; frame-by-frame grayscale normalization processing of the stable image sequence, and construction of an optical flow field matrix based on pixel differences between adjacent frames, the optical flow field matrix being used to quantify instantaneous motion vectors of the target object in an image plane; The global motion feature is formed by using a multi-resolution pyramid algorithm to hierarchically analyze the optical flow field matrix, extracting a motion direction distribution histogram at different scales, and concatenating motion direction distribution histogram features of each layer. A local region centered on the target object is intercepted from the stable image sequence, and a spatial domain convolution kernel filter is applied to the local region to extract a multi-channel texture response map; The multi-channel texture response map is aggregated in a sliding window along the time axis, the mean and variance of the response of each channel in the sliding window are counted, and the local texture feature is generated; The global motion feature is formed by using a multi-resolution pyramid algorithm to hierarchically analyze the optical flow field matrix, extracting a motion direction distribution histogram at different scales, and concatenating motion direction distribution histogram features of each layer. A Gaussian pyramid generation operation is performed on the optical flow field matrix to obtain a Gaussian pyramid structure containing multiple resolution levels, wherein the highest resolution layer corresponds to the original optical flow field matrix, and each lower resolution layer is generated by level-by-level downsampling based on a Gaussian kernel; Starting from the highest resolution layer of the Gaussian pyramid structure, the optical flow field matrix of each level is divided into uniformly distributed square grid cells, and each square grid cell covers a preset number of optical flow vectors; The direction angle of all optical flow vectors in each square grid cell is counted, and the direction angle is divided into multiple direction intervals according to a preset angle interval to generate a direction angle distribution histogram for each square grid cell; The direction angle distribution histograms of all square grid cells in each level are cross-cell accumulated to obtain the motion direction distribution histogram of the current level; The motion direction distribution histograms of each level are subjected to maximum value normalization processing, and the normalized motion direction distribution histograms of each level are one-dimensionally spliced in order of resolution from high to low to form a direction distribution feature vector fused across scales; The direction distribution feature vector is subjected to principal component analysis dimension reduction operation, and the dimension-reduced direction distribution feature vector is fused with the average amplitude of the optical flow of each level of the Gaussian pyramid structure to generate a final vector representation of the global motion feature. 3.The neural network-based UAV visual tracking method of claim 2, wherein, The global motion feature and the local texture feature are input into a pre-trained neural network model for feature fusion to generate real-time motion prediction parameters of the target object, including: The global motion feature and the local texture feature are respectively mapped to a high-dimensional hidden space of a pre-trained neural network model to obtain corresponding first and second hidden vectors; An association weight matrix between the first and second hidden vectors is calculated through a cross-attention mechanism, and the second hidden vector is reconstructed by weighting according to the association weight matrix to obtain an enhanced third hidden vector; The first and third hidden vectors are added element by element to obtain a fused fourth hidden vector; The fourth hidden vector is input into a multi-layer perceptron for nonlinear transformation to output predicted position coordinates and velocity vectors of the target object at the next time point. According to the predicted position coordinates and the velocity vector, a required heading angle adjustment amount, a flight height correction amount, and a propulsion force parameter of the UAV are calculated, and the real-time motion prediction parameters are generated. 4.The neural network-based UAV visual tracking method of claim 3, wherein, The flight attitude parameters of the UAV are adjusted based on the real-time motion prediction parameters, and corresponding dynamic tracking instructions are generated, including: Current flight state data of the UAV is obtained, including a pitch angle, a yaw angle, a roll angle, an altitude, and a horizontal speed. According to the Euclidean distance between the predicted position coordinates and the current position of the UAV, a heading deviation angle in the horizontal plane and a height compensation amount in the vertical direction are calculated. The target set value of the yaw angle is adjusted based on the heading deviation angle, and the target set value of the altitude is adjusted based on the height compensation amount. According to the vector difference between the velocity vector and the current horizontal speed of the UAV, a torque adjustment coefficient of the propulsion motor and a propeller speed increment are calculated. The adjusted target set value of the yaw angle, the target set value of the altitude, the torque adjustment coefficient, and the propeller speed increment are packaged into a data packet format of the dynamic tracking instructions, and corresponding dynamic tracking instructions are generated. 5.The neural network-based UAV visual tracking method of claim 4, wherein, During the attitude adjustment process of the UAV, updated dynamic image sequences are continuously collected by the camera device, and the latest bounding box position of the target object in the dynamic image is marked, including: In each updated dynamic image, a search area of a preset proportion is expanded centered on the predicted position coordinates; Color histogram back projection is performed on the pixels in the search area to generate a target probability distribution map; The connected region with the maximum probability density in the target probability distribution map is located, and the minimum circumscribed rectangle of the connected region is calculated; The position parameters of the latest bounding box position are determined according to the vertex coordinates of the minimum circumscribed rectangle; The position parameters are Kalman filtered and smoothed with the position parameters of the historical bounding box to eliminate jitter noise. 6.The neural network-based UAV visual tracking method of claim 4, wherein, The consistency of the latest bounding box position and the predicted position coordinates of the target object at the next time is verified, and if the consistency verification result indicates that the position deviation exceeds a preset threshold, the multi-scale spatio-temporal feature extraction step is reinitialized, including: The pixel distance between the center point coordinates of the latest bounding box position and the predicted position coordinates is calculated; The pixel distance is converted into an actual physical distance, specifically by perspective projection inversion through the current altitude of the UAV and the focal length parameters of the camera device; If the actual physical distance is greater than the preset threshold, it is determined that the target object has a violent motion or is occluded, and the execution of the current tracking instruction is paused; The historical feature data cached in the neural network model is emptied, and the multi-scale spatio-temporal feature extraction step is re-executed from the current frame; After re-executing the multi-scale spatio-temporal feature extraction step, the position parameters of the latest bounding box are preferentially used as the initial input, and the dependence on historical feature data is skipped. 7.The neural network-based UAV visual tracking method of claim 6, wherein, The updated latest bounding box position is checked, and the local area centered on the target object is updated in terms of the cutting range, and the global motion feature and the local texture feature are recalculated, including: A new local area is formed by expanding the cutting range according to a preset scaling factor, with the latest bounding box position after checking as the center; The new local area is down-sampled by a Gaussian pyramid to generate a multi-resolution image block; A direction gradient histogram algorithm is applied to the multi-resolution image block to extract spatial gradient distribution features at different resolutions; The spatial gradient distribution features at different resolutions are superimposed according to weights to generate updated local texture features, and the global motion feature is updated in terms of the direction distribution histogram, and the flow field matrix calculation is performed again in the expanded cutting range. 8.The neural network-based UAV visual tracking method of claim 6, wherein, The updated global motion feature and local texture feature are input into the neural network model to iteratively generate new real-time motion prediction parameters until the relative motion relationship between the unmanned aerial vehicle and the target object meets the preset tracking accuracy, including: In each iteration process, the change trend of the actual physical distance is recorded; If the decline rate of the actual physical distance in a continuous preset number of iterations is less than a convergence threshold, it is determined that the tracking steady state is entered, and in the tracking steady state, the sampling frequency of the camera device is reduced, and the operation level of the multi-scale spatiotemporal feature extraction is reduced; When the actual physical distance continuously remains within the preset threshold for a predetermined time, a tracking completion flag is activated, the feature update iteration is stopped according to the tracking completion flag, and the current flight attitude parameter is maintained until a new visual tracking task is received.

9. A neural network based unmanned aerial vehicle vision tracking system, characterized by, The neural network-based unmanned aerial vehicle visual tracking system includes a processor and a memory, the memory and the processor are connected, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to realize the neural network-based unmanned aerial vehicle visual tracking method in any one of claims 1-8.

Images