Dynamic target tracking method and system based on image return

Abstract

The invention provides a dynamic target tracking method and system based on image return, and relates to the technical field of image data processing, and the dynamic target tracking method based on image return comprises the following steps: collecting image data of a target scene, recognizing a to-be-tracked target and position information of the to-be-tracked target by using a double-layer random forest recognition algorithm, and predicting a motion track of the to-be-tracked target; according to the predicted motion track of the to-be-tracked target and the position information of the unmanned aerial vehicle, and based on a target function of the distance between the unmanned aerial vehicle and the to-be-tracked target, using a cooperative control strategy to adjust flight parameters of the unmanned aerial vehicle; and the unmanned aerial vehicle continuously tracks the to-be-tracked target, and after tracking is finished, the cloud server stores the position information and the movement track of the to-be-tracked target in the tracking process. The system has the advantages of real-time performance, high efficiency, intelligence, data storage and subsequent analysis, expansibility and flexibility, safety improvement, risk reduction and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a dynamic target tracking method and system based on image feedback. Background Art

[0002] Tracking tasks for ground targets are a key component of the UAV system's monitoring mission. During the execution process, the UAV first takes aerial photos in the designated area to capture key image data. This data is then transmitted to the processing center and analyzed through advanced image processing technology to identify and accurately locate ground targets. Based on the location information of the target, the system will automatically dispatch the tracking unit to respond. This process not only includes identifying static targets, but also responding to dynamically changing targets to ensure the real-time and accuracy of tracking. In this way, UAVs can effectively support a wide range of monitoring and security operations, improving the efficiency and success rate of mission execution.

[0003] At present, in complex dynamic environments, target recognition and positioning are often interfered by various factors, such as light fluctuations, cluttered backgrounds, and target occlusion. Existing recognition algorithms are difficult to deal with effectively, affecting the accuracy of recognition and positioning. In addition, motion trajectory prediction in dynamic tracking is crucial, but existing prediction technologies often fail to capture the precise motion patterns when faced with complex or irregular motion patterns, and the prediction methods are not robust enough and are easily affected by noise and abnormal data. Further optimization is needed to improve the overall tracking effect and reliability.

[0004] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0005] In view of this, the present invention provides a dynamic target tracking method and system based on image feedback to solve the above-mentioned problems.

[0006] In order to solve the above problems, the specific technical solutions adopted by the present invention are as follows: According to one aspect of the present invention, a dynamic target tracking method based on image return is provided, and the dynamic target tracking method based on image return comprises the following steps: S1. Use the camera equipment mounted on the drone to collect image data of the target scene, and use wireless communication technology to transmit the image data of the target scene back to the cloud server; S2. According to the image data of the target scene, a double-layer random forest recognition algorithm is used to identify the target to be tracked and its location information, and a mixed Gaussian trajectory prediction method is used to predict the motion trajectory of the target to be tracked; S3, adjusting the flight parameters of the UAV using a collaborative control strategy based on the predicted motion trajectory of the target to be tracked and the position information of the UAV, and based on the objective function of the distance between the UAV and the target to be tracked; S4. The drone continues to track the target, and after the tracking is completed, the cloud server stores the location information and movement trajectory of the target during the tracking process.

[0007] Preferably, according to the image data of the target scene, a double-layer random forest recognition algorithm is used to identify the target to be tracked and the position information of the target to be tracked, and a mixed Gaussian trajectory prediction method is used to predict the motion trajectory of the target to be tracked, which includes the following steps: S21, performing a preprocessing operation on the image data of the target scene, the preprocessing operation including image denoising and contrast enhancement; S22, identifying the target to be tracked based on the preprocessed image data of the target scene and determining the position information of the target to be tracked based on the double-layer random forest recognition algorithm; S23, based on the mixed Gaussian background difference method, background modeling and foreground separation are performed on the identified target to be tracked to obtain a foreground target image; S24: Predict the motion trajectory of the target to be tracked by combining the position information of the target to be tracked and the foreground target image.

[0008] Preferably, according to the preprocessed image data of the target scene, identifying the target to be tracked based on the double-layer random forest recognition algorithm, and determining the position information of the target to be tracked includes the following steps: S221, collecting historical image data of the target to be tracked, and performing preprocessing operations on the historical image data; S222, extracting features from the preprocessed historical image data, and constructing and training a two-layer random forest recognition model based on the extracted features; S223, taking the pre-processed image data of the target scene as input and inputting it into a double-layer random forest recognition model to obtain the category of the target to be tracked; S224: Determine the position information of the target to be tracked according to the category of the target to be tracked and the position in the target scene image data.

[0009] Preferably, performing feature extraction on the preprocessed historical image data, and constructing and training a two-layer random forest recognition model according to the extracted features comprises the following steps: S2221, combining the extracted features and their corresponding target category labels into a training data set; S2222. Use the training data set to construct the first layer of random forest, calculate the cumulative contribution rate of each feature on all decision trees in the random forest, and evaluate the importance of each feature; S2223. According to the importance evaluation result, select features that meet the preset importance threshold, and use the selected features to construct and train the second-layer random forest; S2224. Use a pre-set validation set to perform validation evaluation and model adjustment on the second-layer random forest to obtain a two-layer random forest recognition model.

[0010] Preferably, the first layer of random forest is constructed using the training data set, and the cumulative contribution rate of each feature on all decision trees in the random forest is calculated, and the importance of each feature is evaluated, including the following steps: S22221. Input the training data in the training data set into a random forest containing decision trees, and train each decision tree based on a random sample and a feature subset in the training data; S22222. Obtain node information of each decision tree in the random forest, and calculate the Gini index of each node; The calculation formula of the Gini index of each node is: ; In the formula, HI n Indicates the current node n The Gini index of K Represents the target category label set; q nk Indicates the current node n Middle k The ratio of the target category labels to the target category label set; q nk′ Indicates the current node n Middle k The ratio of ′ class target category labels to the target category label set; S22223. For each feature, on each decision tree of the random forest, the contribution rate of the feature on each node is calculated by comparing the change in the Gini index before and after the node branch; and the cumulative contribution rate of the feature on all decision trees of the random forest is obtained by adding the contribution rate of the feature on each node; S22224. Normalize the cumulative contribution rate of each feature in all decision trees of the random forest to obtain the importance of each feature.

[0011] Preferably, based on the mixed Gaussian background difference method, background modeling and foreground separation are performed on the identified target to be tracked to obtain a foreground target image, which includes the following steps: S231, initializing a mixed Gaussian background model based on each pixel in the preprocessed image data of the target scene; S232, updating Gaussian distribution parameters of each pixel according to the difference between the current frame and the background model, where the Gaussian distribution parameters include weight, mean, and variance; S233, evaluating the Gaussian component of each pixel, deleting redundant Gaussian components according to the ratio of weight to variance, and adjusting the weights of the remaining components to obtain an optimized mixed Gaussian background model; S234, using the optimized mixed Gaussian background model to separate the foreground of the target to be tracked, and obtaining a foreground target image through binarization processing; S235. Reconstruct the background image according to the Gaussian distribution parameters of the optimized mixed Gaussian background model.

[0012] Preferably, combining the position information of the target to be tracked and the foreground target image, predicting the motion trajectory of the target to be tracked includes the following steps: S241, obtaining the position information of the target to be tracked and the foreground target image of the continuous frames, and extracting the motion features of the target to be tracked respectively; S242, establishing a tracking target motion model using a linear regression method, and training the tracking target motion model based on the extracted motion features; S243: Use the trained tracking target motion model to predict the trajectory of the target to be tracked, and smooth the trajectory prediction result to obtain a final trajectory prediction result.

[0013] Preferably, adjusting the flight parameters of the UAV using a collaborative control strategy according to the predicted motion trajectory of the target to be tracked and the position information of the UAV, and based on the objective function of the distance between the UAV and the target to be tracked, includes the following steps: S31, using the background image to establish an environment model including obstacles, and establishing an objective function to minimize the distance between the UAV and the target to be tracked; S32, based on the environmental model and in combination with the predicted motion trajectory of the target to be tracked and the position information of the UAV, the optimal flight path of the UAV is calculated through a path planning algorithm; S33. According to the calculated optimal flight path, the flight parameters of the UAV are dynamically adjusted using a collaborative control strategy.

[0014] Preferably, based on the environmental model and in combination with the predicted motion trajectory of the target to be tracked and the position information of the drone, calculating the optimal flight path of the drone through a path planning algorithm includes the following steps: S321, taking the current position of the UAV as the starting point and the predicted motion trajectory of the target to be tracked as the initial flight path; S322, analyzing the position, size and shape of obstacles in the environment model, and determining whether the initial flight path intersects with the obstacles; S323. If the initial flight path intersects with an obstacle, the initial flight path is adjusted through a path planning algorithm in combination with the objective function of minimizing the distance between the UAV and the target to be tracked, and the adjusted initial flight path is used as the optimal flight path. Otherwise, the initial flight path is used as the optimal flight path.

[0015] According to another aspect of the present invention, a dynamic target tracking system based on image return is provided, the dynamic target tracking system based on image return comprises: an image data acquisition module, a motion trajectory prediction module, a flight control module and a data storage module; An image data acquisition module is used to collect image data of a target scene using a camera mounted on a drone, and transmit the image data of the target scene back to a cloud server using wireless communication technology; The motion trajectory prediction module is used to identify the target to be tracked and its location information based on the image data of the target scene using a double-layer random forest recognition algorithm, and predict the motion trajectory of the target to be tracked using a mixed Gaussian trajectory prediction method; The flight control module is used to adjust the flight parameters of the UAV using a collaborative control strategy based on the predicted motion trajectory of the target to be tracked and the position information of the UAV, and based on the objective function of the distance between the UAV and the target to be tracked; The data storage module is used for the drone to continuously track the target to be tracked, and after the tracking is completed, the cloud server stores the location information and movement trajectory of the target to be tracked during the tracking process.

[0016] The beneficial effects of the present invention are: 1. The present invention realizes real-time remote monitoring of the target, so that the operator can grasp the dynamic information of the target scene in time when far away from the scene. The double-layer random forest recognition algorithm can accurately and quickly identify the target to be tracked and its position information. At the same time, the mixed Gaussian trajectory prediction method can accurately predict the motion trajectory of the target, providing strong support for the tracking of the UAV. According to the predicted motion trajectory of the target to be tracked and the position information of the UAV, the flight parameters of the UAV are adjusted through the collaborative control strategy, realizing the intelligent control of the UAV. The dynamic target tracking method based on image feedback has many advantages such as real-time, high efficiency, intelligence, data storage and subsequent analysis, scalability and flexibility, as well as improved safety and reduced risks.

[0017] 2. The present invention provides a more accurate data basis for motion trajectory prediction by combining the mixed Gaussian background difference method with the foreground target image, which can more effectively separate the target to be tracked and reduce background interference, thereby improving the accuracy of trajectory prediction. By combining the position information of the target to be tracked and the foreground target image, the motion features are extracted, and a tracking target motion model is established, which can more comprehensively capture the motion information of the target, thereby improving the accuracy and reliability of trajectory prediction.

[0018] 3. By establishing an environmental model that includes obstacles, the present invention enables the UAV to perceive the complex environment around it and effectively avoid obstacles during flight, thereby greatly improving the safety and flight efficiency of the UAV and reducing the risk of damage or accidents caused by collisions. An objective function is established to minimize the distance between the UAV and the target to be tracked, so that the UAV can always maintain an optimal distance from the target during the tracking process, thereby improving the accuracy and efficiency of tracking. Taking into account the motion state of the target and the current position of the UAV, an efficient and safe flight route is planned for the UAV, reducing the flight time and energy consumption of the UAV, while improving the real-time and accuracy of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 is a flow chart of a dynamic target tracking method based on image return according to an embodiment of the present invention; Figure 2 It is a principle block diagram of a dynamic target tracking system based on image feedback according to an embodiment of the present invention.

[0020] In the figure: 1. Image data acquisition module; 2. Motion trajectory prediction module; 3. Flight control module; 4. Data storage module. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0022] According to an embodiment of the present invention, a dynamic target tracking method and system based on image feedback are provided.

[0023] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to an embodiment of the present invention, a dynamic target tracking method based on image return is provided, and the dynamic target tracking method based on image return includes the following steps: S1. Use the camera equipment mounted on the drone to collect image data of the target scene, and use wireless communication technology to transmit the image data of the target scene back to the cloud server; It should be noted that the drone is equipped with high-quality camera equipment, such as a high-definition camera or a thermal imaging camera, to collect image data of the target scene in real time or at a fixed time, including video streams or static images. According to the distance between the drone and the cloud server and the environmental conditions, the appropriate wireless communication technology is selected. Common options include: Wi-Fi: Suitable for short-distance, high-bandwidth applications.

[0024] Cellular network (4G / 5G): suitable for medium and long distance transmission, providing stable connection.

[0025] Satellite communication: Suitable for applications over long distances and in remote areas, but the cost is relatively high.

[0026] Configure communication module: Install and configure the corresponding wireless communication module on the drone, such as 4G / 5G module, Wi-Fi module or satellite communication module.

[0027] A stable connection between the drone and the cloud server is established through wireless communication technology.

[0028] Data upload: Upload image data to the cloud server in real time or in batches via wireless network. TCP / IP protocol or other suitable transmission protocols can be used to ensure reliable data transmission.

[0029] S2. According to the image data of the target scene, a double-layer random forest recognition algorithm is used to identify the target to be tracked and its location information, and a mixed Gaussian trajectory prediction method is used to predict the motion trajectory of the target to be tracked; As a preferred implementation, according to the image data of the target scene, a double-layer random forest recognition algorithm is used to identify the target to be tracked and the position information of the target to be tracked, and a mixed Gaussian trajectory prediction method is used to predict the motion trajectory of the target to be tracked, including the following steps: S21, performing a preprocessing operation on the image data of the target scene, the preprocessing operation including image denoising and contrast enhancement; Specifically, the purpose of image denoising is to remove noise from the image to make the image clearer. Common denoising methods include: Median filtering: Remove noise by replacing the grayscale value of each pixel with the median grayscale value of all pixels in the neighborhood of that pixel.

[0030] Gaussian filtering: Gaussian filtering is a linear smoothing filter that uses a Gaussian function as a weight for convolution operations to smooth the image and reduce noise.

[0031] Bilateral filtering: Bilateral filtering is a nonlinear filter that smoothes an image while preserving edge information. It combines two factors: spatial proximity and pixel intensity similarity.

[0032] The purpose of contrast enhancement is to increase the difference between different areas in an image, making the image clearer and easier to identify. Common contrast enhancement methods include: Histogram equalization: Histogram equalization redistributes the grayscale values ​​of the image so that the grayscale histogram of the image is evenly distributed, thereby improving the contrast of the image.

[0033] Adaptive Histogram Equalization (CLAHE): Adaptive histogram equalization divides the image into multiple small blocks, performs histogram equalization on each small block separately, and then merges the results. This can avoid the over-enhancement problem caused by global histogram equalization.

[0034] S22, identifying the target to be tracked based on the preprocessed image data of the target scene and determining the position information of the target to be tracked based on the double-layer random forest recognition algorithm; As a preferred embodiment, according to the pre-processed image data of the target scene, identifying the target to be tracked based on the double-layer random forest recognition algorithm, and determining the position information of the target to be tracked includes the following steps: S221, collecting historical image data of the target to be tracked, and performing preprocessing operations on the historical image data; S222, extracting features from the preprocessed historical image data, and constructing and training a two-layer random forest recognition model based on the extracted features; As a preferred implementation, feature extraction is performed on the preprocessed historical image data, and a two-layer random forest recognition model is constructed and trained based on the extracted features, including the following steps: S2221, combining the extracted features and their corresponding target category labels into a training data set; It should be noted that a series of features are extracted from the preprocessed historical image data. These features can be manually designed (such as color histogram, texture features, etc.) or automatically extracted by a deep learning model.

[0035] Label each image with its corresponding class label. For example, if the task is to identify different types of vehicles, then each image might be labeled "car", "truck", "motorcycle", etc.

[0036] The extracted feature vectors and corresponding category labels are combined into a training data set. This data set usually consists of two parts: feature matrix (X) and label vector (y). Each row in the feature matrix represents the feature vector of a sample, and each element in the label vector represents the category label of the sample.

[0037] S2222. Use the training data set to construct the first layer of random forest, calculate the cumulative contribution rate of each feature on all decision trees in the random forest, and evaluate the importance of each feature; As a preferred implementation, the first layer of random forest is constructed using the training data set, and the cumulative contribution rate of each feature on all decision trees in the random forest is calculated, and the importance of each feature is evaluated, including the following steps: S22221. Input the training data in the training data set into a random forest containing decision trees, and train each decision tree based on a random sample and a feature subset in the training data; It should be noted that a random forest consists of multiple decision trees, and each tree uses a random subset of the dataset and a random subset of features during training.

[0038] Each decision tree is trained based on a random sample and feature subset from the training data. This can reduce the variance of the model and improve generalization ability.

[0039] S22222. Obtain node information of each decision tree in the random forest, and calculate the Gini index of each node; It should be noted that in random forests, the node information of each decision tree can be used to evaluate the purity of the node through its Gini Index. The Gini Index is a commonly used impurity measurement method used to measure the uncertainty of a data set.

[0040] The calculation formula of the Gini index of each node is: ; In the formula, HI n Indicates the current node n The Gini index of K Represents the target category label set; q nk Indicates the current node n Middle kThe ratio of the target category labels to the target category label set; q nk′ Indicates the current node n Middle k The ratio of ′ class target category labels to the target category label set; S22223. For each feature, on each decision tree of the random forest, the contribution rate of the feature on each node is calculated by comparing the change in the Gini index before and after the node branch; and the cumulative contribution rate of the feature on all decision trees of the random forest is obtained by adding the contribution rate of the feature on each node; It should be noted that for each feature, at each node, the comparison of the change in the Gini index before and after the node branch is usually achieved by calculating the weighted sum of the Gini indexes of the left and right child nodes after the branch (based on the number of samples or proportion) and the difference between the Gini index of the node before the branch. This change reflects the degree to which the feature improves the classification purity of the model at this node, that is, the importance of the feature.

[0041] The contribution rate can be defined as the ratio of the change in the Gini index caused by a feature at a certain node to the total change in the Gini index caused by all features at that node.

[0042] This means that for each node, the change in the Gini index of each feature needs to be calculated and divided by the sum of the changes in all features to obtain the contribution rate of the feature on the node.

[0043] For each feature, its contribution rate on all nodes of all decision trees needs to be added up to get the cumulative contribution rate of the feature in the random forest.

[0044] S22224. Normalize the cumulative contribution rate of each feature in all decision trees of the random forest to obtain the importance of each feature.

[0045] It should be noted that the purpose of normalization is to convert the contribution rate into relative importance so that the sum of the importance of all features is 1. The following are the steps for normalization: Calculate the cumulative contribution rate: For each feature, first calculate its cumulative contribution rate on all decision trees of the random forest. This is usually obtained by adding the contribution rate of the feature at each node.

[0046] Sum: Calculate the sum of the cumulative contribution rates of all features. This will be used for subsequent normalization calculations.

[0047] Normalization: For each feature, divide its cumulative contribution by the sum of the cumulative contributions of all features. This will result in a value between 0 and 1, indicating the relative importance of the feature.

[0048] The mathematical formula is: Importance = cumulative contribution rate / sum of cumulative contribution rates of all features; Sorting and Visualization: Rank the features based on their normalized importance to make it easier to identify which features are most important to the model’s predictive performance.

[0049] S2223. According to the importance evaluation result, select features that meet the preset importance threshold, and use the selected features to construct and train the second-layer random forest; It should be noted that, based on the importance evaluation results, screening out features that meet the preset importance threshold, and using the screened features to construct and train the second-layer random forest is an effective way to improve model performance and reduce overfitting. The specific implementation steps are as follows: (1) Set a preset importance threshold. This threshold can be determined based on the requirements and experience of the specific task. For example, the threshold can be set to 0.1, which means that only features with importance greater than or equal to 0.1 are selected.

[0050] (2) According to the set importance and threshold, filter out the features that meet the conditions. Assume the following features and their importance: Feature 1: importance 0.5; Feature 2: importance 0.333; Feature 3: importance 0.167; If the importance threshold is set to 0.2, the filtered features will be: Feature 1: importance 0.5; Feature 2: importance 0.333; (3) Use the filtered features (feature 1 and feature 2) to build and train the second layer of random forest. The following are the detailed steps: Extract filtered features: Extract feature 1 and feature 2 from the original dataset.

[0051] Combine the filtered features and the corresponding category labels into a new training data set.

[0052] Initialize Random Forest Model: Create a new random forest model.

[0053] Adjust the parameters of random forest as needed, such as the number of trees, maximum depth, etc.

[0054] The filtered features and labels are input into the new random forest model for training.

[0055] The random forest model builds multiple decision trees based on the screened features and performs classification through these decision trees.

[0056] S2224. Use a pre-set validation set to perform validation evaluation and model adjustment on the second-layer random forest to obtain a two-layer random forest recognition model.

[0057] It should be noted that the second-layer random forest is used to predict the validation set and obtain the prediction results. According to the real labels of the validation set and the prediction results, the accuracy, recall, F1 score, AUC-ROC curve, confusion matrix and other evaluation indicators are calculated to comprehensively evaluate the performance of the model. By analyzing the evaluation indicators, the advantages and disadvantages of the model can be identified. For example, if the accuracy of the model is high but the recall rate is low, it may mean that the model's ability to identify positive samples is not strong enough.

[0058] Based on the evaluation results, adjust the parameters of the second layer of random forest. For example, if the model is too complex and causes overfitting, you can try reducing the number of decision trees or reducing the depth of the tree.

[0059] S223, taking the pre-processed image data of the target scene as input and inputting it into a double-layer random forest recognition model to obtain the category of the target to be tracked; Specifically, the extracted feature vector is input into the trained two-layer random forest recognition model, and the two-layer random forest model will classify the input feature vector according to the feature importance and classification rules learned during training; The model outputs the category of the object to be tracked. For example, if the task is to identify different types of vehicles, the model may output categories such as "car", "truck", "motorcycle", etc.

[0060] S224: Determine the position information of the target to be tracked according to the category of the target to be tracked and the position in the target scene image data.

[0061] Specifically, the category information of the target to be tracked is obtained from the output of the two-layer random forest model. For example, suppose the model output is "car".

[0062] Locate the objects belonging to this category in the image. Object detection algorithms (such as YOLO, SSD, FasterR-CNN, etc.) are usually used to detect all possible object areas in the image. The object detection algorithm outputs the bounding box of each detected object, including the coordinates of the upper left corner and the lower right corner.

[0063] According to the category information output by the model, the objects belonging to this category are filtered out from the results of the object detection algorithm. For example, if the model outputs "sedan", all bounding boxes marked as "sedan" are filtered out from the object detection results.

[0064] For each filtered object, record the coordinates of its bounding box. These coordinates represent the specific location of the object in the image. Usually, the coordinates of the bounding box are expressed in the form of (x, y, w, h), where (x, y) is the coordinate of the upper left corner of the bounding box, and w and h are the width and height of the bounding box, respectively.

[0065] S23, based on the mixed Gaussian background difference method, background modeling and foreground separation are performed on the identified target to be tracked to obtain a foreground target image; As a preferred implementation, based on the mixed Gaussian background difference method, background modeling and foreground separation are performed on the identified target to be tracked, and obtaining a foreground target image includes the following steps: S231, initializing a mixed Gaussian background model based on each pixel in the preprocessed image data of the target scene; It should be noted that for each pixel in the image data of the target scene after preprocessing, a mixture model containing multiple Gaussian distributions (usually 3-5) is initialized. Each Gaussian distribution represents a possible background state of the pixel, with a weight (indicating the probability of the state occurring), a mean (indicating the pixel value of the state) and a variance (indicating the fluctuation range of the pixel value of the state).

[0066] S232, updating Gaussian distribution parameters of each pixel according to the difference between the current frame and the background model, where the Gaussian distribution parameters include weight, mean, and variance; It should be noted that for each pixel in the current frame, its pixel value is matched with the Gaussian distribution in the background model. If the match is successful (that is, the pixel value is within the confidence interval of a certain Gaussian distribution), the weight, mean and variance of the Gaussian distribution are updated; if the match fails, it may be necessary to add a new Gaussian distribution to represent the new background state, or adjust the weight of the existing Gaussian distribution to reduce the weight of the mismatched distribution.

[0067] S233, evaluating the Gaussian component of each pixel, deleting redundant Gaussian components according to the ratio of weight to variance, and adjusting the weights of the remaining components to obtain an optimized mixed Gaussian background model; It should be noted that for each pixel, the Gaussian component is evaluated based on the ratio of weight to variance. Gaussian components with small weights and large variances may represent noise or short-term changes and can be deleted. After removing redundant components, the weights of the remaining components are adjusted to ensure that their sum is 1.

[0068] S234, using the optimized mixed Gaussian background model to separate the foreground of the target to be tracked, and obtaining a foreground target image through binarization processing; It should be noted that the optimized mixed Gaussian background model is used to perform foreground judgment on each pixel in the current frame. If the pixel value does not match any Gaussian distribution in the background model (that is, it is not within the confidence interval of any Gaussian distribution), the pixel is marked as foreground. Finally, through binarization processing, the foreground pixels are set to white (or other colors) and the background pixels are set to black (or other colors), and the foreground target image is obtained.

[0069] S235. Reconstruct the background image according to the Gaussian distribution parameters of the optimized mixed Gaussian background model.

[0070] It should be noted that for each pixel, the Gaussian distribution with the largest weight is selected (or weighted average is performed according to the weight), and its mean is used as the background value of the pixel. The background values ​​of all pixels are combined to obtain the reconstructed background image.

[0071] S24: predicting the motion trajectory of the target to be tracked by combining the position information of the target to be tracked and the foreground target image.

[0072] As a preferred implementation, combining the position information of the target to be tracked and the foreground target image, predicting the motion trajectory of the target to be tracked includes the following steps: S241, obtaining the position information of the target to be tracked and the foreground target image of the continuous frames, and extracting the motion features of the target to be tracked respectively; Specifically, the steps of obtaining the position information of the target to be tracked and the foreground target image of the continuous frames and extracting the motion features of the target to be tracked respectively include: The position information of the target to be tracked is obtained from consecutive frames. This position information usually includes the bounding box coordinates (such as the coordinates of the upper left corner and the lower right corner) or the center point coordinates of the target in each frame, and the position information of the target in each frame is recorded to form a time series.

[0073] By using the mixed Gaussian background difference method, the foreground target images of each frame are obtained. These images only contain the foreground target and the background has been removed.

[0074] Extract motion features from location information. Common motion features include: Speed: The rate of change of the target's displacement between adjacent frames.

[0075] Acceleration: The rate of change of target velocity.

[0076] Direction: The direction the target is moving (can be expressed as an angle).

[0077] Size change: The size change of the object bounding box.

[0078] Position change: The position change of the target center point.

[0079] S242, establishing a tracking target motion model using a linear regression method, and training the tracking target motion model based on the extracted motion features; Specifically, by selecting features related to the target's motion as input variables, for example, the target's speed, acceleration, direction, and other features can be used, and then the target's position at a certain moment in the future is selected as the output variable.

[0080] The extracted motion features and the corresponding target position information are combined into a training data set. Each row of data contains a set of feature values ​​and the corresponding target position.

[0081] For example, given the following features and location information: Features: velocity v, acceleration a, direction θ; Position: (x,y); The motion model is established using linear regression. The basic form of linear regression is: y=Xw+b; Among them, y is the target position vector, X is the feature matrix, ww is the weight vector, and b is the bias term.

[0082] Use the training data set to train the linear regression model and find the optimal weight vector w and bias term b. You can use the least squares method or other optimization algorithms to solve w and b.

[0083] S243: Use the trained tracking target motion model to predict the trajectory of the target to be tracked, and smooth the trajectory prediction result to obtain a final trajectory prediction result.

[0084] It should be noted that the trained linear regression model is used to input the motion features of the current frame to predict the target position at a certain moment in the future. For example, if you want to predict the target position of the next frame, you can input the features of the current frame into the model to get the predicted position.

[0085] Smooth the predicted trajectory to reduce the impact of noise and outliers. Commonly used smoothing methods include: Kalman filter: combines predictions and observations to provide more accurate and smoother trajectory estimates.

[0086] Moving average: The predicted trajectory is averaged within the window.

[0087] Exponential smoothing: Gives higher weight to recent data and reduces the impact of historical data.

[0088] Savitzky-Golay filter: Smoothing via polynomial fitting.

[0089] S3, adjusting the flight parameters of the UAV using a collaborative control strategy based on the predicted motion trajectory of the target to be tracked and the position information of the UAV, and based on the objective function of the distance between the UAV and the target to be tracked; As a preferred implementation, according to the predicted motion trajectory of the target to be tracked and the position information of the UAV, and based on the objective function of the distance between the UAV and the target to be tracked, the flight parameters of the UAV are adjusted using a collaborative control strategy, including the following steps: S31, using the background image to establish an environment model including obstacles, and establishing an objective function to minimize the distance between the UAV and the target to be tracked; It should be noted that, based on the background image, the position, size and shape of static obstacles are identified and marked through image processing techniques (such as edge detection, contour analysis, deep learning, etc.). The environment is divided into several grid cells, each of which represents an area. According to the position of the obstacle, it is marked as an obstacle in the corresponding grid cell to obtain an environment model containing the obstacle.

[0090] Specifically, the goal of distance minimization is to minimize the Euclidean distance between the drone and the target to be tracked. In order to ensure the safe flight of the drone, some constraints need to be added to the objective function, such as: Obstacle avoidance constraint: The drone cannot collide with any obstacles in the environment.

[0091] Speed ​​and acceleration limits: The speed and acceleration of the drone should be within a safe range.

[0092] Boundary restrictions: The flight range of drones should be within the designated area.

[0093] S32, based on the environmental model and in combination with the predicted motion trajectory of the target to be tracked and the position information of the UAV, the optimal flight path of the UAV is calculated through a path planning algorithm; As a preferred implementation, based on the environmental model and in combination with the predicted motion trajectory of the target to be tracked and the location information of the drone, calculating the optimal flight path of the drone through a path planning algorithm includes the following steps: S321, taking the current position of the UAV as the starting point and the predicted motion trajectory of the target to be tracked as the initial flight path; S322, analyzing the position, size and shape of obstacles in the environment model, and determining whether the initial flight path intersects with the obstacles; Specifically, it is assumed that the predicted motion trajectory of the target to be tracked has been obtained and used as the initial flight path of the UAV. The initial flight path can be represented as a series of waypoints P={p0,p1,p2,…,pn}, where pn represents the position of the nth waypoint.

[0094] The initial flight path is represented as a series of line segments, with one line segment formed between every two adjacent waypoints.

[0095] For each line segment Li, it can be expressed by a parametric equation: Li(t)=(1−t)pi+tpi+1,t∈[0,1]; For each line segment Li, calculate all the grid cells it passes through. Check whether these grid cells are marked as obstacles. If there are cells marked as obstacles, the path is considered to have an intersection with the obstacle.

[0096] S323. If the initial flight path intersects with an obstacle, the initial flight path is adjusted through a path planning algorithm in combination with the objective function of minimizing the distance between the UAV and the target to be tracked, and the adjusted initial flight path is used as the optimal flight path. Otherwise, the initial flight path is used as the optimal flight path.

[0097] Specifically, when adjusting the initial flight path through the path planning algorithm, the A* algorithm can be used to adjust the initial flight path, which specifically includes the following steps: The current position of the drone is taken as the starting point S, and the target position is taken as the end point G; Initialize the open list and closed list, and add the starting point to the open list.

[0098] For each node n in the open list, calculate its cost function f(n)=g(n)+h(n); g(n) is the actual cost from the starting point to node n, and h(n) is the heuristic estimated cost from node n to the end point (such as Euclidean distance).

[0099] Select the node with the smallest f(n) value as the current node.

[0100] Expand all neighbor nodes of the current node and check whether they are within the obstacle area.

[0101] If the neighbor node is not in the obstacle area and has not been visited, it is added to the open list and its cost is updated.

[0102] When the end point G is found, the path is backtracked to generate the optimal path.

[0103] The resulting path is smoothed to reduce unnecessary turns and improve path continuity.

[0104] S33. According to the calculated optimal flight path, the flight parameters of the UAV are dynamically adjusted using a collaborative control strategy.

[0105] Specifically, in order to ensure that the drone can safely and efficiently follow the path, speed and acceleration need to be controlled. Specifically, it includes: PID Controller: Use a proportional-integral-derivative (PID) controller to regulate the speed of the drone to keep it within the desired speed range.

[0106] Feedforward control: Based on information such as the curvature and slope of the path, the speed is adjusted in advance to adapt to changes in the path.

[0107] Acceleration control: Limit acceleration: Set the maximum allowed acceleration to prevent the drone from losing control due to excessive acceleration or deceleration.

[0108] Smooth acceleration changes: Through filters or smoothing algorithms, ensure that acceleration changes are smooth and reduce impact on the drone structure.

[0109] Among them, the collaborative control strategy is used to dynamically adjust the flight parameters of the UAV to ensure that the UAV can safely and efficiently track the optimal flight path. It monitors the flight parameters and status of the UAV in real time and dynamically adjusts the flight parameters to cope with environmental changes and changes in mission requirements.

[0110] S4. The drone continues to track the target, and after the tracking is completed, the cloud server stores the location information and movement trajectory of the target during the tracking process.

[0111] Specifically, the drone continues to track the target, and after the tracking is completed, the cloud server stores the location information and motion trajectory of the target during the tracking process, including the following steps: The position information, speed information of the UAV, the position information and motion trajectory data of the target are collected during the tracking process, and pre-processed (such as filtering, denoising, etc.) to improve the data quality.

[0112] Determine whether the tracking task is completed based on the preset tracking time, target loss or other conditions.

[0113] All data collected during the tracking process are sorted and packaged, including the flight parameters of the drone, the target's location information and movement trajectory, etc.

[0114] The sorted data is transmitted to the cloud server through wireless communication technology (such as 4G / 5G, Wi-Fi, etc.).

[0115] After receiving the data, the cloud server stores it in the database and manages and maintains it. This data can be used for subsequent analysis, mining and application.

[0116] like Figure 2As shown, according to one embodiment of the present invention, a dynamic target tracking system based on image return is provided, and the dynamic target tracking system based on image return includes: an image data acquisition module 1, a motion trajectory prediction module 2, a flight control module 3 and a data storage module 4; The image data acquisition module 1 is used to collect the image data of the target scene using the camera device mounted on the drone, and transmit the image data of the target scene back to the cloud server using wireless communication technology; The motion trajectory prediction module 2 is used to identify the target to be tracked and its position information by using a double-layer random forest recognition algorithm according to the image data of the target scene, and to predict the motion trajectory of the target to be tracked by using a mixed Gaussian trajectory prediction method; The flight control module 3 is used to adjust the flight parameters of the UAV by using a collaborative control strategy based on the predicted motion trajectory of the target to be tracked and the position information of the UAV and the objective function of the distance between the UAV and the target to be tracked; The data storage module 4 is used for the drone to continuously track the target to be tracked, and after the tracking is completed, the cloud server stores the position information and movement trajectory of the target to be tracked during the tracking process.

[0117] In summary, with the help of the above technical solution of the present invention, the present invention realizes real-time remote monitoring of the target, so that the operator can grasp the dynamic information of the target scene in time when far away from the scene, and the double-layer random forest recognition algorithm can be used to accurately and quickly identify the target to be tracked and its position information. At the same time, through the mixed Gaussian trajectory prediction method, the motion trajectory of the target can be accurately predicted, providing strong support for the tracking of the drone. According to the predicted motion trajectory of the target to be tracked and the position information of the drone, the flight parameters of the drone are adjusted through the collaborative control strategy, and the intelligent control of the drone is realized. The dynamic target tracking method based on image return has multiple advantages such as real-time, high efficiency, intelligence, data storage and subsequent analysis, scalability and flexibility, and improved safety and reduced risks. The present invention provides a more accurate data basis for motion trajectory prediction by combining the foreground target image with the mixed Gaussian background difference method, which can more effectively separate the target to be tracked and reduce background interference, thereby improving the accuracy of trajectory prediction, combining the position information of the target to be tracked and the foreground target image, extracting motion features, and establishing a tracking target motion model, which can more comprehensively capture the motion information of the target, thereby improving the accuracy and reliability of trajectory prediction. By establishing an environment model including obstacles, the present invention enables the UAV to perceive the complex environment around it and effectively avoid obstacles during flight, thereby greatly improving the safety and flight efficiency of the UAV and reducing the risk of damage or accidents caused by collisions. An objective function based on minimizing the distance between the UAV and the target to be tracked is established, so that the UAV can always maintain an optimal distance from the target during the tracking process, thereby improving the accuracy and efficiency of tracking. Taking into account the motion state of the target and the current position of the UAV, an efficient and safe flight route is planned for the UAV, reducing the flight time and energy consumption of the UAV and improving the real-time and accuracy of tracking.

[0118] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program code.

[0119] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A dynamic target tracking method based on image return, characterized in that: The dynamic target tracking method based on image return includes the following steps: S1. Use the camera equipment mounted on the drone to collect image data of the target scene, and use wireless communication technology to transmit the image data of the target scene back to the cloud server; S2. According to the image data of the target scene, a double-layer random forest recognition algorithm is used to identify the target to be tracked and its location information, and a mixed Gaussian trajectory prediction method is used to predict the motion trajectory of the target to be tracked; S3, adjusting the flight parameters of the UAV using a collaborative control strategy based on the predicted motion trajectory of the target to be tracked and the position information of the UAV, and based on the objective function of the distance between the UAV and the target to be tracked; S4. The drone continues to track the target, and after the tracking is completed, the cloud server stores the location information and movement trajectory of the target during the tracking process.

2. The method for dynamic target tracking based on image transmission according to claim 1, characterized in that: The method of using a double-layer random forest recognition algorithm to identify the target to be tracked and the position information of the target to be tracked according to the image data of the target scene, and predicting the motion trajectory of the target to be tracked by a mixed Gaussian trajectory prediction method includes the following steps: S21, performing a preprocessing operation on the image data of the target scene, wherein the preprocessing operation includes image denoising and contrast enhancement; S22, identifying the target to be tracked based on the preprocessed image data of the target scene and determining the position information of the target to be tracked based on the double-layer random forest recognition algorithm; S23, based on the mixed Gaussian background difference method, background modeling and foreground separation are performed on the identified target to be tracked to obtain a foreground target image; S24: Predict the motion trajectory of the target to be tracked by combining the position information of the target to be tracked and the foreground target image.

3. The method for dynamic target tracking based on image return according to claim 2, characterized in that: The method of identifying the target to be tracked based on the preprocessed image data of the target scene and determining the position information of the target to be tracked based on the double-layer random forest recognition algorithm includes the following steps: S221, collecting historical image data of the target to be tracked, and performing preprocessing operations on the historical image data; S222, extracting features from the preprocessed historical image data, and constructing and training a two-layer random forest recognition model based on the extracted features; S223, taking the pre-processed image data of the target scene as input and inputting it into a double-layer random forest recognition model to obtain the category of the target to be tracked; S224: Determine the position information of the target to be tracked according to the category of the target to be tracked and the position in the target scene image data.

4. The method for dynamic target tracking based on image return according to claim 3, characterized in that: The feature extraction of the preprocessed historical image data and the construction and training of the double-layer random forest recognition model according to the extracted features include the following steps: S2221, combining the extracted features and their corresponding target category labels into a training data set; S2222. Use the training data set to construct the first layer of random forest, calculate the cumulative contribution rate of each feature on all decision trees in the random forest, and evaluate the importance of each feature; S2223. According to the importance evaluation result, select features that meet the preset importance threshold, and use the selected features to construct and train the second-layer random forest; S2224. Use a pre-set validation set to perform validation evaluation and model adjustment on the second-layer random forest to obtain a two-layer random forest recognition model.

5. The method for dynamic target tracking based on image return according to claim 4, characterized in that: The first layer of random forest is constructed using the training data set, and the cumulative contribution rate of each feature on all decision trees in the random forest is calculated, and the importance of each feature is evaluated, including the following steps: S22221. Input the training data in the training data set into a random forest containing decision trees, and train each decision tree based on a random sample and a feature subset in the training data; S22222. Obtain node information of each decision tree in the random forest, and calculate the Gini index of each node; The calculation formula of the Gini index of each node is: ； In the formula, HI n Indicates the current node n The Gini index of K Represents the target category label set; q nk Indicates the current node n Middle k The ratio of the target category labels to the target category label set; q nk′ Indicates the current node n Middle k The ratio of ′ class target category labels to the target category label set; S22223. For each feature, on each decision tree of the random forest, the contribution rate of the feature on each node is calculated by comparing the change in the Gini index before and after the node branch; and the cumulative contribution rate of the feature on all decision trees of the random forest is obtained by adding the contribution rate of the feature on each node; S22224. Normalize the cumulative contribution rate of each feature in all decision trees of the random forest to obtain the importance of each feature.

6. The method for dynamic target tracking based on image return according to claim 5, characterized in that: The method of performing background modeling and foreground separation on the identified target to be tracked based on the mixed Gaussian background difference method to obtain a foreground target image includes the following steps: S231, initializing a mixed Gaussian background model based on each pixel in the preprocessed image data of the target scene; S232, updating Gaussian distribution parameters of each pixel according to the difference between the current frame and the background model, wherein the Gaussian distribution parameters include weight, mean and variance; S233, evaluating the Gaussian component of each pixel, deleting redundant Gaussian components according to the ratio of weight to variance, and adjusting the weights of the remaining components to obtain an optimized mixed Gaussian background model; S234, using the optimized mixed Gaussian background model to separate the foreground of the target to be tracked, and obtaining a foreground target image through binarization processing; S235. Reconstruct the background image according to the Gaussian distribution parameters of the optimized mixed Gaussian background model.

7. The method for dynamic target tracking based on image return according to claim 6, characterized in that: The method of predicting the motion trajectory of the target to be tracked by combining the position information of the target to be tracked and the foreground target image comprises the following steps: S241, obtaining the position information of the target to be tracked and the foreground target image of the continuous frames, and extracting the motion features of the target to be tracked respectively; S242, establishing a tracking target motion model using a linear regression method, and training the tracking target motion model based on the extracted motion features; S243: Use the trained tracking target motion model to predict the trajectory of the target to be tracked, and smooth the trajectory prediction result to obtain a final trajectory prediction result.

8. The method for dynamic target tracking based on image return according to claim 6, characterized in that: The method of adjusting the flight parameters of the UAV by using a collaborative control strategy based on the predicted motion trajectory of the target to be tracked and the position information of the UAV, and based on the objective function of the distance between the UAV and the target to be tracked, includes the following steps: S31, using the background image to establish an environment model including obstacles, and establishing an objective function to minimize the distance between the UAV and the target to be tracked; S32, based on the environmental model and in combination with the predicted motion trajectory of the target to be tracked and the position information of the UAV, the optimal flight path of the UAV is calculated through a path planning algorithm; S33. According to the calculated optimal flight path, the flight parameters of the UAV are dynamically adjusted using a collaborative control strategy.

9. The method for dynamic target tracking based on image return according to claim 8, characterized in that: The method of calculating the optimal flight path of the UAV by a path planning algorithm based on the environmental model and in combination with the predicted motion trajectory of the target to be tracked and the position information of the UAV includes the following steps: S321, taking the current position of the UAV as the starting point and the predicted motion trajectory of the target to be tracked as the initial flight path; S322, analyzing the position, size and shape of obstacles in the environment model, and determining whether the initial flight path intersects with the obstacles; S323. If the initial flight path intersects with an obstacle, the initial flight path is adjusted through a path planning algorithm in combination with the objective function of minimizing the distance between the UAV and the target to be tracked, and the adjusted initial flight path is used as the optimal flight path. Otherwise, the initial flight path is used as the optimal flight path.

10. A dynamic target tracking system based on image transmission, used to implement the dynamic target tracking method based on image transmission according to any one of claims 1 to 9, characterized in that: The dynamic target tracking system based on image feedback includes: an image data acquisition module, a motion trajectory prediction module, a flight control module and a data storage module; The image data acquisition module is used to collect image data of the target scene using the camera device mounted on the drone, and transmit the image data of the target scene back to the cloud server using wireless communication technology; The motion trajectory prediction module is used to identify the target to be tracked and its position information by using a double-layer random forest recognition algorithm according to the image data of the target scene, and to predict the motion trajectory of the target to be tracked by using a mixed Gaussian trajectory prediction method; The flight control module is used to adjust the flight parameters of the UAV using a collaborative control strategy based on the predicted motion trajectory of the target to be tracked and the position information of the UAV, and based on the objective function of the distance between the UAV and the target to be tracked; The data storage module is used for the drone to continuously track the target to be tracked, and after the tracking is completed, the cloud server stores the position information and movement trajectory of the target to be tracked during the tracking process.

Images