A high-altitude parabolic tracking method with target dynamic convolution kernel

Abstract

The present invention discloses a method for tracking high-altitude parabolic objects using a target dynamic convolution kernel. The present invention focuses on the learning of dynamic convolution kernels that changes with the target category. The present invention regards the target center point as a key feature, and the pixels near the target center point as a query feature, and uses the key and query matching method to learn the weights of the nearby pixels. The present invention extracts the weights at each position in the dynamic convolution kernel by sequentially matching the pixels near the target center point. In order to suppress the influence of interference targets and difficult targets on target tracking, the present invention adds a dynamic convolution kernel learning process for interference target classes and difficult target classes in addition to the tracking target class, and designs a dynamic convolution kernel target detection network that integrates multiple target categories. In the high-altitude parabolic object tracking process of the test video, the present invention uses a target detection model with a multi-target dynamic convolution kernel, which can effectively suppress interference information and difficult information, thereby improving the accuracy of high-altitude parabolic object tracking.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a high-altitude parabolic tracking method using a target dynamic convolution kernel. Background Art

[0002] With the rapid development of urbanization, high-rise buildings are constantly emerging, creating more potential for objects to be thrown from heights. High-rise buildings and overpasses, among other high-altitude locations, have become potential sources of objects being thrown, making the problem of objects being thrown from heights even more prominent in the context of urbanization. Simultaneously, population growth has led to more crowded cities and increased density in public spaces. This increases the probability of injury from objects being thrown from heights, as more people may be caught in the path of objects.

[0003] Current solutions include: deploying smart surveillance cameras to monitor potential high-altitude object dropping through computer vision and image recognition technology. Once an abnormality is detected, the system can automatically sound an alarm and notify law enforcement agencies or relevant personnel; developing a sound recognition system that can identify the sound produced by objects such as falling windows hitting the ground. Once the system recognizes a similar sound pattern, it will trigger an alarm; installing vibration sensors in areas prone to high-altitude object dropping, which can sense the vibrations generated when objects fall from heights and thus issue timely alarms; developing an early warning system to predict potential object dropping risks by real-time monitoring the stability and movement of objects at high altitudes such as large buildings and bridges; using artificial intelligence and machine learning technology to analyze historical data and patterns to predict the possible location and time of high-altitude object dropping incidents so that preventive measures can be taken in advance; using drones for high-altitude inspections to regularly check high-altitude areas such as building exterior walls and windows to detect and repair potential high-altitude object dropping risks; and developing smart alarm devices that can be installed on windows or high-altitude locations. Once abnormal object dropping behavior is detected, it can issue alarms such as sound, light, or text messages.

[0004] Existing patent number one, "A Method for Estimating the Occupants of Objects Thrown from High Altitudes Based on a Parabolic Model," CN114545386A, discloses a method for estimating the occupants of objects thrown from high altitudes based on a parabolic model. This method uses a parabolic model to estimate the location of the object's occupants when a millimeter-wave radar system is monitoring objects thrown from high altitudes in real time. This method uses a classical parabolic model to perform nonlinear curve fitting based on the partial two-dimensional or three-dimensional target trajectory detected by the radar, predicting and completing the actual parabolic trajectory, and then inferring the location range of the object's occupants.

[0005] Existing patent 1 has the following problems: it uses the results of the parabolic projection to reversely infer the location of residents, lacking an immediate warning function for high-altitude objects; in terms of trajectory prediction, it uses a classical mechanics model, but only considers air resistance and does not take into account abnormal conditions such as wind direction, which may lead to large prediction errors;

[0006] Existing patent 2, "A method for monitoring high-altitude parabolic objects," CN111539388A: determines whether parabolic objects have occurred by comparing adjacent picture frames, then finds the timestamp of the picture frame where the parabolic target first appears and finds the images taken by all cameras at that moment for binarization to obtain the binary image corresponding to each camera; establishes a rectangular coordinate system corresponding to the room number along the monitoring surface of the building, determines the coordinates corresponding to each camera and the geometric relationship between the parabolic point and the camera, and calculates the parabolic coordinates corresponding to the parabolic point based on the binary image;

[0007] The existing second patent has the following main problems in target detection of moving objects: the design requires the surveillance video files to be sent to the control center, which takes a lot of time and lacks timeliness; at the same time, it does not propose a specific and effective parabolic tracking model, and cannot obtain the target motion trajectory;

[0008] Existing patent three "A system for detecting objects thrown from high altitude", CN111368741A: provides a system for detecting objects thrown from high altitude, including: an image acquisition module for acquiring images of high-rise buildings; a high-altitude object detection module for detecting whether there are objects thrown from high altitude based on the images; a target tracking module for tracking the high-altitude object graphics detected by the high-altitude object detection module and determining whether it is an object thrown from high altitude; a result display module for displaying the results according to the determination result of the target tracking module; and a storage module for storing video information of objects thrown from high altitude.

[0009] The existing patent three has the following problems in target tracking of moving objects: the technology is mainly aimed at image detection, and there is a high possibility of missed detection and false detection; it also requires manual monitoring, which is costly and affects efficiency. Summary of the Invention

[0010] The purpose of the present invention is to remedy the defects of the existing technology and provide a high-altitude parabolic tracking method with a target dynamic convolution kernel.

[0011] The present invention is achieved through the following technical solutions:

[0012] A high-altitude parabolic tracking method using a target dynamic convolution kernel specifically comprises the following steps:

[0013] S1: Constructing dynamic convolution kernel for tracking target

[0014] Collect high-altitude parabolic video data, mark the target detection box and target category of the video frame, extract the key and query features of the tracking target, and calculate the dynamic convolution kernel of the tracking target;

[0015] S2: Training the object detection model

[0016] Train the target detection model after integrating the dynamic convolution kernel to obtain the initial target detection model;

[0017] S3: Constructing dynamic convolution kernels for difficult targets

[0018] Use the initial target detection model to obtain the target detection frame, calculate the target detection loss value, obtain difficult samples based on the threshold of the target detection loss value, extract the key and query features of the difficult target, and calculate the dynamic convolution kernel of the difficult target;

[0019] S4: Constructing dynamic convolution kernel of interference target

[0020] Determine the interference sample category, obtain the interference sample set from the ImageNet dataset, expand the interference sample set, extract the key and query features of the interference target, and calculate the dynamic convolution kernel of the interference target;

[0021] S5: Training an object detection model with dynamic convolution kernels for multiple object categories

[0022] Train the initial target detection model to obtain a target detection model with multi-target dynamic convolution kernels;

[0023] S6: Test video of high-altitude object tracking

[0024] The target detection model with a multi-target dynamic convolution kernel is used to detect the test video frame to obtain the target detection frame. The tracking frame is predicted through the state transition matrix. The tracking frame and the detection frame are matched using an algorithm. The same operation is performed on subsequent frames to obtain the motion trajectory of the high-altitude parabolic target.

[0025] Step S1 specifically includes the following steps:

[0026] Step 1-1: Use the camera to collect video of objects thrown from high altitude;

[0027] Step 1-2: Sampling the video data into continuous video frame data to obtain a video frame set;

[0028] Step 1-3: Label the bounding box of the moving object in each video frame and the target center coordinates in is the coordinate of the upper left corner of the bounding box, is the coordinate of the lower right corner of the bounding box;

[0029] Step 1-4: Mark the category of the moving target in each video frame to obtain the category label of the target;

[0030] Step 1-5: Obtain the initial sample set: Init= <Frame init ,Box init ,Label init>, where Frame init Represents the frame obtained by merging the bounding box with the label, Box init Represents the initial sample bounding box, Label init represents the initial sample label;

[0031] Step 1-6: Create a dynamic convolution module. At the current layer, the key features and query features are obtained from the pixels of the video frame:

[0032] Step 1-6-1: Extract the tracking target area: At different resolutions, the corresponding pixels are different, and the target center point coordinates are The pixels in the h×w range are used as key features: H and W are the height resolution and width resolution of the current layer video frame respectively. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1};

[0033] Step 1-6-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region, and use the Softmax function as the weight for the similarity score:

[0034]

[0035] Among them, C track is the feature dimension;

[0036] Step 1-6-3: Perform the above operation on all pixels to obtain the weight set

[0037] Step 1-6-4: Use the obtained weight value as the convolution kernel to obtain the dynamic convolution kernel of the tracking target.

[0038] Step S2 specifically includes the following steps:

[0039] Step 2-1: Select RetinaNet as the pre-trained model. Before the model convolution operation, call the dynamic convolution module in step 1-6 to obtain the dynamic convolution kernel. Replace the original convolution kernel with the tracking target dynamic convolution kernel. Use this model to detect the initial sample set annotated in step 1-5. Process each sample to obtain the target detection frame. And target detection score pre,i , where i is the sample number, is the coordinate of the upper left corner of the target detection box, The coordinates of the lower right corner of the target detection box;

[0040] Step 2-2: Given the number of samples is M init Calculate the target detection loss value, which includes two loss values L pre,box and L pre,label ;

[0041] Step 2-2-1: Calculate the loss of the target detection box and the annotated bounding box:

[0042]

[0043] The calculation formula of SmoothL1Loss is:

[0044]

[0045] * indicates digital multiplication; x1 is the detected coordinate value, and x2 is the actual coordinate value.

[0046] Step 2-2-2: Calculate the target prediction probability score pre ,i and true probability score score gt Losses:

[0047]

[0048] Step 2-2-3: The total loss is the sum of the bounding box loss and the prediction score loss:

[0049] L pre =L pre,box +L pre,label

[0050] Step 2-3: Use the total loss calculated in step 2-3-3 to train the model using backpropagation to obtain the initial target detection model.

[0051] Step S3 specifically includes the following steps:

[0052] Step 3-1: Use the initial target detection model obtained in step S2-3 to detect the data annotated in steps 1-5 to obtain the target detection frame;

[0053] Step 3-2: Get the target prediction probability score init,i ;

[0054] Step 3-3: Set the prediction score threshold for difficult samples;

[0055] Step 3-4: Samples with prediction scores less than the threshold are considered difficult samples;

[0056] Step 3-5: Obtain difficult sample set: Diff = <Frame diff ,Box diff ,Label diff>, where Frame diff Indicates difficult frames, Box diff Indicates the difficult sample bounding box, Label diff Indicates the difficult sample label;

[0057] Step 3-6: Create a dynamic convolution module. At the current layer, the key features and query features are obtained from the pixels of the video frame.

[0058] Step 3-6-1: Extract the tracking target area: At different resolutions, the corresponding pixels are different. The pixels within the h×w range of the target center coordinates are used as key features: H and W are the height resolution and width resolution of the current layer video frame respectively. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1};

[0059] Step 3-6-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region;

[0060]

[0061] Among them, C diff is the feature dimension;

[0062] Step 3-6-3: Perform the above operation on all pixels to obtain the weight set

[0063] Step 3-6-4: Use the obtained weight value as the convolution kernel to obtain the difficult target dynamic convolution kernel.

[0064] Step S4 specifically includes the following steps:

[0065] Step 4-1: Determine the interference sample categories as flying birds, leaves, clothes, and quilts;

[0066] Step 4-2: Obtain an interference target dataset containing flying birds, leaves, clothes, and quilts from the ImageNet dataset;

[0067] Step 4-3: Data preprocessing and data enhancement;

[0068] Step 4-3-1: Scale the images in the interference target dataset to a resolution of 256x256 by normalizing the size;

[0069] Step 4-3-2: Randomly flip the image within the range of 0-180°;

[0070] Step 4-3-3: Set the target bounding box position, put the bounding box into the image, and synthesize the picture to get the Frame disturb ;

[0071] Step 4-4: Obtain interference sample set: Disturb= <Frame disturb ,Box disturb ,Label disturb >, where Frame disturb Indicates interference frame, Box disturb Represents the bounding box, Label disturb Indicates a label;

[0072] Step 4-5: Create a dynamic convolution module. At the current layer, the key features and query features are obtained from the pixels of the video frame.

[0073] Step 4-5-1: Extract the tracking target area: At different resolutions, the corresponding pixels are different. The pixels within the h×w range of the target center coordinates are used as key features: H and W are the height resolution and width resolution of the current layer video frame respectively. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1};

[0074] Step 4-5-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region;

[0075]

[0076] Among them, C disturb is the feature dimension;

[0077] Step 4-5-3: Perform the above operation on all pixels to obtain the weight set

[0078] Step 4-5-4: Use the obtained weight value as the convolution kernel to obtain the dynamic convolution kernel of the interference target.

[0079] Step S5 specifically includes the following steps:

[0080] Step 5-1: Fusion of the initial sample set Init, the difficult sample set Diff, and the interference sample set Disturb of the tracking target class to obtain the active sample set for training the active target detection model

[0081] Fuse=<Init,Diff,Disturb> ,

[0082] Fuse= <Frame fuse ,Box Fuse ,Label fuse >

[0083] Frame fuse Indicates fusion frame, Box fuse Represents the bounding box, Label fuse Represents the label, active means obtaining the difficult sample set through the detection score threshold;

[0084] Step 5-2: Process the active sample set, use the initial target detection model obtained in step 2-3 for the difficult image, and replace the convolution kernels of the model Stage 2 to Stage 5 with dynamic convolution kernels;

[0085] Step 5-2-1: Using the active sample set obtained in step 5-1, the initial resolution of the sample input model is 640×480;

[0086] Step 5-2-2: Stage 1 uses a normal convolution kernel and downsampling to process the sample, obtaining an output resolution of 320×240;

[0087] Step 5-2-3: Stage 2 uses dynamic convolution kernel to extract sample features in the active sample set, uses the tracking sample dynamic convolution kernel obtained in steps 1-6 to extract features for the tracking sample, uses the difficult sample dynamic convolution kernel obtained in steps 3-6 to extract features for the difficult sample, and uses the interference sample dynamic convolution kernel obtained in steps 4-5 to extract features for the interference sample, and then uses maximum pooling for downsampling to obtain the result Feature stage3 =MaxPool(DynamicConv(Frame fuse )), the processed resolution is 160×120;

[0088] Step 5-2-4: Stage 3 uses dynamic convolution kernel to extract sample features. The tracking sample dynamic convolution kernel obtained in steps 1-6 is used to extract features. The difficult sample dynamic convolution kernel obtained in steps 3-6 is used to extract features. The interference sample dynamic convolution kernel obtained in steps 4-5 is used to extract features. Then, the maximum pooling is used for downsampling to obtain the result Feature. stage4 =MaxPool(DynamicConv(Feature stage3 ), the resolution after processing is 80×60;

[0089] Step 5-2-5: Stage 4 uses dynamic convolution kernel to extract sample features in the active sample set, uses the tracking sample dynamic convolution kernel obtained in steps 1-6 to extract features for the tracking sample, uses the difficult sample dynamic convolution kernel obtained in steps 3-6 to extract features for the difficult sample, and uses the interference sample dynamic convolution kernel obtained in steps 4-5 to extract features for the interference sample, and then uses maximum pooling for downsampling to obtain the result Feature stage5 =MaxPool(DynamicConv(Feature stage4 ), the resolution after processing is 40×30;

[0090] Step 5-2-6: Feature by deconvolution stage5 Upsampling, and feature after convolution with 1*1 convolution kernel stage4 Residual connection to obtain fusion features

[0091]

[0092] Step 5-2-7: Deconvolution Upsampling, and feature after convolution with 1*1 convolution kernel stage3 Residual connection to obtain fusion features

[0093]

[0094] Step 5-2-8: Deconvolution Upsampling, and feature after convolution with 1*1 convolution kernel stage2 Residual connection to obtain fusion features

[0095]

[0096] Step 5-3: Use the classification and bounding box sub-networks to process the features obtained in steps 5-3-4, 5-3-5, and 5-3-6 respectively to obtain the target detection frame And target detection score fuse,i , where i is the sample number, is the coordinate of the upper left corner of the target detection box, The coordinates of the lower right corner of the target detection box;

[0097] Step 5-4: Given the number of samples is M fuse Calculate the target detection loss value, which includes two loss values L fuse,box and L fuse,label ;

[0098] Step 5-4-1: Calculate the loss of the target detection box and the annotated bounding box:

[0099]

[0100] The SmoothL1Loss calculation formula is:

[0101]

[0102] * indicates digital multiplication, x1 is the detected coordinate value, and x2 is the real coordinate value;

[0103] Among them, M fuse is the number of samples in the initial sample set;

[0104] Step 5-4-2: Calculate the target prediction probability score fuse,i and the true probability score gt Losses:

[0105]

[0106] Among them, M fuse is the number of samples in the fusion sample set;

[0107] Step 5-4-3: The total loss value is the sum of the bounding box loss and the prediction score loss:

[0108] L fuse =L fuse,box +L fuse,label

[0109] Step 5-5: Use the total loss calculated in step 5-4-3, use backpropagation to train the model, and obtain the target detection model with multi-target dynamic convolution kernel.

[0110] Step S6 specifically includes the following steps:

[0111] Step 6-1: Sample the test video, parse the video data into continuous video frame data, and obtain a video frame set Where i represents the video frame number, represents the i-th frame of a video frame set;

[0112] Step 6-2: Use the active sample target detection model obtained in step S5-5 to detect the first frame frame1 of the video frame set obtained in step S6-1 test , get the target detection frame, get the width and height of the target detection frame;

[0113] Step 6-3: Calculate the first frame frame1 test The position and velocity of each target in the 1st frame are obtained by

[0114] Obj 1,i =[cenx 1,i ,ceny 1,i ,vx 1,i ,vx 1,i ] T

[0115] Among them cenx 1,i ,ceny 1,i Respectively represent the x-axis coordinate and y-axis coordinate of the target center point, vx 1,i ,vx 1,i Represent the x-direction speed and y-direction speed of the target center point respectively;

[0116] Step 6-4: When the time is short and the range of motion is small, the linear motion of the target in the x and y directions is used to approximate the falling motion. The state transition matrix is used to describe the temporal state change relationship from the first frame to the second frame. Δt = 1 / 32 seconds represents the time interval between the two frames. The state transition matrix Trans is expressed as

[0117]

[0118] Step 6-5: Use the target state in step 6-3 and the state transition matrix obtained in step 6-4 to predict the state vector of the target at the next moment

[0119] Obj 2,i =Trans×Obj 1,i

[0120] Where × represents matrix multiplication;

[0121] Using the above target state and the height and width of the detection frame in step 6-2, obtain the target tracking frame is the coordinate of the upper left corner of the target tracking box, is the coordinate of the lower right corner of the target tracking box;

[0122] Step 6-6: Repeat steps 6-3 to 6-5 to predict n tracking boxes of n targets in the second frame;

[0123] Step 6-7: Use the active sample target detection model obtained in step S5-5 to detect the second frame frame2 of the video frame set obtained in step S6-1 test , get the target detection frame, get the width and height of the target detection frame;

[0124] Steps 6-8: Track the coordinates of frame i based on the second frame The coordinates of the second frame detection box j Calculate the position relationship score between the two:

[0125] Step 6-8-1: Calculate the intersection area:

[0126]

[0127] InterArea i,j =InterWidth i,j xInterHeight i,j

[0128] Step 6-8-2: Calculate the union area:

[0129]

[0130] UnionArea i,j =AreaI+AreaJ-InterArea i,j

[0131] Among them, x represents the multiplication of numbers;

[0132] Step 6-8-3: Use the intersection ratio as the position relationship score:

[0133] IoU i,j =InterArea i,j / UnionArea i,j

[0134] IOU i,j The larger the value, the higher the overlap between the detection frame and the tracking frame. Selecting the maximum value will achieve the best match between the detection frame and the tracking frame.

[0135] Step 6-9: Select the n target tracking frames of the first frame in turn and traverse the m target detection frames of the second frame. Repeat steps 6-8 to obtain n*m position relationship scores and fill them into the n*m matrix IoU. 1,2 ,

[0136]

[0137] IoU i,j Represents the positional relationship score between the i-th tracking frame in the first frame and the j-th detection frame in the second frame;

[0138] Steps 6-10: Perform the Hungarian algorithm on the position relationship matrix IoU:

[0139] Step 6-10-1: For each row of the position relationship matrix IoU, find the minimum value, and then subtract the minimum value from all elements in the row;

[0140] Step 6-10-2: For each column of the position relationship matrix IoU, find the minimum value, and then subtract the minimum value from all elements in that column;

[0141] Step 6-10-3: For each row, find a zero element and mark the row where the zero element is located as the selected row;

[0142] Step 6-10-4: In the selected row, find a zero element that does not intersect with the previously selected column, and mark the column where the zero element is located as the selected column;

[0143] Step 6-10-5: Select the sum of the number of rows and columns as n, match the tracking boxes and detection boxes corresponding to the rows and columns, and obtain the optimal matching solution that maximizes the total IOU score;

[0144] Step 6-11: Update the tracking frame with the detection frame successfully matched in step 6-10-5;

[0145] Step 6-12: Repeat steps 6-2 to 6-11 for subsequent frames of the test video until the end of the video frame to obtain the motion trajectory of the high-altitude parabolic target.

[0146] The advantages of the present invention are: 1. The dynamic convolution kernel of the present invention can adaptively adjust the convolution kernel according to the characteristics of the input data, thereby adapting to tracking targets, difficult targets, and interfering targets. This enables the network to capture features of different scales and levels according to the needs of the task without the need to define a fixed convolution kernel in advance, making it suitable for detecting and tracking deformable targets;

[0147] 2. In addition to learning normal tracking targets, the present invention also learns many difficult targets and interference targets, which can cover many categories, reduce the risk of model overfitting, and enhance the model's identification ability;

[0148] 3. In target tracking, the target may be affected by occlusion and posture changes. The dynamic convolution kernel can adaptively capture the changing parts of the target, thereby accurately tracking the target even under partial occlusion and posture changes, effectively solving the problem of missed detection in tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0149] Figure 1 Flowchart of the high-altitude parabolic tracking method with target dynamic convolution kernel;

[0150] Figure 2 To calculate the dynamic convolution kernel of the tracking target;

[0151] Figure 3 Dynamic convolution kernels for computationally difficult targets;

[0152] Figure 4To calculate the dynamic convolution kernel of the interference target;

[0153] Figure 5 Target detection model for training dynamic convolution kernels for multiple target categories;

[0154] Figure 6 To test the video of high altitude parabolic tracking. DETAILED DESCRIPTION

[0155] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments;

[0156] like Figure 1 As shown in FIG, a high-altitude parabolic tracking method using a target dynamic convolution kernel specifically includes the following steps:

[0157] S1: Constructing dynamic convolution kernel for tracking target

[0158] like Figure 2 As shown in the figure, high-altitude parabolic video data is collected, the target detection frame and target category of the video frame are marked, the key and query features of the tracking target are extracted, and the dynamic convolution kernel of the tracking target is calculated;

[0159] Step 1-1: Use the camera to collect video of objects thrown from high altitude;

[0160] Step 1-2: Sampling the video data into continuous video frame data to obtain a video frame set;

[0161] Step 1-3: Label the bounding box of the moving object in each video frame and the target center coordinates in is the coordinate of the upper left corner of the bounding box, is the coordinate of the lower right corner of the bounding box;

[0162] Step 1-4: Mark the category of the moving target in each video frame to obtain the category label of the target;

[0163] Step 1-5: Obtain the initial sample set: Init= <Frame init ,Box init ,Label init >, where Frame init Represents the frame obtained by merging the bounding box with the label, Box init Represents the initial sample bounding box, Label init represents the initial sample label;

[0164] Step 1-6: Create a dynamic convolution module to obtain the dynamic convolution kernel before the model convolution operation and apply it to the convolution operation. At the current layer, the key feature and query feature are obtained from the pixels of the video frame; Step 1-6-1: Extract the tracking target area: At different resolutions, the corresponding pixels are changing, and the target center point coordinates are The pixels in the h×w range are used as key features: H and W are the height resolution and width resolution of the current layer video frame. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1};

[0165] Step 1-6-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region, and use the Softmax function as the weight for the similarity score:

[0166]

[0167] Among them, C track is the feature dimension;

[0168] Step 1-6-3: Perform the above operation on all pixels to obtain the weight set

[0169] Step 1-6-4: Use the obtained weight value as the convolution kernel to obtain the dynamic convolution kernel of the tracking target.

[0170] S2: Training the object detection model

[0171] Train the target detection model after integrating the dynamic convolution kernel to obtain the initial target detection model;

[0172] Step 2-1: Select RetinaNet as the pre-trained model. Before the model convolution operation, call the dynamic convolution module in step 1-6 to obtain the dynamic convolution kernel. Replace the original convolution kernel with the tracking target dynamic convolution kernel. Use this model to detect the initial sample set annotated in step 1-5. Process each sample to obtain the target detection frame. And target detection score pre ,i, where i is the sample number;

[0173] Step 2-2: Given the number of samples is M init Calculate the target detection loss value, which includes two loss values L pre,box and L pre,label ;

[0174] Step 2-2-1: Calculate the loss of the target detection box and the annotated bounding box:

[0175]

[0176] The calculation formula of SmoothL1Loss is:

[0177]

[0178] * indicates digital multiplication;

[0179] Step 2-2-2: Calculate the target prediction probability score pre,i and the true probability score gt Losses:

[0180]

[0181] Step 2-2-3: The total loss is the sum of the bounding box loss and the prediction score loss:

[0182] L pre =L pre,box +L pre,label

[0183] Step 2-3: Use the total loss calculated in step 2-3-3 to train the model using backpropagation to obtain the initial target detection model.

[0184] S3: Constructing dynamic convolution kernels for difficult targets

[0185] like Figure 3 As shown in the figure, the initial target detection model is used to obtain the target detection frame, the target detection loss value is calculated, the difficult sample is obtained according to the threshold of the target detection loss value, the Key and Query features of the difficult target are extracted, and the dynamic convolution kernel of the difficult target is calculated;

[0186] Step 3-1: Use the initial target detection model obtained in step S2-3 to detect the data annotated in steps 1-5 to obtain the target detection frame;

[0187] Step 3-2: Get the target prediction probability score init,i ;

[0188] Step 3-3: Set the prediction score threshold for difficult samples;

[0189] Step 3-4: Samples with prediction scores less than the threshold are considered difficult samples;

[0190] Step 3-5: Obtain difficult sample set: Diff = <Frama diff ,Box diff ,Labeldiff >, where Frame diff Indicates difficult frames, Box diff Indicates the difficult sample bounding box, Label diff Indicates the difficult sample label;

[0191] Step 3-6: Create a dynamic convolution module to obtain the dynamic convolution kernel before the model convolution operation and apply it to the convolution operation. At the current layer, the key feature and query feature are obtained from the pixels of the video frame;

[0192] Step 3-6-1: Extract the tracking target area: At different resolutions, the corresponding pixels are different. The pixels within the h×w range of the target center coordinates are used as key features: H and W are the height resolution and width resolution of the current layer video frame. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1};

[0193] Step 3-6-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region;

[0194]

[0195] Among them, C diff is the feature dimension;

[0196] Step 3-6-3: Perform the above operation on all pixels to obtain the weight set

[0197] Step 3-6-4: Use the obtained weight value as the convolution kernel to obtain the difficult target dynamic convolution kernel;

[0198] S4: Construct the interference target dynamic convolution kernel.

[0199] like Figure 4 As shown in the figure, the interference sample category is determined, the interference sample set is obtained from the ImageNet dataset, the interference sample set is expanded, the Key and Query features of the interference target are extracted, and the dynamic convolution kernel of the interference target is calculated;

[0200] Step 4-1: Determine the interference sample categories as flying birds, leaves, clothes, and quilts;

[0201] Step 4-2: Obtain an interference target dataset containing flying birds, leaves, clothes, and quilts from the ImageNet dataset;

[0202] Step 4-3: Data preprocessing and data enhancement;

[0203] Step 4-3-1: Scale the images in the interference target dataset to a resolution of 256x256 by normalizing the size;

[0204] Step 4-3-2: Randomly flip the image within the range of 0-180°;

[0205] Step 4-3-3: Set the target bounding box position, put the bounding box into the image, and synthesize the picture to get the Frame disturb ;

[0206] Step 4-4: Obtain interference sample set: Disturb= <Frame disturb ,Box disturb ,Label disturb >, where Frame disturb Indicates interference frame, Box disturb Represents the bounding box, Label disturb Indicates a label;

[0207] Step 4-5: Create a dynamic convolution module to obtain the dynamic convolution kernel before the model convolution operation and apply it to the convolution operation. At the current layer, the key feature and query feature are obtained from the pixels of the video frame;

[0208] Step 4-5-1: Extract the tracking target area: At different resolutions, the corresponding pixels are different. The pixels within the h×w range of the target center coordinates are used as key features: H and W are the height resolution and width resolution of the current layer video frame. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1};

[0209] Step 4-5-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region;

[0210]

[0211] Among them, C disturb is the feature dimension;

[0212] Step 4-5-3: Perform the above operation on all pixels to obtain the weight set

[0213] Step 4-5-4: Use the obtained weight value as the convolution kernel to obtain the dynamic convolution kernel of the interference target.

[0214] S5: Training an object detection model with dynamic convolution kernels for multiple object categories

[0215] like Figure 5 As shown, the initial target detection model is trained to obtain the target detection model of multi-target dynamic convolution kernel;

[0216] Step 5-1: Fusion of the initial sample set Init, the difficult sample set Diff, and the interference sample set Disturb of the tracking target class to obtain the active sample set for training the active target detection model

[0217] Fuse=<Init,Diff,Disturb> ,

[0218] Fuse= <Frame fuse ,Box Fuse ,Label fuse >

[0219] Frame fuse Indicates fusion frame, Box fuse Represents the bounding box, Label fuse Represents the label, active means obtaining the difficult sample set through the detection score threshold;

[0220] Step 5-2: Process the active sample set and use the initial target detection model obtained in steps 2-3 for difficult images, replacing the convolution kernels of the model Stage 2 to Stage 5 with dynamic convolution kernels;

[0221] Step 5-2-1: Using the active sample set obtained in step 5-1, the initial resolution of the sample input model is 640×480;

[0222] Step 5-2-2: Stage 1 uses a normal convolution kernel and downsampling to process the sample, obtaining an output resolution of 320×240;

[0223] Step 5-2-3: Stage 2 uses dynamic convolution kernels to extract sample features from the active sample set. Specifically, the tracking sample dynamic convolution kernels obtained in steps 1-6 are used to extract features, the difficult sample dynamic convolution kernels obtained in steps 3-6 are used to extract features, and the interference sample dynamic convolution kernels obtained in steps 4-5 are used to extract features. Then, the maximum pooling is used for downsampling to obtain the result Feature stage3 =MaxPool(DynamicConv(Frame fuse )), the processed resolution is 160×120;

[0224] Step 5-2-4: Stage 3 uses dynamic convolution kernels to extract sample features. Specifically, the tracking sample dynamic convolution kernels obtained in steps 1-6 are used to extract features, the difficult sample dynamic convolution kernels obtained in steps 3-6 are used to extract features, and the interference sample dynamic convolution kernels obtained in steps 4-5 are used to extract features. Then, the maximum pooling is used for downsampling to obtain the result Feature. stage4 =MaxPool(DynamicConv(Feature stage3 ), the resolution after processing is 80×60;

[0225] Step 5-2-5: Stage 4 uses dynamic convolution kernels to extract sample features from the active sample set. Specifically, the tracking sample dynamic convolution kernels obtained in steps 1-6 are used to extract features, the difficult sample dynamic convolution kernels obtained in steps 3-6 are used to extract features, and the interference sample dynamic convolution kernels obtained in steps 4-5 are used to extract features. Then, the maximum pooling is used for downsampling to obtain the result Feature. stage5 =MaxPool(DynamicConv(Feature stage4 ), the resolution after processing is 40×30;

[0226] Step 5-2-6: Feature by deconvolution stage5 Upsampling, and feature after convolution with 1*1 convolution kernel stage4 Residual connection to obtain fusion features

[0227]

[0228] Step 5-2-7: Deconvolution Upsampling, and feature after convolution with 1*1 convolution kernel stage3 Residual connection to obtain fusion features

[0229]

[0230] Step 5-2-8: Deconvolution Upsampling, and feature after convolution with 1*1 convolution kernel stage2 Residual connection to obtain fusion features

[0231]

[0232] Step 5-3: Use the classification and bounding box sub-networks to process the features obtained in steps 5-3-4, 5-3-5, and 5-3-6 respectively to obtain the target detection frame And target detection score fuse,i , where i is the sample number;

[0233] Step 5-4: Given the number of samples is M fuse Calculate the target detection loss value, which includes two loss values L fuse,box and L fuse,label ;

[0234] Step 5-4-1: Calculate the loss of the target detection box and the annotated bounding box:

[0235]

[0236] The SmoothL1Loss calculation formula is:

[0237]

[0238] * indicates digital multiplication, x1 is the detected coordinate value, and x2 is the real coordinate value;

[0239] Among them, M fuse is the number of samples in the initial sample set;

[0240] Step 5-4-2: Calculate the target prediction probability score fuse,i and the true probability score gt Losses:

[0241]

[0242] Among them, M fuse is the number of samples in the fusion sample set;

[0243] Step 5-4-3: The total loss value is the sum of the bounding box loss and the prediction score loss:

[0244] L fuse =L fuse,box +L fuse,label

[0245] Step 5-5: Use the total loss calculated in step 5-4-3, use backpropagation to train the model, and obtain the target detection model with multi-target dynamic convolution kernel.

[0246] S6: Test video of high-altitude object tracking

[0247] like Figure 6As shown in the figure, the target detection model with a multi-target dynamic convolution kernel is used to detect the test video frame to obtain the target detection frame, the tracking frame is predicted through the state transition matrix, and the tracking frame is matched with the detection frame using an algorithm. The same operation is performed on subsequent frames to obtain the motion trajectory of the high-altitude parabolic target.

[0248] Step 6-1: Sample the test video, parse the video data into continuous video frame data, and obtain a video frame set Where i represents the video frame number;

[0249] Step 6-2: Use the active sample target detection model obtained in step S5-5 to detect the first frame frame1 of the video frame set obtained in step S6-1 test , get the target detection frame, get the width and height of the target detection frame;

[0250] Step 6-3: Calculate the first frame frame1 test The position and velocity of each target in the 1st frame are obtained by

[0251] Obj 1,i =[cenx 1,i ,ceny 1,i ,vx 1,i ,vx 1,i ] T

[0252] Among them cenx 1,i ,ceny 1,i Respectively represent the x-axis coordinate and y-axis coordinate of the target center point, vx 1,i ,vx 1,i Represent the x-direction speed and y-direction speed of the target center point respectively;

[0253] Step 6-4: When the time is short and the range of motion is small, the linear motion of the target in the x and y directions can be used to approximate the falling motion. The state transition matrix is used to describe the temporal relationship between the state changes from the first frame to the second frame. Δt = 1 / 32 seconds represents the time interval between the two frames. The state transition matrix Trans can be expressed as

[0254]

[0255] Step 6-5: Use the target state in step 6-3 and the state transition matrix obtained in step 6-4 to predict the state vector of the target at the next moment

[0256] Obj 2,i =Trans×Obj 1,i

[0257] Where × represents matrix multiplication;

[0258] Using the above target state and the height and width of the detection frame in step 6-2, obtain the target tracking frame

[0259] Step 6-6: Repeat steps 6-3 to 6-5 to predict n tracking boxes of n targets in the second frame;

[0260] Step 6-7: Use the active sample target detection model obtained in step S5-5 to detect the second frame frame2 of the video frame set obtained in step S6-1 test , get the target detection frame, get the width and height of the target detection frame;

[0261] Steps 6-8: Track the coordinates of frame i based on the second frame The coordinates of the second frame detection box j Calculate the position relationship score between the two:

[0262] Step 6-8-1: Calculate the intersection area:

[0263]

[0264] InterArea i,j =InterWidth i,j xInterHeight i,j

[0265] Step 6-8-2: Calculate the union area:

[0266]

[0267] UnionArea i,j =AreaI+AreaJ-InterArea i,j

[0268] Among them, x represents the multiplication of numbers;

[0269] Step 6-8-3: Use the intersection ratio as the position relationship score:

[0270] IoU i,j =InterArea i,j / UnionArea i,j

[0271] IOU i,j The larger the value, the higher the overlap between the detection frame and the tracking frame. Selecting the maximum value will achieve the best match between the detection frame and the tracking frame.

[0272] Step 6-9: Select the n target tracking frames of the first frame in turn and traverse the m target detection frames of the second frame. Repeat steps 6-8 to obtain n*m position relationship scores and fill them into the n*m matrix IoU. 1,2 ,

[0273]

[0274] IoU i,j Represents the positional relationship score between the i-th tracking frame in the first frame and the j-th detection frame in the second frame;

[0275] Steps 6-10: Perform the Hungarian algorithm on the position relationship matrix IoU:

[0276] Step 6-10-1: For each row of the position relationship matrix IoU, find the minimum value, and then subtract the minimum value from all elements in the row;

[0277] Step 6-10-2: For each column of the position relationship matrix IoU, find the minimum value, and then subtract the minimum value from all elements in that column;

[0278] Step 6-10-3: For each row, find a zero element and mark the row where the zero element is located as the selected row;

[0279] Step 6-10-4: In the selected row, find a zero element that does not intersect with the previously selected column, and mark the column where the zero element is located as the selected column;

[0280] Step 6-10-5: Select the sum of the number of rows and columns as n, match the tracking boxes and detection boxes corresponding to the rows and columns, and obtain the optimal matching solution that maximizes the total IOU score;

[0281] Step 6-11: Update the tracking frame with the detection frame successfully matched in step 6-10-5;

[0282] Step 6-12: Repeat steps 6-2 to 6-11 for subsequent frames of the test video until the end of the video frame to obtain the motion trajectory of the high-altitude parabolic target.

Claims

1. A high-altitude parabolic tracking method using a target dynamic convolution kernel, characterized by: The specific steps include: S1: Constructing dynamic convolution kernel for tracking target Collect high-altitude parabolic video data, mark the target detection box and target category of the video frame, extract the key and query features of the tracking target, and calculate the dynamic convolution kernel of the tracking target; S2: Training the object detection model Train the target detection model after integrating the dynamic convolution kernel to obtain the initial target detection model; S3: Constructing dynamic convolution kernels for difficult targets Use the initial target detection model to obtain the target detection frame, calculate the target detection loss value, obtain difficult samples based on the threshold of the target detection loss value, extract the key and query features of the difficult target, and calculate the dynamic convolution kernel of the difficult target; S4: Constructing dynamic convolution kernel of interference target Determine the interference sample category, obtain the interference sample set from the ImageNet dataset, expand the interference sample set, extract the key and query features of the interference target, and calculate the dynamic convolution kernel of the interference target; S5: Training an object detection model with dynamic convolution kernels for multiple object categories Train the initial target detection model to obtain a target detection model with multi-target dynamic convolution kernels; S6: Test video of high-altitude object tracking The target detection model with a multi-target dynamic convolution kernel is used to detect the test video frame to obtain the target detection frame. The tracking frame is predicted through the state transition matrix. The tracking frame and the detection frame are matched using an algorithm. The same operation is performed on subsequent frames to obtain the motion trajectory of the high-altitude parabolic target.

2. The high-altitude parabolic tracking method using a target dynamic convolution kernel according to claim 1, wherein: Step S1 specifically includes the following steps: Step 1-1: Use the camera to collect video of objects thrown from high altitude; Step 1-2: Sampling the video data into continuous video frame data to obtain a video frame set; Step 1-3: Label the bounding box of the moving object in each video frame and the target center coordinates in is the coordinate of the upper left corner of the bounding box, is the coordinate of the lower right corner of the bounding box; Step 1-4: Mark the category of the moving target in each video frame to obtain the category label of the target; Step 1-5: Obtain the initial sample set: Init= <Frame init ,Box init ,Label init >, where Frame init Represents the frame obtained by merging the bounding box with the label, Box init Represents the initial sample bounding box, Label init represents the initial sample label; Step 1-6: Create a dynamic convolution module. At the current layer, the key features and query features are obtained from the pixels of the video frame: Step 1-6-1: Extract the tracking target area: At different resolutions, the corresponding pixels are different, and the target center point coordinates are The pixels in the h×w range are used as key features: H and W are the height resolution and width resolution of the current layer video frame respectively. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1}; Step 1-6-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region, and use the Softmax function as the weight for the similarity score: Among them, C track is the feature dimension; Step 1-6-3: Perform the above operation on all pixels to obtain the weight set Step 1-6-4: Use the obtained weight value as the convolution kernel to obtain the dynamic convolution kernel of the tracking target.

3. The high-altitude parabolic tracking method using a target dynamic convolution kernel according to claim 2, wherein: Step S2 specifically includes the following steps: Step 2-1: Select RetinaNet as the pre-trained model. Before the model convolution operation, call the dynamic convolution module in steps 1-6 to obtain the dynamic convolution kernel. Replace the original convolution kernel with the tracking target dynamic convolution kernel. Use this model to detect the initial sample set annotated in steps 1-5. Process each sample to obtain the target detection frame. And target detection score pre,i , where i is the sample number, is the coordinate of the upper left corner of the target detection box, The coordinates of the lower right corner of the target detection box; Step 2-2: Given the number of samples is M init Calculate the target detection loss value, which includes two loss values L pre,box and L pre,label ; Step 2-2-1: Calculate the loss of the target detection box and the annotated bounding box: The calculation formula of Smooth L1 Loss is: * indicates digital multiplication, x1 is the detected coordinate value, and x2 is the real coordinate value; Step 2-2-2: Calculate the target prediction probability score pre,i and the true probability score gt Losses: Step 2-2-3: The total loss is the sum of the bounding box loss and the prediction score loss: L pre =L pre,box +L pre,label Step 2-3: Use the total loss calculated in step 2-3-3 to train the model using backpropagation to obtain the initial target detection model.

4. The high-altitude parabolic tracking method using a target dynamic convolution kernel according to claim 3, wherein: Step S3 specifically includes the following steps: Step 3-1: Use the initial target detection model obtained in step S2-3 to detect the data annotated in steps 1-5 to obtain the target detection frame; Step 3-2: Get the target prediction probability score init,i ; Step 3-3: Set the difficult sample prediction score threshold; Step 3-4: Samples with prediction scores less than the threshold are considered difficult samples; Step 3-5: Obtain difficult sample set: Diff = <Frame diff ,Box diff ,Label diff >, where Frame diff Indicates difficult frames, Box diff Indicates the difficult sample bounding box, Label diff Indicates the difficult sample label; Step 3-6: Create a dynamic convolution module. At the current layer, the key features and query features are obtained from the pixels of the video frame. Step 3-6-1: Extract the tracking target area: At different resolutions, the corresponding pixels are different. The pixels within the h×w range of the target center coordinates are used as key features: H and W are the height resolution and width resolution of the current layer video frame respectively. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1}; Step 3-6-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region; Among them, C diff is the feature dimension; Step 3-6-3: Perform the above operation on all pixels to obtain the weight set Step 3-6-4: Use the obtained weight value as the convolution kernel to obtain the difficult target dynamic convolution kernel.

5. The high-altitude parabolic tracking method using a target dynamic convolution kernel according to claim 4, characterized in that: Step S4 specifically includes the following steps: Step 4-1: Determine the interference sample categories as flying birds, leaves, clothes, and quilts; Step 4-2: Obtain an interference target dataset containing flying birds, leaves, clothes, and quilts from the ImageNet dataset; Step 4-3: Data preprocessing and data enhancement; Step 4-3-1: Scale the images in the interference target dataset to a resolution of 256x256 by normalizing the size; Step 4-3-2: Randomly flip the image within the range of 0-180°; Step 4-3-3: Set the target bounding box position, put the bounding box into the image, and synthesize the picture to get the Frame disturb ; Step 4-4: Obtain interference sample set: Disturb= <Frame disturb ,Box disturb ,Label disturb >, where Frame disturb Indicates interference frame, Box disturb Represents the bounding box, Label disturb Indicates a label; Step 4-5: Create a dynamic convolution module. At the current layer, the key features and query features are obtained from the pixels of the video frame. Step 4-5-1: Extract the tracking target area: At different resolutions, the corresponding pixels are different. The pixels within the h×w range of the target center coordinates are used as key features: H and W are the height resolution and width resolution of the current layer video frame respectively. In the area near the target center point, the pixels adjacent to the center point are used as query features: The coordinate range of adjacent pixels is u∈{h-1,h,h+1},v∈{w-1,w,w+1}; Step 4-5-2: Use dot product multiplication to calculate the similarity between the key feature and all query features in the region; Among them, C disturb is the feature dimension; Step 4-5-3: Perform the above operation on all pixels to obtain the weight set Step 4-5-4: Use the obtained weight value as the convolution kernel to obtain the dynamic convolution kernel of the interference target.

6. The high-altitude parabolic tracking method using a target dynamic convolution kernel according to claim 5, characterized in that: Step S5 specifically includes the following steps: Step 5-1: Fusion of the initial sample set Init, the difficult sample set Diff, and the interference sample set Disturb of the tracking target class to obtain the active sample set for training the active target detection model Fuse=<Init,Diff,Disturb> , Fuse=<Frame fuse ,Box Fuse ,Label fuse > Frame fuse Indicates fusion frame, Box fuse Represents the bounding box, Label fuse Represents the label, active means obtaining the difficult sample set through the detection score threshold; Step 5-2: Process the active sample set, use the initial target detection model obtained in step 2-3 for the difficult image, and replace the convolution kernels of the model Stage 2 to Stage 5 with dynamic convolution kernels; Step 5-2-1: Using the active sample set obtained in step 5-1, the initial resolution of the sample input model is 640×480; Step 5-2-2: Stage 1 uses a normal convolution kernel and downsampling to process the sample, obtaining an output resolution of 320×240; Step 5-2-3: Stage 2 uses dynamic convolution kernel to extract sample features in the active sample set, uses the tracking sample dynamic convolution kernel obtained in steps 1-6 to extract features for the tracking sample, uses the difficult sample dynamic convolution kernel obtained in steps 3-6 to extract features for the difficult sample, and uses the interference sample dynamic convolution kernel obtained in steps 4-5 to extract features for the interference sample, and then uses maximum pooling for downsampling to obtain the result Feature stage3 =MaxPool(DynamicConv(Frame fuse )), the processed resolution is 160×120; Step 5-2-4: Stage 3 uses dynamic convolution kernel to extract sample features. The tracking sample dynamic convolution kernel obtained in steps 1-6 is used to extract features. The difficult sample dynamic convolution kernel obtained in steps 3-6 is used to extract features. The interference sample dynamic convolution kernel obtained in steps 4-5 is used to extract features. Then, the maximum pooling is used for downsampling to obtain the result. Feature stage4 =MaxPool(DynamicConv(Feature stage3 ), the resolution after processing is 80×60; Step 5-2-5: Stage 4 uses dynamic convolution kernel to extract sample features in the active sample set, uses the tracking sample dynamic convolution kernel obtained in steps 1-6 to extract features for the tracking sample, uses the difficult sample dynamic convolution kernel obtained in steps 3-6 to extract features for the difficult sample, and uses the interference sample dynamic convolution kernel obtained in steps 4-5 to extract features for the interference sample, and then uses maximum pooling for downsampling to obtain the result Feature stage5 =MaxPool(DynamicConv(Feature stage4 ), the resolution after processing is 40×30; Step 5-2-6: Feature by deconvolution stage5 Upsampling, and feature after convolution with 1*1 convolution kernel stage4 Residual connection to obtain fusion features Step 5-2-7: Deconvolution Upsampling, and feature after convolution with 1*1 convolution kernel stage3 Residual connection to obtain fusion features Step 5-2-8: Deconvolution Upsampling, and feature after convolution with 1*1 convolution kernel stage2 Residual connection to obtain fusion features Step 5-3: Use the classification and bounding box sub-networks to process the features obtained in steps 5-3-4, 5-3-5, and 5-3-6 respectively to obtain the target detection frame And target detection score fuse,i , where i is the sample number, is the coordinate of the upper left corner of the target detection box, The coordinates of the lower right corner of the target detection box; Step 5-4: Given the number of samples is M fuse Calculate the target detection loss value, which includes two loss values L fuse,box and L fuse,label ; Step 5-4-1: Calculate the loss of the target detection box and the annotated bounding box: The Smooth L1 Loss calculation formula is: * indicates digital multiplication, x1 is the detected coordinate value, and x2 is the real coordinate value; Among them, M fuse is the number of samples in the initial sample set; Step 5-4-2: Calculate the target prediction probability score fuse,i and the true probability score gt Losses: Among them, M fuse is the number of samples in the fusion sample set; Step 5-4-3: The total loss value is the sum of the bounding box loss and the prediction score loss: L fuse =L fuse,box +L fuse,label Step 5-5: Use the total loss calculated in step 5-4-3, use backpropagation to train the model, and obtain the target detection model with multi-target dynamic convolution kernel.

7. The high-altitude parabolic tracking method using a target dynamic convolution kernel according to claim 6, characterized in that: Step S6 specifically includes the following steps: Step 6-1: Sample the test video, parse the video data into continuous video frame data, and obtain a video frame set Where i represents the video frame number, represents the i-th frame of a video frame set; Step 6-2: Use the active sample target detection model obtained in step S5-5 to detect the first frame frame1 of the video frame set obtained in step S6-1 test , get the target detection frame, get the width and height of the target detection frame; Step 6-3: Calculate the first frame frame1 test The position and velocity of each target in the 1st frame are obtained by Obj 1,i =[cenx 1,i ,pressure 1,i ,vx 1,i ,vx 1,i ] T where cenx 1,i ,ceny 1,i Respectively represent the x-axis coordinate and y-axis coordinate of the target center point, vx 1,i ,vx 1,i Represent the x-direction speed and y-direction speed of the target center point respectively; Step 6-4: When the time is short and the range of motion is small, the linear motion of the target in the x and y directions is used to approximate the falling motion. The state transition matrix is used to describe the temporal state change relationship from the first frame to the second frame. Δt = 1 / 32 seconds represents the time interval between the two frames. The state transition matrix Trans is expressed as Step 6-5: Use the target state in step 6-3 and the state transition matrix obtained in step 6-4 to predict the state vector of the target at the next moment Obj 2,i =Trans×Obj 1,i Where × represents matrix multiplication; Using the above target state and the height and width of the detection frame in step 6-2, obtain the target tracking frame is the coordinate of the upper left corner of the target tracking box, is the coordinate of the lower right corner of the target tracking box; Step 6-6: Repeat steps 6-3 to 6-5 to predict n tracking boxes of n targets in the second frame; Step 6-7: Use the active sample target detection model obtained in step S5-5 to detect the second frame frame2 of the video frame set obtained in step S6-1 test , get the target detection frame, get the width and height of the target detection frame; Steps 6-8: Track the coordinates of frame i based on the second frame The coordinates of the second frame detection box j Calculate the position relationship score between the two: Step 6-8-1: Calculate the intersection area: InterArea i,j =InterWidth i,j x InterHeight i,j Step 6-8-2: Calculate the union area: UnionArea i,j =AreaI+AreaJ-InterArea i,j Among them, x represents the multiplication of numbers; Step 6-8-3: Use the intersection ratio as the position relationship score: I love you ,j =InterArea i,j / UnionArea i,j IOU i,j The larger the value, the higher the overlap between the detection frame and the tracking frame. Selecting the maximum value will achieve the best match between the detection frame and the tracking frame. Step 6-9: Select the n target tracking frames of the first frame in turn and traverse the m target detection frames of the second frame. Repeat steps 6-8 to obtain n*m position relationship scores and fill them into the n*m matrix IoU. 1,2 , IoU i,j Represents the positional relationship score between the i-th tracking frame in the first frame and the j-th detection frame in the second frame; Steps 6-10: Perform the Hungarian algorithm on the position relationship matrix IoU: Step 6-10-1: For each row of the position relationship matrix IoU, find the minimum value, and then subtract the minimum value from all elements in the row; Step 6-10-2: For each column of the position relationship matrix IoU, find the minimum value, and then subtract the minimum value from all elements in that column; Step 6-10-3: For each row, find a zero element and mark the row where the zero element is located as the selected row; Step 6-10-4: In the selected row, find a zero element that does not intersect with the previously selected column, and mark the column where the zero element is located as the selected column; Step 6-10-5: Select the sum of the number of rows and columns as n, match the tracking boxes and detection boxes corresponding to the rows and columns, and obtain the optimal matching solution that maximizes the total IOU score; Step 6-11: Update the tracking frame with the detection frame successfully matched in step 6-10-5; Step 6-12: Repeat steps 6-2 to 6-11 for subsequent frames of the test video until the end of the video frame to obtain the motion trajectory of the high-altitude parabolic target.

Images