A water surface floating object target detection and tracking method based on space-time information fusion

Abstract

A water surface floating object target detection and tracking method based on space-time information fusion. First, obtain the water surface floating object video dataset under different time, place, season and weather conditions. Second, input the video data in single frame detection, and obtain the water surface floating object detection frame of the current frame through the improved SSD detection algorithm. Third, in multi-frame filtering, the target detection frame of the first frame of the video is taken as the input of the improved KCF tracking algorithm for target tracking, and the water surface floating object target tracking frame under the current video frame is obtained. Finally, after completing the tracking of a fixed number of frames in the video frame, the SSD detection algorithm is introduced again in the next video frame to obtain the new detection frame of the floating object target, and the coincidence degree of the new detection frame and the old tracking frame is compared by adopting the candidate frame selection strategy to judge the tracking. If the coincidence degree meets the condition, the current floating object target is continued to be tracked; if the coincidence degree does not meet the condition, the floating object target is regarded as a new target and is output, and participates in the initialization tracking of the new target by the KCF tracking algorithm. The application judges whether the detection and tracking information belongs to a new floating object target through the space-time fusion strategy, reduces the false detection rate and the missing detection rate of the target object, and improves the precision of the floating object detection and tracking.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of machine learning, target detection and tracking, and relates to a water surface floating object target detection and tracking method based on spatiotemporal information fusion. BACKGROUND

[0002] Floating objects are an important source of pollution of rivers and lakes, seriously damaging the water surface landscape and ecological environment. Visualized supervision of rivers and lakes through cameras is implemented to improve the appearance of rivers and lakes. At present, the detection and tracking technology of floating objects is affected by the problems of difficulty in sampling floating objects, morphological changes, variable environment and frequent occlusion, etc. Especially, many noise problems such as water wave disturbance, dynamic light and shadow and strong light reflection make it difficult for existing methods to meet the actual management needs of water surface floating objects. Therefore, finding an efficient and fast water surface floating object detection and tracking method is one of the problems to be solved in water pollution control.

[0003] At present, the detection technology of water surface floating objects mainly focuses on two aspects of traditional image processing and deep learning method. The traditional image processing is to identify the foreground and background features of the image and the filtering theory, and the characteristic is fast detection speed, but the false detection and missed detection rate is high, which cannot meet the robustness requirement. With the rapid development of artificial intelligence and deep learning technology, the target feature information and model training of floating objects are extracted through multi-layer convolutional neural network, realizing the target detection in complex water surface environment. Among them, the SSD model becomes the mainstream method applied to the detection and identification of water surface floating objects with its own advantages of precision and speed, but when the floating object target scale is small, the motion is slow and the background complexity changes greatly, the detection error rate rises continuously, and the algorithm needs the support of high-power experimental equipment, which is difficult to transplant to embedded devices. At present, the water surface floating object detection technology is mostly single frame detection, and the time correlation and floating object motion state information between video frames have not been fully utilized, which may cause target missed detection and other problems when processing the video.

[0004] Compared with target detection, target tracking has more stringent requirements in real-time. The floating object tracking technology is mainly divided into traditional target tracking method, deep learning algorithm and kernel correlation filter. The particle filter, Kalman filter, optical flow algorithm and intelligent particle filter in the traditional target tracking technology have a significant performance decline when tracking in a complex background. The target tracking algorithm based on deep learning extracts the features of the target by using a multi-layer convolutional neural network, replacing the original manual features used in the tracking algorithm framework, such as ECO, MDNet, SANet, BranchOut, DaSiamRPN and SPM-Tracker. However, the process of extracting features from the deep network often requires multiple operations, which reduces the speed of the tracking algorithm and makes it difficult to meet the real-time requirements of the monitoring system. The correlation filter tracking algorithm based on time-frequency transform is computationally efficient and has received widespread attention in recent years. The kernel correlation filter algorithm improves the performance of floating object tracking in complex river scenes through classifier training, target detection and model updating, but the dynamic changes in the size of the floating object often cause tracking drift, and cannot solve the problem of size changes caused by the near-far motion state of the floating object in the fixed monitoring scene. The above target tracking algorithms mainly process the target based on the time information in the video sequence, and the accuracy of determining the specific position and size of the target is not high. Errors may accumulate during the tracking process, causing target drift and leading to tracking failure.

[0005] Based on the actual demand for floating object detection and tracking on the water surface, the existing research results at home and abroad have problems in accuracy, speed and influence of complex water surface environment, which cannot meet the demand for floating object detection and tracking on the water surface. Based on the continuous expansion of deep learning technology, the present application proposes a water surface floating object target detection and tracking method based on spatiotemporal information fusion, which achieves accurate detection and tracking of floating objects on the water surface. In the single-frame detection stage, the spatial information of small-scale floating objects is obtained by deleting the deep low-resolution detection layer in the SSD network and enhancing the shallow high-resolution feature layer to solve the problem of small-scale floating object detection; in the multi-frame filtering stage, the correlation filter tracking is used to calculate the time information correlation of the gradient features between video frames to reduce the missed detection rate; in the information fusion stage, the information obtained by the improved SSD detection network and the kernel correlation filter algorithm is fused through feature comparison to ensure the speed and accuracy of detection and tracking. SUMMARY

[0006] The present application aims to provide a water surface floating object target detection and tracking method based on spatiotemporal information fusion, which aims to solve the problems of accuracy, speed and scale of existing target detection and tracking algorithms, and reduce the influence of external light, obstacles and deformation on target detection and tracking.

[0007] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is:

[0008] A water surface floating object target detection and tracking method based on space-time information fusion, comprising the following steps:

[0009] S1: collecting water surface floating object video data sets at different times, places, seasons and weather conditions through a camera;

[0010] S2: inputting the water surface floating object video data in single-frame detection, and obtaining the water surface floating object target detection frame of the current frame through an improved SSD detection algorithm; specifically:

[0011] The improved SSD target detection algorithm mainly adjusts the structure of the deep low-resolution detection layer and the shallow high-resolution detection layer. First, since the detection layer with a resolution of 5*5 and below cannot extract important features of small-scale floating object targets, the technical solution deletes the detection layers with a resolution of 5*5, 3*3 and 1*1 in the SSD network; second, the shallow high-resolution feature layer is enhanced by using the feature summation (Add) method. First, the feature layer with a resolution of 76*76 is added to the feature layer F1 with a resolution of 38*38, 1*1 point convolution is used to reduce the dimension of F1, and upsampling is performed to obtain F0 a , to ensure that F1 and F0 a have the same resolution and channel number; second, F0 a layer is added to F0 layer pixel by pixel, and feature fusion is obtained through convolution layer smoothing processing to obtain F0 o . The improved SSD detection algorithm for obtaining the video target detection frame mainly includes the following steps:

[0012] (2.1) using the feature pyramid network in the improved SSD target detection algorithm to generate default boxes with different aspect ratios and the same area on video frame feature maps of different scales.

[0013] (2.2) input the default boxes in step (2.1) as abstract features into the convolution predictor in the improved SSD target detection algorithm for training, and use the trained convolution predictor for floating object target position offset prediction and classification to obtain the target detection frame.

[0014] (2.3) in all target detection frames obtained in step (2.2), delete the detection frames with a confidence lower than 0.7, and use the non-maximum suppression method to delete redundant and repeated detection frames, and through twice screening, all target detection frames of the water surface floating object in the i-th frame (i=1, 2, 3……X) of the video are obtained.

[0015] S3: input the target detection frame of the first frame in step S2 into the improved KCF tracking algorithm as input for target tracking to obtain the water surface floating object tracking frame under the current video frame; specifically:

[0016] The improved KCF tracking algorithm is composed of position estimation and scale estimation. First, the present technical solution extracts the oriented gradient fast histogram (FHOG) features of the floating object target, trains KCF to obtain a feature response map to determine the position information of the target object; second, the present technical solution adopts a pyramid sampling scale estimation strategy, performs pyramid multi-scale sampling around the position of the floating object, adopts an image training scale filter to determine the best scale information of the floating object. The improved KCF tracking algorithm for obtaining a video target tracking box mainly includes the following steps:

[0017] (3.1) Obtain N water surface floating object detection boxes in the first frame of video frame through step S2, first delete the target detection boxes with a confidence lower than 0.7 in the N water surface floating object detection boxes, and then delete the detection boxes with high redundancy and repeated detection again by using the non-maximum suppression algorithm to obtain M target detection boxes.

[0018] (3.2) Input the coordinate values of the M floating object detection boxes in step (3.1) into the improved KCF tracking algorithm, initialize the improved KCF position filter template and scale filter template, construct a loop matrix, and train the improved KCF position filter by using the FHOG features. Then construct the improved KCF scale filter, and select the best scale of the floating object target near the position generated by the position filter by using the scale adaptive strategy.

[0019] (3.3) Set the fixed frame number of video frame tracking as T (T < X), according to the fixed frame number, repeat the above steps (3.1)-(3.3) until the water surface floating object tracking box under the Tth frame is obtained.

[0020] S4: After completing the tracking of the fixed frame number T in the video frame in the information fusion, repeat step S2 to obtain the new detection box of the floating object target in the next frame of video frame, adopt a candidate box selection strategy to compare the coincidence degree of the new detection box and the old tracking box for tracking judgment. If the coincidence degree meets the condition, the current floating object target is continuously tracked; if the coincidence degree does not meet the condition, the floating object target is regarded as a new target and output, participates in the initialization of the improved KCF tracking algorithm, and tracks the new target; specifically:

[0021] (4.1) After tracking the fixed frame number T (T < X) through step S3, the water surface floating object tracking box is obtained;

[0022] (4.2) After completing the tracking of the fixed frame number T, the improved SSD detection algorithm is introduced again in the next frame of video frame to obtain the new detection frame of the floating object target. The overlap degree of the new detection frame and the old tracking frame under step S3 is calculated to determine whether the detection and tracking are the same target. If the overlap degree is less than or equal to 0.4, the floating object in the video frame is set as a new target, the improved KCF algorithm is initialized, and step S3 is repeated; if the overlap degree is greater than 0.4, the new detection frame and the old tracking frame are set as the same target, the confidence of the new detection frame and the normalized response of the old tracking frame are compared, and the candidate frame with the greater confidence is taken as the output.

[0023] (4.3) The total number of video frames is set as X, and the above steps (4.1)-(4.3) are repeated until the tracking frame of the water surface floating object target under the Xth frame is obtained.

[0024] Compared with the prior art, the beneficial effects of the present application are as follows:

[0025] (1) The present application comprises three parts of single-frame detection, multi-frame filtering and space-time information fusion, adopts the improved SSD detection algorithm to obtain the spatial position information of the floating object target, adopts the improved KCF tracking algorithm to obtain the time and motion information of the floating object target, and judges whether the detection and tracking information belong to a new floating object target through the space-time fusion strategy, so as to reduce the false detection rate and the missing detection rate of the target object and improve the precision of the floating object detection and tracking.

[0026] (2) The present application improves the traditional SSD detection algorithm, adjusts the structure of the deep low-resolution detection layer and the shallow high-resolution detection layer, and improves the detection precision of the SSD detection algorithm for small floating object targets.

[0027] (3) The present application improves the traditional KCF tracking algorithm, introduces a pyramid sampling floating object scale estimation algorithm with good robustness, alleviates the problem of tracking performance decline caused by the fact that the KCF algorithm cannot realize scale self-adaptive adjustment, and enables the floating object target to be accurately tracked and positioned when the scale changes and occlusion occur in the tracking process. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a flow chart of a water surface floating object target detection and tracking method based on space-time information fusion.

[0029] Figure 2 is a schematic diagram of the improved SSD detection algorithm.

[0030] Figure 3 is a feature fusion schematic diagram.

[0031] Figure 4 is a schematic diagram of the improved KCF tracking algorithm.

[0032] Figure 5 is a schematic diagram of a spatio-temporal information fusion strategy.

[0033] Figure 6 is a schematic diagram of performance comparison of the method of the present application and other detection and tracking algorithms. Figure 6 (a) is a detection and tracking diagram for the 15th frame; Figure 6 (b) is a detection and tracking diagram for the 232nd frame; Figure 6 (c) is a detection and tracking diagram for the 467th frame; Figure 6 (d) is a detection and tracking diagram for the 23rd frame; Figure 6 (e) is a detection and tracking diagram for the 145th frame; Figure 6 (f) is a detection and tracking diagram for the 379th frame. DETAILED DESCRIPTION

[0034] In order to enable personnel in the technical field to have a clearer understanding of the present application, the specific implementation content and mode of the present application will be described in detail. However, it needs to be made clear that the present application is not limited to the content of the specific implementation mode, and as long as part of the technical changes are within the scope of the claims, all the creations beneficial to the concept of the present application are within the scope of legal protection.

[0035] As shown in Figure 1 , a water surface floating object target detection and tracking method based on spatio-temporal information fusion, the specific implementation process is as follows:

[0036] S1: record complete video frames of water surface floating objects for 70 days through 5 cameras;

[0037] S2: use K-means clustering analysis algorithm to process the width-height ratio of floating object targets in the video frame data set of step S1, obtain n cluster centers as the initial value of the default box of the target detection algorithm. Use the feature pyramid network in the improved SSD target detection algorithm to generate default boxes with different width-height ratios and same areas on different scale feature maps of the video frames, input to the convolution predictor for training, perform floating object target position offset prediction and classification, and obtain the target detection box. By deleting the detection boxes with confidence lower than 0.7, and using the non-maximum suppression method to delete redundant and repeated detection boxes, the detection box of the water surface floating object target in the i-th frame (i=1, 2, 3…X) of the video is obtained. The generation method of the initial default box is as follows:

[0038]

[0039] In the formula, s k represents the area of the default box on the kth feature map, m represents the number of feature maps, s max represents the maximum area of the default box, s min represents the minimum area of the default box, and k represents the kth feature map.

[0040] S3: Take the spatial coordinate information of the detection frame of the i-th frame (i=1, 2, 3…T, T

[0041] ω = (X T X + λI) -1 X T y

[0042]

[0043] In the formula, X represents the diagonalization of the circulant matrix, X T X represents the kernel matrix of the kernel space, λ represents the regularization parameter, y represents the target label vector, and ∧ represents the Fourier transform operation. By using the characteristics of the circulant matrix, the solution of the ridge regression based on the kernel function is obtained as:

[0044] α = (K + λI) -1 y

[0045]

[0046] In the formula, K represents the first row of the kernel matrix K, and the new sample is obtained by extracting the kernel circulant matrix for the next frame of video frame sample z. The kernel circulant matrix of the training sample x and the new sample z is xz, and the response of all test samples in the Fourier domain is:

[0047]

[0048] In the formula, represents the kernel matrix K z the first row of data, K z is the circulant matrix composed of the training sample set x and the sample set to be tested z, ⊙ represents convolution operation, and the position of the selected data with the maximum value in the response map f(z) is the position of the floating object target.

[0049] A scale filter is constructed, and the position f(z) of the floating object target is obtained through convolution operation. The scale filter is trained by using the minimization objective function. The loss function is as follows:

[0050]

[0051] In the formula, represents the i-th feature channel in the scale filter, r represents the sample for training the scale filter, y s represents the expected output value, d s represents the number of feature channels.

[0052] The scale filter is updated, the scale filter is applied to the dimensionality reduction compressed to-be-measured sample, scale correlation scores are calculated, the highest scale correlation score is the final floating object scale, and therefore adaptive estimation of the floating object scale is realized.

[0053] S4: based on the target tracking result of the fixed frame number T, the step S2 is repeated to introduce the improved SSD detection algorithm again to obtain a new detection box S of the floating object target of the next frame of video frames i , the coincidence degree of the new detection box S i and the old tracking box K i under the step S3 is calculated, the Intersection Over Union (IOU) is used as the judging standard of the coincidence degree, and the calculation method is as follows:

[0054]

[0055] In the formula, S i ∩K i represents the intersection of the new detection box S i and the old tracking box K i , S i ∪K i represents the union of the new detection box S i and the old tracking box K i . If the floating object in the video frame is set as a new target and participates in the initialization of the tracking algorithm; if the new detection box S i and the old tracking box K i are set as the same target, the detection box confidence conf(S i ) of the improved SSD detection algorithm is compared with the tracking box confidence conf(K i ) of the improved KCF tracking algorithm, and the candidate box with the larger confidence is taken as the floating object target tracking box.

[0056] The improved SSD target detection algorithm in the application is used as a water surface floating object target detection tool and aims at identifying the type, quantity and scale of the floating object and the like. The main structure of the improved SSD target detection algorithm in the step S2 is shown in Figure 2The first, delete the deep low-resolution detection layer. In the input image with a resolution of 300*300, the water surface floating object contains part of the low pixel and small scale image, which contains less than 10*10 pixels, and the detection difficulty is increased due to low resolution, blurred image, less information and more noise. The resolution of the detection layer F4 is 5*5, which is reduced by 58 times compared with the original input image. The floating object under the detection layer is blurred and the shape and appearance information is greatly reduced. Therefore, the detection layer F4 with a resolution of 5*5 and below cannot extract the important features of the small scale floating object target. The technical solution deletes the detection layer with a resolution of 5*5, 3*3 and 1*1 in the SSD network. The second, enhance the shallow high-resolution detection layer. The technical solution uses a high-resolution feature layer with a resolution of 76*76, but the shallow high-resolution feature layer will lead to insufficient semantic information, which makes it difficult to distinguish the target. Therefore, it is necessary to enhance the feature. The technical solution uses the feature summation (Add) method in feature fusion to enhance the shallow high-resolution feature layer. Compared with the feature concatenation (Concat) fusion method, it has the advantages of saving parameters and calculation amount. The feature fusion process is shown in Figure 3 . First, the feature layer F0 with a resolution of 76*76 is added to the feature layer F1 with a resolution of 38*38, and the F1 layer is reduced in dimension by using 1*1 point convolution and is up-sampled to obtain F0 a , to ensure that F1 and F0 a have the same resolution and channel number; second, the F0 a layer is added to the F0 layer pixel by pixel, and the feature fusion is obtained by convolution layer smoothing processing to obtain F0 o .

[0057] The improved KCF target tracking algorithm main structure in the above step S3 is shown in Figure 4 . The technical solution proposes a scale adaptive floating target tracking method based on multi-feature fusion, including position estimation and scale estimation. In the position estimation stage, the directional gradient fast histogram (FHOG) feature of the floating object is extracted, the KCF is trained to obtain the feature response map to determine the position of the target object; in the scale estimation stage, the pyramid sampling scale estimation strategy is adopted, the pyramid multi-scale sampling is carried out around the position of the floating object, the image training scale filter is adopted to determine the best scale of the floating object.

[0058] The time and space information fusion strategy in the above S4 is shown in Figure 5 . In the fusion, the detection candidate box S i is obtained by improving the SSD detection algorithm in the first frame image to determine the spatial position information of the target object; then the position information of the first frame target is taken as the input of the improved KCF tracking algorithm, and the improved KCF algorithm is used to track the target in the following several frames to obtain the tracking candidate box Kj The running and detection mechanism is tracked after a fixed number of frames, and the accuracy of continuous detection and tracking is ensured by improving the SSD detection algorithm. The detection and tracking of the same floating object is determined by calculating the intersection over union (IOU) of S i and K j .

[0059] In order to verify the feasibility and effectiveness of the present application, the present application is further described below in conjunction with examples.

[0060] The experiment is based on Ubuntu 18.04LTS operating system, the computer is configured as Inter i7 CPU, 32G running memory, and the graphics card is RTX 3080, and the algorithm platform is Python3.8 version of Pytorch 1.10 deep learning framework. The performance of the present technology is evaluated by center location error (CLE), overlap area ratio (OR) and detection and tracking accuracy (DP).

[0061] Figure 6 The performance of the method of the present application is compared with other detection and tracking algorithms. In Figure 6 (a), the traditional SSD detection algorithm can detect large-scale floating objects, but cannot detect small-scale floating objects, and causes false detection. The improved SSD detection algorithm obtains more semantic information by enhancing the shallow high-resolution 76*76 feature layer, detects small-scale floating object targets, and at this time the background of the floating object movement area is relatively simple, and several detection and tracking algorithms achieve relatively accurate tracking; in Figure 6 (b), the floating object enters a large range of reflection and light shadow change area, the water surface light shadow change amplitude is large, the floating object and the reflection area are low in color differentiation, the improved SSD detection algorithm can detect floating object targets, but the tracking frame deviates from the center position of the target. The improved KCF tracking algorithm automatically deletes the floating object deviating from the filter center position based on the historical information of 232 frames, and has a certain robustness to light and shadow. In Figure 6 (c), the area background appears multiple strip reflections, the traditional SSD detection algorithm appears false detection, and two small-scale floating objects are detected as one detection frame, the improved SSD algorithm and the improved KCF algorithm maintain good detection and tracking results, but the tracking accuracy is lower than the fusion algorithm. At the same time, the traditional SSD algorithm in Figure 6 (d), Figure 6 (e) and Figure 6(f) There is a problem of missing detection in the middle, and the improved SSD algorithm optimizes the deep low-resolution and shallow high-resolution feature layer structure, effectively identifying small-scale water surface floating objects. In the initial 23 frames, the tracking results of the improved SSD algorithm and the improved KCF algorithm are relatively small, and the tracking frame is basically overlapped, but the tracking accuracy is lower than that of the fusion algorithm; at 145 frames, the improved SSD detection algorithm misdetected the reflective object as a floating object target, while the improved KCF algorithm avoided tracking frame drift due to adaptive adjustment of the tracking scale; at 379 frames, the degree of strong light reflection in the region is reduced, and the tracking frame is basically overlapped. In a complex water surface environment, the fusion of small-scale detection algorithm and KCF tracking algorithm can achieve better detection and tracking performance.

[0062] The above-described embodiments only express the implementation of the present application, but cannot be interpreted as a limitation on the scope of the patent of the present application. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application.

Claims

1. A method for water surface floating object target detection and tracking based on spatio-temporal information fusion, characterized in that, The method comprises the following steps: Step S1: collecting water surface floating object video data sets of different times, places, seasons and weather conditions through a camera; Step S2: inputting the water surface floating object video data in single-frame detection, and obtaining the water surface floating object target detection frame of the current frame through the improved SSD target detection algorithm; specifically: Step (2.1) generating default frames with different width-height ratios and the same area on different scale feature maps of the video frame by using the feature pyramid network in the improved SSD target detection algorithm; Step (2.2) inputting the default frame in step (2.1) into the convolution predictor in the improved SSD target detection algorithm for training, and using the trained convolution predictor to predict the position offset of the floating object target and classify, to obtain the target detection frame; Step (2.3) deletes the detection frame whose confidence is lower than 0.7 among all the target detection frames obtained in step (2.2), and deletes the redundant and repeated detection frames by using the non-maximum suppression method, and obtains the first target detection frame of the water surface floating object in the video through twice screening. all the target detection frames of the water surface floating object in the frame, wherein i = 1, 2, 3, …, X. Step S3: inputting the target detection frame of the first frame in step S2 into the improved KCF tracking algorithm as input for target tracking to obtain the water surface floating object target tracking frame under the current video frame; Specifically: Step (3.1) obtaining N water surface floating object target detection frames in the first frame of the video frame through step S2, first deleting the target detection frames with a confidence lower than 0.7 in the N water surface floating object target detection frames, and then deleting the detection frames with high redundancy and repeated detection again by using the non-maximum suppression algorithm to obtain M target detection frames; Step (3.2) inputting the coordinate values of the M floating object target detection frames in step (3.1) into the improved KCF tracking algorithm, initializing the improved KCF position filter template and scale filter template, constructing a loop matrix, and training the position filter of the improved KCF by using the FHOG feature; Then, the improved KCF scale filter is constructed, and the best scale of the floating object target is selected near the position generated on the position filter by using the scale adaptive strategy; Step (3.3) sets the fixed number of video frame tracking as T, wherein According to the fixed number of frames, repeat the above steps (3.1)-(3.3) until the water surface floating object tracking frame under the Tth frame is obtained. Step S4: in information fusion, after completing the tracking of a fixed frame number T in the video frame, repeating step S2 to obtain the new detection frame of the floating object target in the next frame of the video frame, and adopting a candidate frame selection strategy to compare the coincidence degree of the new detection frame and the old tracking frame for tracking judgment; If the coincidence degree meets the condition, the current floating object target is continuously tracked; if the coincidence degree does not meet the condition, the floating object target is regarded as a new target and is output, participates in the initialization of the improved KCF tracking algorithm, and tracks the new target; specifically: Step (4.1) is tracked by step S3 after a fixed number of frames T, wherein , obtain the water surface floating object tracking frame; Step (4.2) introduces the improved SSD detection algorithm again to obtain a new detection frame of the floating object in the next video frame after completing the tracking of the fixed frame number T; calculates the overlap degree of the new detection frame and the old tracking frame under step S3 to determine whether the detection and tracking are the same target; if the overlap degree is greater than a preset threshold, the detection and tracking are the same target, and the tracking is continued; if the overlap degree is less than the preset threshold, the detection and tracking are not the same target, and the tracking is terminated, and the new detection frame is taken as the new tracking frame. 0.4, set the floating object in the video frame as a new target, initialize the improved KCF algorithm, and repeat step S3; if the coincidence degree is less than 0.4, set the new detection frame and the old tracking frame as the same target, compare the confidence of the new detection frame with the normalized response of the old tracking frame, and take the candidate frame with the greater confidence as the output. 0.4, set the new detection frame and the old tracking frame as the same target, compare the confidence of the new detection frame with the normalized response of the old tracking frame, and take the candidate frame with the greater confidence as the output. Step (4.3) setting the total number of video frames as X, repeating the above steps (4.1)-(4.3) until the water surface floating object target tracking frame under the Xth frame is obtained.

2. The water surface floating object target detection and tracking method based on space-time information fusion according to claim 1, characterized in that, The step S2 improves the SSD target detection algorithm by adjusting the structure of the deep low-resolution detection layer and the shallow high-resolution detection layer; first, deleting the detection layers with resolutions of 5*5, 3*3 and 1*1 in the original SSD network; second, enhancing the shallow high-resolution feature layer by using a feature summation method; first, performing feature summation processing on the feature layer with a resolution of 76*76 and the feature layer F1 with a resolution of 38*38, performing dimension reduction on the F1 layer by using 1*1 point convolution, and performing upsampling to obtain F0 a , ensuring that F1 and F0 a have the same resolution and channel number; second, adding the F0 a layer to the F0 layer pixel by pixel, and performing feature fusion by convolution layer smoothing processing to obtain F0 o .

3. The water surface floating object target detection and tracking method based on space-time information fusion according to claim 1, characterized in that, The improved KCF tracking algorithm in step S3 is composed of position estimation and scale estimation; first, the oriented gradient histogram feature of the floating object target is extracted, the KCF is trained to obtain the feature response graph to determine the position information of the target object; Second, the pyramid sampling scale estimation strategy is adopted, pyramid multi-scale sampling is performed around the position of the floating object, the scale filter is trained by using the image, and the best scale information of the floating object is determined.

Images