Single-target tracking method and system based on multi-layer feature fusion twin neural network

Abstract

The application provides a single target tracking method and system based on a multi-layer feature fusion twin neural network, comprising: 1) frame extraction is performed on a video to be tracked, and an initial position box of a target to be tracked in a first frame is obtained as a template frame; 2) a search frame region with a size of 255*255 of a next frame is cut with the initial position box of the target as a center; 3) the template frame and the search frame are respectively input into a ResNet-50 backbone feature extraction network to form two branches; 4) Conv3_3, Conv4_6 and Conv_5_3 layer features of the ResNet-50 are extracted; 5) cross-correlation operation is performed on the template frame features as a convolution kernel and the search frame features as input; 6) high response interval features obtained from the search frame are regressed into a bbox coordinate form through an RPN network; and 7) results of regression of different depth features are fused to comprehensively obtain a specific position of movement of the target in a next frame. The application obtains more accurate tracking effect, has stronger foreground and background distinguishing and discriminating capability, and greatly guarantees tracking speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent analysis of monitoring video and target tracking, and particularly relates to a single target tracking method and system based on a multi-layer feature fusion twin neural network. BACKGROUND

[0002] Target tracking, as an important research direction of computer vision, has wide application prospects in many fields of real life. For example, in the field of intelligent monitoring, it can greatly improve the efficiency of monitoring by replacing human labor to continuously track suspicious targets; in the field of human-computer interaction, tracking technology is the key to computer recognition of human gestures, actions, postures and the like; in the field of medical diagnosis, more accurate results can be obtained by using the continuity of the target in geometry and the correlation in time in the sequence images through tracking technology. In addition, tracking technology is also needed for robot vision navigation to calculate the motion trajectory of the photographed object.

[0003] The field of video tracking is a very important and challenging problem. It is difficult to accurately locate the target object due to the changes such as occlusion, deformation and appearance of the tracked object. In addition, in a large number of applications such as automatic driving and video monitoring, real-time requirements are needed. Therefore, designing a real-time and high-precision tracking algorithm has become the main challenge in the field of video tracking.

[0004] Based on the type of target appearance, tracking algorithms can be generally divided into generative and discriminative two categories. In the tracking process of the generative algorithm, a target appearance model is first designed, then the most similar candidate region to the target template is searched in the subsequent video frames, and the image block corresponding to the region is taken as the tracking result in the current frame. The discriminative algorithm regards the tracking problem as a binary classification or regression problem, and learns a classification function to distinguish the target from the surrounding background. Based on the powerful representation ability of deep learning features, convolutional neural networks have been successfully applied in target tracking.

[0005] The current mainstream target tracking algorithm can be divided into two categories: correlation filter-based target tracking algorithm and deep learning-based target tracking algorithm. The correlation filter-based target tracking algorithm has the characteristics of fast speed, but the accuracy is low. The deep learning-based target tracking algorithm needs to use a large-scale dataset to pre-train the model for a long time, but the tracking accuracy is high. Experimental results show that the target tracking algorithm based on twin neural network well balances the speed and accuracy. Under the premise of ensuring the accuracy of the target tracking algorithm, the speed is also improved. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a single target tracking method and system based on a multi-layer feature fusion twin neural network, which can solve the existing problems.

[0007] In order to achieve the above purpose, the application provides a single target tracking method based on a multi-layer feature fusion twin neural network, which comprises the following steps:

[0008] 1) frame extraction is performed on a to-be-tracked video, and an initial position box of a to-be-tracked target in a first frame is obtained as a template frame;

[0009] 2) a search frame region with a size of 255*255 is cropped from a subsequent frame with the initial position box of the target as the center;

[0010] 3) the template frame and the search frame are respectively input into a ResNet-50 backbone feature extraction network to form two branches, and the two branches share network weights;

[0011] 4) Conv3_3, Conv4_6 and Conv_5_3 layer features of the ResNet-50 are extracted to obtain different depth features;

[0012] 5) the template frame features are used as a convolution kernel, and the search frame features are input to perform cross-correlation operation;

[0013] 6) the high response interval features obtained from the search frame are regressed into a bbox coordinate form through a region extraction network;

[0014] 7) the results regressed by the different depth features are fused to obtain the specific position of the target in the next frame.

[0015] Further, in the step 1), frame extraction is performed on an initial video to obtain a set of images of each frame; the initial position of a target to be tracked is framed in the first frame, and is normalized to a size of 127*127 to serve as a template frame for tracking.

[0016] Further, in the step 2), a region with a size of 255*255 is cropped from a subsequent frame with the center coordinates of the template frame as the center to serve as a search frame to be tracked.

[0017] Further, in the step 4), the different depth features include color texture features of the target in the shallow layer and deep semantic level features of the target in the deep layer.

[0018] Further, in the step 5), cross-correlation operation is performed on the template frame and the search frame for the shallow layer, the middle layer and the deep layer features respectively, and the convolution result is a response feature map of the template frame in the search frame.

[0019] Further, in the step 6), the shallow layer, the middle layer and the deep layer response feature maps are respectively input into a region extraction network, and a high response interval is regressed into a bbox coordinate form.

[0020] Further, after step 7), the specific position of the next frame motion of the target is taken as a template frame for the next round of tracking, and the previous tracking process is repeated until the target motion ends, and all tracking results are synthesized to obtain the target motion trajectory.

[0021] Based on the above purpose, the application further provides a single target tracking system based on a multi-layer feature fusion twin neural network, comprising:

[0022] A template frame module is configured to frame the video to be tracked and obtain an initial position box of the target to be tracked in a first frame as a template frame;

[0023] A search frame module is configured to cut a search frame region of 255*255 size in a next frame with the initial position box of the target as the center;

[0024] An input module is configured to input the template frame and the search frame into a ResNet-50 backbone feature extraction network to form two branches, and the two branches share network weights;

[0025] A feature extraction module is configured to extract Conv3_3, Conv4_6, and Conv_5_3 layer features of the ResNet-50 to obtain different depth features;

[0026] A cross-correlation module is configured to perform cross-correlation operation on the template frame features as a convolution kernel and the search frame features as input;

[0027] A regression module is configured to regress the high response interval features obtained from the search frame into a bbox coordinate form through a region extraction network;

[0028] A fusion tracking module is configured to fuse the results of regression of different depth features to comprehensively obtain the specific position of the next frame motion of the target.

[0029] In general, the advantages of the application and the experience brought to the user are as follows:

[0030] I. The method of multi-layer feature fusion is adopted, the color texture features of the shallow layer and the semantic features of the deep layer are considered at the same time, and more robust tracking can be realized.

[0031] II. The RPN regression coordinate mode is used, which greatly reduces the multi-scale sampling problem in tracking, obtains more accurate tracking effect, and has stronger foreground and background discrimination ability.

[0032] III. The model parameter amount is greatly reduced compared with the traditional twin neural network, and the tracking speed is greatly guaranteed. BRIEF DESCRIPTION OF DRAWINGS

[0033] In the drawings, like reference numerals refer to same or similar components throughout the several views. The drawings are not necessarily to scale. It should be understood that the drawings are merely schematic representations, which are illustrate only some embodiments of the application, and therefore are not to be considered limitations of its scope.

[0034] Figure 1 A flow chart of a single target tracking method based on a multi-layer feature fusion twin neural network according to an embodiment of the application is shown.

[0035] Figure 2 A schematic diagram of a tracking framework of the method of the application is shown.

[0036] Figure 3 A schematic diagram of a performance test comparison of the method of the application and other methods on two benchmarks of UAV123 and LaSOT is shown.

[0037] Figure 4 A schematic diagram of a bbox obtained by high response interval regression of shallow, middle and deep features of the application is shown.

[0038] Figure 5 A composition diagram of a single target tracking system based on a multi-layer feature fusion twin neural network according to an embodiment of the application is shown.

[0039] Figure 6 A structural schematic diagram of an electronic device provided by an embodiment of the application is shown.

[0040] Figure 7 A schematic diagram of a storage medium provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0041] The application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0042] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict. The application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0043] Figure 1 A flow chart of a single target tracking method based on a multi-layer feature fusion twin neural network according to an embodiment of the application is shown. As shown in Figure 1 the single target tracking method based on the multi-layer feature fusion twin neural network includes:

[0044] Step one: frame extraction of initial video, get each frame image set. In the first frame, the initial position of the target to be tracked is framed, and is normalized to 127*127 size as the template frame for tracking. The target frame is represented by , where, is the center point coordinate, is the template frame width and height;

[0045] Step two: cut the interval with a size of 255*255 on the next frame as the tracking area to be tracked with the center coordinate of the template frame , called search frame.

[0046] Step three: the template frame and the search frame are respectively input into the ResNet-50 backbone feature extraction network to form two branches, and the two branches share network weights;

[0047] F templete =Backbone(img templete )

[0048] F search =Backbone(img search )

[0049] Where Backbone represents the backbone feature extraction network, img templete is the input template frame picture, F templete is the extracted template frame feature, img search is the input search frame picture, and F search is the extracted search frame feature.

[0050] Step four: extract the Conv3_3, Conv4_6, Conv_5_3 layer features of ResNet-50 to obtain different depth features, wherein the shallow layer features contain more color and texture features of the target, and the deep layer features contain more deep semantic level features of the target. The shallow, medium and deep layer feature extraction is shown in Figure 2 . Figure 2 is the tracking frame diagram of the present application, target is the template frame (also called templete), and search is the search frame, which are input into the network to form a double-flow branch. The shallow, medium and deep features are extracted at conv3_3, conv4_6 and conv5_3 respectively to do cross-correlation operation, and are regressed into boundary box coordinates through rpn, bboxregression is the boundary box regression loss, and CLS is the classification loss of judging whether there is an object in the boundary box.

[0051] Step five: for the shallow, medium and deep layer features respectively, the template frame and the search frame are cross-correlated, that is, the template frame feature is taken as the convolution kernel, the search frame feature is input, and the convolution result is the response feature map of the template frame in the search frame:

[0052]

[0053]

[0054]

[0055] wherein represents the cross-correlation operation, F corr1 , F corr2 , F corr3 correspond to the results of cross-correlation of the features of the template frame and the search frame using the shallow, middle, and deep layers, respectively.

[0056] Step six: input the shallow, middle, and deep response feature maps into the RPN network (Region Proposal Network) respectively, and regress the high response interval into the form of bbox coordinates. For the RPN network, a large number of anchors are first generated, and then candidate boxes with sizes of 64, 128, and 256 and scales of 1:1, 1:2, and 2:1 are generated on the anchors, and finally the correct bounding box containing the object is obtained through the classification and regression process.

[0057] In the regression process, first, the offset of the generated anchor compared to the original coordinates is calculated:

[0058]

[0059]

[0060] where [x a , y a , w a , h a ] are the center point coordinates and width and height of the anchor, and [t x , t y , t w , t h ] are the offset predicted by this regression layer, and the corrected anchor coordinates [x, y, w, h]

[0061] g x = p x + d x × p w

[0062] g y = p y + d y × p h

[0063]

[0064]

[0065] wherein, [p x ,p y ,p w ,p h ] represents the coordinates of the original anchor, [d x ,d y ,d w ,d h ] represents the coordinate offset predicted by the RPN network, [g x ,g y ,g w ,g h ] represents the corrected anchor coordinates.

[0066] Step seven: fuse the results of the weighted regression of different depth features S l , and obtain the specific position of the next frame of the target motion S all , and take this as the template frame for the next round of tracking, repeat the previous tracking process until the target motion ends, and obtain all the tracking results to synthesize the target motion trajectory. The weighting uses simple linear weighted sum.

[0067]

[0068] wherein, a i is the weighting coefficient.

[0069] The specific implementation steps of the present application are as follows:

[0070] Figure 2 is a schematic diagram of the tracking framework of the present application, and the specific implementation is as follows:

[0071] A single target tracking method based on a multi-layer feature fusion twin neural network, comprising the following steps:

[0072] 1. Frame extraction of the video to be tracked, and obtaining the initial position frame of the target to be tracked in the first frame as a template frame;

[0073] 2. Cutting a 255*255 size search frame region centered on the initial position frame of the target;

[0074] 3. Inputting the template frame and the search frame into the ResNet-50 backbone feature extraction network to form two branches, and the two branches share network weights;

[0075] 4. Extracting the Conv3_3, Conv4_6, and Conv_5_3 layer features of ResNet-50 to obtain different depth features;

[0076] 5、Template frame features as convolution kernels, search frame features as input, do cross-correlation operation;

[0077] 6、The high response interval features obtained by the search frame are regressed into bbox coordinates by an RPN network (Region Proposal Network); as shown in the figure, the high response interval regression of the features of the shallow layer, the middle layer, and the deep layer obtains the bbox. Figure 4

[0078] 7、Fusion of the regression results of different depth features, comprehensive target next frame movement specific location.

[0079] 8、Repeat the above steps 1-7 until the target moves to the last frame, and combine the positions of each frame of the target to generate a motion trajectory.

[0080] Figure 3 For performance test comparison of the method of the application and other methods on the UAV123 and LaSOT two benchmarks, Figure 3 The horizontal coordinate in the upper left and lower left two subgraphs represents the (overlap rate) IOU threshold of the tracked bbox and the actual bbox, and the vertical coordinate is the overlap accuracy on the UAV dataset; Figure 3 The horizontal coordinate in the upper right and lower right two subgraphs represents the pixel distance threshold of the center points of the tracked bbox and the actual bbox, and the vertical coordinate is the distance accuracy on the UAV dataset; the larger the area under the curve, the better the tracking effect of the tracker. From Figure 3 It can be seen that the accuracy and success rate of the method of the application are the highest. From this, it can be seen that the advantages of the application are as follows:

[0081] I. The method of multi-layer feature fusion is adopted, which considers the color texture features of the shallow layer and the deep semantic features, and can realize more robust tracking.

[0082] II. The RPN regression coordinate method is used, which greatly reduces the multi-scale sampling problem in tracking and obtains more accurate tracking effect, and has stronger foreground and background discrimination ability

[0083] III. The model parameter amount is greatly reduced compared with the traditional twin neural network, and the tracking speed is greatly guaranteed.

[0084] The application embodiment provides a single target tracking system based on a multi-layer feature fusion twin neural network, which is used to execute the single target tracking method based on the multi-layer feature fusion twin neural network described in the above embodiment, as shown in the figure, the system comprises: Figure 5

[0085] The template frame module 501 is used to frame the to-be-tracked video and obtain the initial position box of the to-be-tracked target in the first frame as a template frame.​​

[0086] The search frame module 502 is configured to cut a search frame region with a size of 255*255 at the center of the target initial position box in the next frame;

[0087] The input module 503 is configured to input the template frame and the search frame into a ResNet-50 backbone feature extraction network to form two branches, and the two branches share network weights.

[0088] The feature extraction module 504 is configured to extract Conv3_3, Conv4_6 and Conv_5_3 layer features of the ResNet-50 to obtain different depth features.

[0089] The cross-correlation module 505 is configured to perform cross-correlation operation on the search frame features by taking the template frame features as a convolution kernel.

[0090] The regression module 506 is configured to regress the high response interval features of the search frame into a bbox coordinate form through a region extraction network.

[0091] The fusion tracking module 507 is configured to fuse the results of regression of different depth features to obtain a specific position of the target in the next frame.

[0092] The single target tracking system based on the multi-layer feature fusion twin neural network provided in the above embodiments of the application and the single target tracking method based on the multi-layer feature fusion twin neural network provided in the embodiments of the application have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0093] The embodiments of the application further provide an electronic device corresponding to the single target tracking method based on the multi-layer feature fusion twin neural network provided in the above embodiments, to execute the single target tracking method based on the multi-layer feature fusion twin neural network. The embodiments of the application are not limited.

[0094] Please refer to Figure 6 , which shows a schematic diagram of an electronic device provided in some embodiments of the application. As Figure 6 shown, the electronic device 20 includes a processor 200, a memory 201, a bus 202 and a communication interface 203, the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the memory 201 stores a computer program which can run on the processor 200, and the processor 200 runs the computer program to execute the single target tracking method based on the multi-layer feature fusion twin neural network provided in any of the preceding embodiments of the application.

[0095] The memory 201 can include a high-speed random access memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 203 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used.

[0096] The bus 202 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The single target tracking method based on the multi-layer feature fusion twin neural network disclosed in any of the embodiments of the present application can be applied to the processor 200 or implemented by the processor 200.

[0097] The processor 200 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 200. The processor 200 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready-to-program gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and combines the hardware to complete the steps of the above method.

[0098] The electronic device provided by the embodiments of the present application and the single target tracking method based on the multi-layer feature fusion twin neural network provided by the embodiments of the present application have the same beneficial effects as the method they adopt, run or implement.

[0099] The embodiment of the present application further provides a computer readable storage medium corresponding to the single-target tracking method based on the multi-layer feature fusion twin neural network provided by the foregoing embodiment, please refer to Figure 7 The computer readable storage medium shown in the embodiment of the present application is an optical disc 30, and a computer program (i.e. program product) is stored on the optical disc 30, and the computer program performs the single-target tracking method based on the multi-layer feature fusion twin neural network provided by any of the foregoing embodiments when being run by a processor.

[0100] It should be noted that examples of the computer readable storage medium can further include, but are not limited to, a phase change memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), other types of random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory or other optical, magnetic storage medium, which are not described one by one here.

[0101] The computer readable storage medium provided by the foregoing embodiment of the present application and the single-target tracking method based on the multi-layer feature fusion twin neural network provided by the embodiment of the present application have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0102] It should be noted that:

[0103] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems can be used with these teachings, with or without accompanying specific hardware. The required structure for a variety of these systems will be apparent from the description above. In addition, the present application is not specifically aimed at any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the present application described herein, and the descriptions above of a specific language are provided for the best mode for carrying out the present application.

[0104] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known methods, structures and techniques are not described in detail in order not to obscure the understanding of the specification.

[0105] Similarly, it is to be understood that the embodiments of the present application can be alternately or additionally employed in combination with each other, where appropriate, even though not expressly stated in the above description. Although the present application has been described in accordance with the embodiments shown, one of ordinary skill in the art will readily recognize that there could be variations to the embodiments and modifications of the present application, such as: to the embodiments of the present application, to the methods of the present application, and to the devices of the present application. Accordingly, many modifications can be made by one of ordinary skill in the art without departing from the scope of the present application as disclosed herein. For example, although processes are described with regard to particular operational flows, various other processes can be implemented as steps within a larger process that includes the described steps, or as at least partially parallel processes that are performed concurrently with the described processes. Accordingly, other implementations are within the scope of the following claims.

[0106] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into more sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or of the device disclosed in the specification in combination with all the features disclosed in the specification (including the accompanying claims, abstract and drawings) can be taken, except that at least some of such features and / or processes or units are mutually exclusive. Unless explicitly stated, each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features that serve the same, equivalent or similar purpose.

[0107] Further, those skilled in the art will appreciate that a combination of features of different embodiments means within the scope of the present application and forms different embodiments, although some embodiments described herein include certain features rather than others included in other embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0108] The various component embodiments of the present application can be implemented in hardware, or as software modules running in one or more processors, or in combinations thereof. As will be appreciated by one skilled in the art, a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some or all of the components in the virtual machine creation system according to embodiments of the present application. The present application can also be implemented as a program of instructions for performing part or all of the methods described herein, e.g., a computer program and a computer program product. Such program of the present application can be stored on a computer readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier medium, or in any other form.

[0109] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a system claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different claims does not indicate that a combination of these measures cannot be used to advantage. The use of relative terms such as 'about', 'approximately','substantially' and the like, in connection with a given numerical value, is intended to mean that the value can vary from the indicated one by a margin of error, which is acceptable in the art.

[0110] The above description is only specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various changes or replacements within the technical range disclosed in the present application, and these should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A single-target tracking method based on a multi-layer feature fusion Siamese neural network, characterized in that, Includes the following steps: 1) Extract frames from the video to be tracked, and obtain the initial position box of the target to be tracked in the first frame as the template frame; 2) Using the initial target location bounding box as the center, crop a search frame area of ​​size 255*255 for the next frame; 3) The template frame and the search frame are respectively input into the ResNet-50 backbone feature extraction network to form two branches, and the two branches share the network weights; 4) Extract features from Conv3_3, Conv4_6, and Conv_5_3 layers of ResNet-50 to obtain features at different depths; 5) Use the template frame features as the convolution kernel and the search frame features as input to perform a cross-correlation operation; 6) The high-response interval features obtained from the search frame are regressed into bounding box coordinates through a region extraction network; 7) By fusing the results of regressions at different depths, the specific location of the target's motion in the next frame is obtained; In step 5), for shallow, medium, and deep features, the template frame and the search frame are cross-correlated, and the convolution result is the response feature map of the template frame in the search frame. In step 6), the shallow, middle and deep response feature maps are input into the region extraction network, and the high response intervals are regressed into bounding box coordinates. The region extraction network generates multiple anchor points, and then candidate boxes of sizes 64, 128 and 256 with ratios of 1:1, 1:2 and 2:1 are generated on the anchor points. The correct bounding boxes containing the objects are obtained through classification and regression processes.

2. The single-target tracking method based on a multi-layer feature fusion Siamese neural network according to claim 1, characterized in that, In step 1), the initial video is extracted to obtain a set of images for each frame; the initial position of the target to be tracked is selected in the first frame and normalized to a size of 127*127 as the template frame for tracking.

3. The single-target tracking method based on a multi-layer feature fusion Siamese neural network according to claim 1, characterized in that, In step 2), the search frame to be tracked is a 255*255 interval cropped from the center coordinates of the template frame on the next frame.

4. The single-target tracking method based on a multi-layer feature fusion Siamese neural network according to claim 1, characterized in that, In step 4), the shallow features include the color and texture features of the target, while the deep features include the deep semantic features of the target.

5. The single-target tracking method based on a multi-layer feature fusion Siamese neural network according to claim 1, characterized in that, After step 7), the specific position of the target's next frame motion is used as the template frame for the next round of tracking. The previous tracking process is repeated until the target motion ends, and all tracking results are combined to obtain the target motion trajectory.

6. A single-target tracking system based on a multi-layer feature fusion Siamese neural network, using the method described in any one of claims 1-5, characterized in that, include: The template frame module is used to extract frames from the video to be tracked and obtain the initial position box of the target to be tracked in the first frame as the template frame. The search frame module is used to crop a search frame area of ​​size 255*255, centered on the initial position box of the target. The input module is used to input the template frame and the search frame into the ResNet-50 backbone feature extraction network to form two branches, which share the network weights. The feature extraction module is used to extract features from the Conv3_3, Conv4_6, and Conv_5_3 layers of ResNet-50 to obtain features at different depths. The cross-correlation module is used to perform cross-correlation operations on template frame features as convolution kernels and search frame features as input. The regression module is used to regress the high-response interval features obtained from the search frame into bounding box coordinates through the region extraction network; The fusion tracking module is used to fuse the results of regressions at different depths to obtain the specific position of the target's motion in the next frame.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-5.

Images