Single target tracking system and method based on MCU

Abstract

The invention discloses a single-target tracking system and method based on MCUs, and belongs to the field of target tracking, the tracking system comprises a camera device and a user interaction device, the camera device is connected with a plurality of MCUs, neural networks are arranged on the MCUs, and the tracking system is used for obtaining template images; outputting template image features; obtaining an instance image and carrying out data pre-processing on the instance image; outputting instance image features; inputting the template image features and the instance image features into an image fusion network for operation to obtain unprocessed prediction data; performing confidence data post-processing; position data post-processing is carried out, a predicted calibration frame is obtained, a next frame of instance image is determined again according to a new calibration frame, and target tracking is completed; and performing frame insertion operation to complete tracking. According to the method, the problem of deploying the twin network on the MCU equipment is solved, so that the deployment cost is greatly reduced on the basis of sacrificing a small part of performance levels, and the protection of user privacy data is further improved.

Description

Technical Field

[0001] The present invention relates to the field of target tracking, and in particular to a single target tracking system and method based on MCU. Background Art

[0002] Object tracking is an important technology that has emerged in the fields of security, entertainment, industry, etc. in recent years. Single object tracking is one of its important applications.

[0003] Existing single-target tracking algorithms are usually implemented using deep learning models, and the models used are usually twin network models. Most application scenarios require the deployment of neural networks on edge devices, such as smartphones or server terminals. However, most of these edge devices require considerable energy consumption and cost. In addition, the risk of user information leakage is also a major problem for such devices. MCUs are inexpensive and difficult to crack, so MCUs become the first choice in scenarios that focus on cost and privacy. Therefore, it is necessary to propose corresponding solutions to the problem of deploying neural networks such as twin networks in extremely resource-constrained devices such as MCUs. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a single target tracking system and method based on MCU, aiming to solve the problem of how to deploy a twin network on an MCU device, so as to greatly reduce the deployment cost while sacrificing a small part of the performance level, and further improve the protection of user privacy data.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: a single target tracking system based on MCU, including a camera device and a user interaction device, the camera device is connected to a plurality of MCUs, a neural network is arranged on the plurality of MCUs, the neural network includes an image feature fusion network, the input layer of the image feature fusion network is connected to a template image feature extraction network and an instance image feature extraction network, and the output layer of the image feature fusion network is connected to a confidence output head and a position information output head.

[0006] A further improvement of the technical solution of the present invention lies in that: several MCUs are an algorithm running MCU and a main control MCU connected in parallel with the camera device, and a user interaction device is connected to the main control MCU; a module image feature extraction network, an instance image feature extraction network and an image feature fusion network connected to the two extraction networks through an input layer are deployed on the algorithm running MCU, and a confidence output head and a position information output head connected to the output layer of the image feature fusion network are deployed on the main control MCU.

[0007] A further improvement of the technical solution of the present invention is that the module image feature extraction network and the instance image feature extraction network have the same structure, both of which are convolutional neural networks with lightweight design.

[0008] A further improvement of the technical solution of the present invention is that the image feature fusion network is two parallel convolutional attention CBAM modules, each CBAM module first uses two independent convolutional layers to further extract the features of the template image and the instance image, and after the further extraction is completed, the three-dimensional image features are compressed into a two-dimensional image feature matrix by compressing the latter two dimensions, and then the feature matrices of the template image and the instance image are multiplied and restored to three-dimensional image features. After obtaining the dimensions of each feature, they are sequentially passed through an average pooling layer, two convolutional layers, and a sigmoid operation to obtain the weight of each feature, and finally the channels are weighted to obtain the final operation result.

[0009] A further improvement of the technical solution of the present invention is that the confidence output head and the position information output head are both formed by splicing a number of convolution blocks, and each convolution block is composed of two convolution layers and one standard layer.

[0010] A single target tracking method based on MCU, the specific steps are as follows:

[0011] Step 1: Obtain the template image and perform data preprocessing on the template image; after processing, input the template image feature extraction network to output the template image features;

[0012] Step 2: Obtain the instance image and perform data preprocessing on the instance image; after processing, input the instance image feature extraction network to output the instance image features;

[0013] Step 3: Input the template image features and the instance image features into the image fusion network for calculation to obtain the unprocessed prediction data;

[0014] Step 4: Post-process the confidence data;

[0015] Step 5: Perform post-processing on the position data to obtain the predicted calibration frame, and re-determine the instance image of the next frame according to the new calibration frame, so as to complete the tracking of the target in a loop;

[0016] Step 6: Perform interpolation operation.

[0017] The further improvement of the technical solution of the present invention is that: the specific steps of step 1 are as follows:

[0018] Step 1.1: Manually select the calibration frame, and determine the range of the image to be cropped according to the given calibration frame range. The cropping area is a square. The template image is cropped, and the side length of the cropping area is:

[0019]

[0020] Where L is the side length of the cropped area square, w is the width of the calibration box, and h is the height of the calibration box;

[0021] Step 1.2: Determine the specific area of ​​the cropping area. Assume that the center position of the calibration frame is (x0, y0). Then the final cropping area is a square area with (x0, y0) as the center and a side length of L.

[0022] Step 1.3: Fill the part beyond the image boundary with color. The color used is the average color of the image. The average color is calculated by separating the RGB color channels of the image and calculating the average value of each channel color. Finally, the colors of the RGB color channels are combined.

[0023] Step 1.4: Resize the cropped area to a uniform size. The template image is resized to 127x127.

[0024] The further improvement of the technical solution of the present invention is that the specific steps of step 2 are as follows:

[0025] Step 2.1: According to the range of the given calibration frame, determine the range of the image to be cropped. The cropping area is a square. If the instance image is the first frame, the given calibration frame is a manually selected calibration frame. In addition, if it is not the first frame, the calibration frame is the result predicted by the previous frame instance image. When cropping the instance image, the side length of the cropping area is:

[0026]

[0027] Where L is the side length of the cropped area square, w is the width of the calibration box, and h is the height of the calibration box;

[0028] Step 2.2: Determine the specific area of ​​the cropped area. Assume that the center position of the calibration frame is (x0, y0). Then the final cropped area is a square area with (x0, y0) as the center and a side length of L.

[0029] Step 2.3: Fill the part beyond the image boundary with color, and use the average color of the image. The average color is calculated by separating the RGB color channels of the image and calculating the average value of each channel color, and finally combining them back to the color of the RGB color channel.

[0030] Step 2.4: Resize the cropped area to a uniform size. The example image is resized to 255x255.

[0031] The further improvement of the technical solution of the present invention is that the specific steps of step 4 are as follows:

[0032] Step 4.1: Divide the instance image after data pre-processing into anchor points and label the anchor points; each anchor point contains confidence data and location information data;

[0033] Step 4.2: The confidence output data is 256x2 dual-channel data. The confidence data is the data of the second channel after being processed by the softmax function. The confidence at the i-th anchor point is recorded as c i .

[0034] The further improvement of the technical solution of the present invention is: Step 5.1: by calculating the data channel data of each anchor point, the position of the calibration frame in the instance image after data pre-processing is obtained, and the calculation formula is as follows:

[0035]

[0036] Among them, x1, y1 are the horizontal and vertical coordinates of the upper left corner of the predicted calibration box, x2, y2 are the horizontal and vertical coordinates of the lower right corner of the calibration box, and x a ,y a are the horizontal and vertical coordinates of the anchor point, and v1, v2, v3, and v4 are the data of the anchor point position channel respectively;

[0037] Step 5.2: Calculate the credibility coefficient p of each anchor point in turn.

[0038] For ease of expression, the calculation method of sz(w,h) and change(x) is as follows:

[0039]

[0040]

[0041] The calculation formula of the credibility coefficient p is as follows:

[0042]

[0043] Among them, p is the credibility coefficient, p s is the border credibility coefficient, p r is the proportional credibility coefficient, λ is the coefficient of variation obtained in data preprocessing; w,h are the width and height of the calibration frame calculated in the previous frame; p k is a modifiable hyperparameter;

[0044] Step 5.3: Calculate the corrected credibility coefficient p' based on the confidence and credibility coefficient of each anchor point:

[0045] p i '=c i ×p i (1-lw )+w i × w

[0046] Among them, c i is the confidence on the i-th anchor point, p i is the credibility coefficient on the i-th anchor point, l w is the window influence coefficient, w i is the value corresponding to the anchor point in the added two-dimensional matrix;

[0047] Step 5.4: Take the position information of the anchor point with the largest corrected credible parameter as the prediction result, and obtain the update coefficient according to the selected anchor point position. The calculation formula of the update coefficient is as follows:

[0048] γ=c b ×p b × r

[0049] Where γ is the update coefficient, c b is the confidence of the selected anchor point, p b is the credibility coefficient of the selected anchor point, l r is the update coefficient, which is a hyperparameter;

[0050] Step 5.5: Update the calibration frame information. The calibration frame is represented by the center, width and height of the calibration frame in this method. The calibration frame update formula is as follows:

[0051]

[0052] Where x c ',y c ', w', h' are the coordinates of the center point and width and height of the calibration box after update, x c ,y c , w, h are the coordinates before updating, γ is the updating coefficient, x1, y1 are the horizontal and vertical coordinates of the upper left corner coordinates and the lower right corner coordinates of the calibration box predicted by the selected anchor point;

[0053] Step 5.6: Map the predicted coordinates, project the template image coordinates in the predicted results back to the original image coordinates, and the projection formula is as follows:

[0054]

[0055] In the formula, x cr ',y cr ',w r ',h r ' is the coordinates of the center point and width and height of the calibration frame mapped back to the original image, x c ',y c', w', h' are the coordinates of the center point and the width and height of the calibration frame after update, and λ is the coefficient of change recorded in data preprocessing.

[0056] Due to the adoption of the above technical solution, the technical progress achieved by the present invention is: through block deployment, multiple MCUs are set up, and the feature extraction network and information output heads are deployed on different MCUs, which effectively reduces the deployment cost of the single-target algorithm and makes full use of the computing power of the single-chip MCU. By making lightweight improvements to the image feature extraction network, the computing power consumed by the network is reduced as much as possible while ensuring the accuracy of the model. The target tracking efficiency is improved by inserting frames into the output target data, the long processing time required to run the single target tracking algorithm on the MCU device is solved, and the real-time performance of the deployed algorithm is improved. All programs are run locally through the control device, and the data does not interact through the network. It can also be deployed in various environments, with good adaptability and high security. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 It is a schematic diagram of the structure of the target tracking system of the present invention;

[0059] Figure 2 It is a structural diagram of a neural network deployed in the MCU of the present invention;

[0060] Figure 3 It is a structural schematic diagram of the image feature fusion network of the present invention;

[0061] Figure 4 It is a schematic diagram of the structure of the convolution block of the present invention;

[0062] Figure 5 It is a flow chart of the operation of the single target tracking algorithm of the present invention;

[0063] Figure 6 It is a distribution diagram of the anchor points of the present invention on the example image. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with embodiments:

[0065] Embodiment 1

[0066] like Figure 1As shown, it is a structural schematic diagram of a single target tracking system based on MCU, including a camera device and a user interaction device. The camera device is provided with several MCUs, in this embodiment, there are two MCUs, that is, the camera device is connected with an algorithm operation MCU and a main control MCU in parallel. The camera device uses an OV2640 camera, and the main control MCU and the algorithm operation MCU both use STM32 series chips. Since both MCU chips have the need to obtain image information, in order to reduce delay, the camera device is connected in parallel with the two MCUs. The main control MCU is connected to a user interaction device and a motion control device, wherein the user interaction device uses a 7-inch LCD display screen, which has a touch function. In order to improve the operability of the user, this embodiment uses the TouchGFX framework to write a GUI interface and improve the user interaction function. The motion control device uses a two-degree-of-freedom pan-tilt, which is used to obtain the target position through the image fusion unit, and control the camera device to move through the target position to keep the target within the shooting range of the camera device. The single target tracking system based on MCU is controlled by an infrared receiving device, which is used to obtain the infrared signal sent by the user through the remote control, decode the obtained infrared signal and then send the decoded result to the control transposition to respond to the user operation. The system uses hx1838 and its matching remote control.

[0067] To achieve single target tracking, a neural network is deployed on two MCUs, such as Figure 2 As shown, the neural network is composed of multiple sub-neural networks, and the sub-neural networks include a template image feature extraction network, an instance image feature extraction network, an image feature fusion network, a confidence output head, and a position information output head. The image feature fusion network has two input layers, which are respectively connected to the template image feature extraction network and the instance image feature extraction network, and the two output layers are respectively connected to the confidence output head and the position information output head. The template image feature extraction network is a neural network used to extract features from a first frame image selected as an input to obtain template image features. The instance image feature extraction network is a neural network used to extract features from an input image after a certain frame selected as a subsequent input instance image to obtain instance image features. The image feature fusion network is a neural network used to use the template image features and the instance image features to achieve the single target tracking task. The confidence output head is used to obtain the confidence of the prediction result after data post-processing. The position information output head is used to obtain the specific next frame image information position after data post-processing.

[0068] In order to deploy this neural network on the MCU device, this method uses a multi-module independent deployment architecture and a unique model compression technology to intelligently quantize and deploy the model based on the small operating space and weak computing power of the MCU. The specific deployment method is as follows:

[0069] Step 1: Select and build a neural network based on the computing power of the existing MCU

[0070] The module image feature extraction network and the instance image feature extraction network need to have the same structure. Usually a lightweight convolutional neural network is selected. For example, MobileNet, ShuffleNet, SqueezeNet, Xception, EfficientNet. This embodiment uses the MobileNet network as the specific feature extraction network. It should be noted that no matter which of the above-mentioned networks is used, the image feature data pre-processing before input is the same. And the size of the obtained image feature map also needs to be consistent. The image feature fusion network is as follows: Figure 3 The main body shown is two parallel convolutional attention modules (CBAM) modules. Each CBAM module first uses two independent convolutional layers to further extract the features of the template image and the instance image. After the further extraction is completed, the three-dimensional image features are compressed into a two-dimensional image feature matrix by compressing the latter two dimensions. Then, the feature matrices of the template image and the instance image are multiplied and restored to three-dimensional image features. After obtaining the dimensions of each feature, the average pooling layer, two convolutional layers, and sigmoid operations are sequentially passed to obtain the weight of each feature. Finally, the channels are weighted to obtain the final operation result.

[0071] The confidence output head and the position information output head are both composed of several convolution blocks. The structure of each convolution block is as follows: Figure 4 As shown, it is composed of two convolutional layers and one normalization layer. In this embodiment, the confidence output head uses 4 convolutional blocks, and the position information output head uses 6 convolutional blocks.

[0072] Step 2: Rationally distribute the neural network model to the MCU.

[0073] The amount of computation of each sub-network of each neural network is counted. According to the actual situation, the sub-neural network is quantized while ensuring the performance indicators as much as possible, and the values ​​in the sub-model are converted from FP32 to INT8. At the same time, the amount of computation on each MCU needs to be as consistent as possible.

[0074] In this embodiment, only the instance feature network is quantized, and the sub-neural networks are allocated as follows: a module image feature extraction network, an instance image feature extraction network, and an image feature fusion network connected to the two extraction networks through an input layer are deployed on the algorithm running MCU, and a confidence output head and a position information output head connected to the output layer of the image feature fusion network are deployed on the main control MCU.

[0075] Step 3: Establish data transmission channels between various MCUs.

[0076] This embodiment selects the FIFO data format for data communication between MCUs. After deployment, the algorithm running MCU is deployed with a module image feature extraction network, an instance image feature extraction network, and an image feature fusion network connected to the two extraction networks through the input layer, and the main control MCU is deployed with a confidence output head and a position information output head connected to the output layer of the image feature fusion network, that is, the algorithm running MCU is used to obtain the template image and the instance image, and extract the features of the template image and the instance image by carrying the single target tracking algorithm, and then fuse the features to obtain the result. The main control MCU is used to schedule various types of data, including image feature data from the algorithm running device, target position data, and interactive data transmitted by the user through the user interaction device. Since the control is an MCU, there will be a problem of resource limitation when facing the amount of calculation required for the algorithm operation transposition, so the deployment system as a whole is written using multi-threading and object-oriented ideas. Each type of sensor or sub-network on other devices connected in an MCU device is regarded as an object, and multi-threading is used to perform independent data interaction and control. At the same time, the data is handed over to multiple MCUs for calculation and regulation in the FIFO data transmission method, thereby ensuring the real-time performance of the entire system.

[0077] According to the above system, a single target tracking method based on MCU is implemented, such as Figure 5 As shown, it includes the following steps:

[0078] Step 1: Obtain the template image and perform data preprocessing on the template image; after processing, input the template image feature extraction network to output the template image features;

[0079] Step 1.1: Manually select the calibration frame, and determine the range of the image to be cropped according to the given calibration frame range. The cropping area is a square. The template image is cropped, and the side length of the cropping area is:

[0080]

[0081] Where L is the side length of the cropped area square, w is the width of the calibration box, and h is the height of the calibration box;

[0082] Step 1.2: Determine the specific area of ​​the cropping area. Assume that the center position of the calibration frame is (x0, y0). Then the final cropping area is a square area with (x0, y0) as the center and a side length of L.

[0083] Step 1.3: Fill the part beyond the image boundary with color. The color used is the average color of the image. The average color is calculated by separating the RGB color channels of the image and calculating the average value of each channel color. Finally, the colors of the RGB color channels are combined.

[0084] Step 1.4: Resize the cropped area to a uniform size. The template image is resized to 127x127.

[0085] Step 2: Obtain the instance image and perform data preprocessing on the instance image; after processing, input the instance image feature extraction network to output the instance image features;

[0086] Step 2.1: According to the range of the given calibration frame, determine the range of the image to be cropped. The cropping area is a square. If the instance image is the first frame, the given calibration frame is a manually selected calibration frame. In addition, if it is not the first frame, the calibration frame is the result predicted by the previous frame instance image. When cropping the instance image, the side length of the cropping area is:

[0087]

[0088] Where L is the side length of the cropped area square, w is the width of the calibration box, and h is the height of the calibration box;

[0089] Step 2.2: Determine the specific area of ​​the cropped area. Assume that the center position of the calibration frame is (x0, y0). Then the final cropped area is a square area with (x0, y0) as the center and a side length of L.

[0090] Step 2.3: Fill the part beyond the image boundary with color, and use the average color of the image. The average color is calculated by separating the RGB color channels of the image and calculating the average value of each channel color, and finally combining them back to the color of the RGB color channel.

[0091] Step 2.4: Resize the cropped area to a uniform size. The example image is resized to 255x255.

[0092] Step 3: Input the template image features and the instance image features into the image fusion network for calculation to obtain the unprocessed prediction data;

[0093] Step 4: Post-process the confidence data;

[0094] Step 4.1: Divide the instance image after data pre-processing into anchor points and label the anchor points; each anchor point contains confidence data and location information data; Figure 6 The distribution of the anchor points shown is arranged from the pixel point (0,0) of the example image, from left to right, from top to bottom, with a spacing of 16 pixels between the left and right and the top and bottom. The numbering starts from 0. There are 256 anchor points in total.

[0095] Step 4.2: The confidence output data is 256x2 dual-channel data. The confidence data is the data of the second channel after being processed by the softmax function. The confidence at the i-th anchor point is recorded as c i .

[0096] Step 5: Perform post-processing on the position data to obtain the predicted calibration frame, and re-determine the instance image of the next frame based on the new calibration frame, so as to complete the tracking of the target in a cycle.

[0097] Step 5.1: The data channel data of each anchor point needs to be calculated to obtain the position calculation formula of the calibration box in the instance image after data pre-processing as follows:

[0098]

[0099] where x1, y 1 is the horizontal and vertical coordinates of the upper left corner of the predicted calibration box, x2, y2 are the horizontal and vertical coordinates of the lower right corner of the calibration box, x a ,y a is the horizontal and vertical coordinates of the anchor point. v1, v2, v3, and v4 are the data of the anchor point position channel.

[0100] Step 5.2: Calculate the credibility coefficient p of each anchor point in turn.

[0101] For ease of expression, the calculation method of sz(w,h) and change(x) is as follows:

[0102]

[0103]

[0104] The calculation formula of the credibility coefficient p is as follows:

[0105]

[0106] Among them, p is the credibility coefficient, p s is the border credibility coefficient, p r is the proportional credibility coefficient, λ is the coefficient of variation obtained in data preprocessing. w,h are the width and height of the calibration frame calculated in the previous frame.k is a modifiable hyperparameter, which is 0.15 in this embodiment.

[0107] Step 5.3: Calculate the corrected credibility coefficient p' based on the confidence and credibility coefficient of each anchor point:

[0108] p i '=c i ×p i (1-l w )+w i × w

[0109] Among them, c i is the confidence on the i-th anchor point, p i is the credibility coefficient on the i-th anchor point, l w is the influence coefficient of the window, which is a hyperparameter and is 0.495 in this embodiment. i is the value of the position corresponding to the anchor point in the added two-dimensional matrix. In this embodiment, the added two-dimensional window is a Hanning window with a window length of 16.

[0110] Step 5.4: Take the position information of the anchor point with the largest corrected credible parameter as the prediction result, and obtain the update coefficient according to the selected anchor point position. The calculation formula of the update coefficient is as follows:

[0111] γ=c b ×p b × r

[0112] Where γ is the update coefficient, c b is the confidence of the selected anchor point, p b is the credibility coefficient of the selected anchor point, l r is the update coefficient, which is a hyperparameter and is 0.385 in this embodiment.

[0113] Step 5.5: Update the calibration frame information. The calibration frame is represented by the center, width and height of the calibration frame in this method. The calibration frame update formula is as follows:

[0114]

[0115] Where x c ',y c ', w', h' are the coordinates of the center point and width and height of the calibration box after update, x c ,y c , w, h are the coordinates before update, γ is the update coefficient, x1, y1 are the horizontal and vertical coordinates of the upper left corner coordinates and the lower right corner coordinates of the calibration box predicted by the selected anchor point.

[0116] Step 5.6: Map the predicted coordinates, project the template image coordinates in the predicted results back to the original image coordinates, and the projection formula is as follows:

[0117]

[0118] In the formula, x cr ',y cr ',w r ',h r ' is the coordinates of the center point and width and height of the calibration frame mapped back to the original image, x c ',y c ', w', h' are the coordinates of the center point and the width and height of the calibration frame after update, and λ is the coefficient of change recorded in data preprocessing.

[0119] Step 6: After calculating the predicted coordinates, insert a frame of calibration boxes at possible positions during the next frame prediction process. The calibration box position calculation formula is as follows:

[0120]

[0121] In the formula, x cp ',y cp ',w p ',h p 'The center coordinates, width and height of the calibration frame where the pin is inserted, x cr ',y cr ',w r ',h r 'After the update is completed, the coordinates of the center point and width and height of the calibration frame in the original image of the current frame are cr ”,y cr ”,w r ”,h r ” is the coordinates of the center point and the width and height of the calibration frame in the previous frame of the original image before the update is completed, and Δt is the update speed of the calibration frame.

[0122] The embodiments described above are merely descriptions of preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A single target tracking system based on MCU, comprising a camera device and a user interaction device, characterized in that: The camera device is connected to several MCUs, and neural networks are arranged on the several MCUs. The neural network includes an image feature fusion network. The input layer of the image feature fusion network is connected to a template image feature extraction network and an instance image feature extraction network. The output layer of the image feature fusion network is connected to a confidence output head and a position information output head.

2. The single target tracking system based on MCU according to claim 1, characterized in that: Several MCUs are an algorithm running MCU and a main control MCU connected in parallel with the camera device, and a user interaction device is connected to the main control MCU; a module image feature extraction network, an instance image feature extraction network and an image feature fusion network connected to the two extraction networks through an input layer are deployed on the algorithm running MCU, and a confidence output head and a position information output head connected to the output layer of the image feature fusion network are deployed on the main control MCU.

3. The single target tracking system based on MCU according to claim 2, characterized in that: The module image feature extraction network and the instance image feature extraction network have the same structure, both of which are lightweight convolutional neural networks.

4. The single target tracking system based on MCU according to claim 2, characterized in that: The image feature fusion network consists of two parallel convolutional attention CBAM modules. Each CBAM module first uses two independent convolutional layers to further extract the features of the template image and the instance image. After further extraction, the three-dimensional image features are compressed into a two-dimensional image feature matrix by compressing the latter two dimensions. Then, the feature matrices of the template image and the instance image are multiplied and restored to three-dimensional image features. After obtaining the dimensions of each feature, they are sequentially passed through the average pooling layer, two convolutional layers, and sigmoid operations to obtain the weight of each feature. Finally, the channels are weighted to obtain the final operation result.

5. The single target tracking system based on MCU according to claim 2, characterized in that: The confidence output head and the position information output head are both composed of several convolution blocks, and each convolution block is composed of two convolution layers and one standard layer.

6. A single target tracking method based on MCU, implemented by the tracking system according to any one of claims 1 to 5, characterized in that: The specific steps are as follows: Step 1: Obtain the template image and perform data preprocessing on the template image; after processing, input the template image feature extraction network to output the template image features; Step 2: Obtain the instance image and perform data preprocessing on the instance image; after processing, input the instance image feature extraction network to output the instance image features; Step 3: Input the template image features and the instance image features into the image fusion network for calculation to obtain the unprocessed prediction data; Step 4: Post-process the confidence data; Step 5: Perform post-processing on the position data to obtain the predicted calibration frame, and re-determine the instance image of the next frame according to the new calibration frame, so as to complete the tracking of the target in a loop; Step 6: Perform interpolation operation.

7. The single target tracking method based on MCU according to claim 5, characterized in that: Step 1 The specific steps are as follows: Step 1.1: Manually select the calibration frame, and determine the range of the image to be cropped according to the given calibration frame range. The cropping area is a square. The template image is cropped, and the side length of the cropping area is: Where L is the side length of the cropped area square, w is the width of the calibration box, and h is the height of the calibration box; Step 1.2: Determine the specific area of ​​the cropping area. Assume that the center position of the calibration frame is (x0, y0). Then the final cropping area is a square area with (x0, y0) as the center and a side length of L. Step 1.3: Fill the part beyond the image boundary with color. The color used is the average color of the image. The average color is calculated by separating the RGB color channels of the image and calculating the average value of each channel color. Finally, the colors of the RGB color channels are combined. Step 1.4: Resize the cropped area to a uniform size. The template image is resized to 127x127.

8. The single target tracking method based on MCU according to claim 5, characterized in that: Step 2 The specific steps are as follows: Step 2.1: According to the range of the given calibration frame, determine the range of the image to be cropped. The cropping area is a square. If the instance image is the first frame, the given calibration frame is a manually selected calibration frame. In addition, if it is not the first frame, the calibration frame is the result predicted by the previous frame instance image. When cropping the instance image, the side length of the cropping area is: Where L is the side length of the cropped area square, w is the width of the calibration box, and h is the height of the calibration box; Step 2.2: Determine the specific area of ​​the cropped area. Assume that the center position of the calibration frame is (x0, y0). Then the final cropped area is a square area with (x0, y0) as the center and a side length of L. Step 2.3: Fill the part beyond the image boundary with color, and use the average color of the image. The average color is calculated by separating the RGB color channels of the image and calculating the average value of each channel color, and finally combining them back to the color of the RGB color channel. Step 2.4: Resize the cropped area to a uniform size. The instance image is resized to 255x255.

9. The single target tracking method based on MCU according to claim 5, characterized in that: Step 4 The specific steps are as follows: Step 4.1: Divide the instance image after data pre-processing into anchor points and label the anchor points; each anchor point contains confidence data and location information data; Step 4.2: The confidence output data is 256x2 dual-channel data. The confidence data is the data of the second channel after being processed by the softmax function. The confidence at the i-th anchor point is recorded as c i .

10. The single target tracking method based on MCU according to claim 5, characterized in that: Step 5.1: By calculating the data channel data of each anchor point, the position of the calibration box in the instance image after data pre-processing is obtained. The calculation formula is as follows: Among them, x1, y1 are the horizontal and vertical coordinates of the upper left corner of the predicted calibration box, x2, y2 are the horizontal and vertical coordinates of the lower right corner of the calibration box, a ,y a are the horizontal and vertical coordinates of the anchor point, and v1, v2, v3, and v4 are the data of the anchor point position channel respectively; Step 5.2: Calculate the credibility coefficient p of each anchor point in turn. For ease of expression, the calculation method of sz(w,h) and change(x) is as follows: The calculation formula of the credibility coefficient p is as follows: Among them, p is the credibility coefficient, p s is the border credibility coefficient, p r is the proportional credibility coefficient, λ is the coefficient of variation obtained in data preprocessing; w,h are the width and height of the calibration frame calculated in the previous frame; p k is a modifiable hyperparameter; Step 5.3: According to the confidence and credibility coefficient of each anchor point, calculate the corrected credibility coefficient p': p i '=c i ×p i (1-l w )+w i ×l w Among them, c i is the confidence on the i-th anchor point, p i is the credibility coefficient on the i-th anchor point, l w is the window influence coefficient, w i is the value corresponding to the anchor point in the added two-dimensional matrix; Step 5.4: Take the position information of the anchor point with the largest corrected credible parameter as the prediction result, and obtain the update coefficient according to the selected anchor point position. The calculation formula of the update coefficient is as follows: γ=c b ×p b ×l r Where γ is the update coefficient, c b is the confidence of the selected anchor point, p b is the credibility coefficient of the selected anchor point, l r is the update coefficient, which is a hyperparameter; Step 5.5: Update the calibration frame information. The calibration frame is represented by the center, width and height of the calibration frame in this method. The calibration frame update formula is as follows: Where x c ',y c ', w', h' are the coordinates of the center point and width and height of the calibration box after update, x c ,y c , w, h are the coordinates before updating, γ is the updating coefficient, x1, y1 are the horizontal and vertical coordinates of the upper left corner coordinates and the lower right corner coordinates of the calibration box predicted by the selected anchor point; Step 5.6: Map the predicted coordinates, project the template image coordinates in the predicted results back to the original image coordinates, and the projection formula is as follows: In the formula, x cr ',y cr ',w r ',h r ' is the coordinates of the center point and width and height of the calibration frame mapped back to the original image, x c ',y c ', w', h' are the coordinates of the center point and the width and height of the calibration frame after update, and λ is the coefficient of change recorded in data preprocessing.

Images