A single-target tracking system and method based on MCU

Abstract

The application discloses a single-target tracking system and method based on MCU, and belongs to the field of target tracking. The tracking system comprises a camera device and a user interaction device. A plurality of MCUs are connected to the camera device. A neural network is arranged on the MCU. A template image is acquired by using the tracking system. Template image features are output. An instance image is acquired and data preprocessing is performed on the instance image. Instance image features are output. The template image features and the instance image features are input into an image fusion network for operation to obtain unprocessed prediction data. Confidence data post-processing is performed. Position data post-processing is performed to obtain a predicted calibration frame. The next frame of instance image is determined again according to the new calibration frame, and the tracking of the target is completed. Interpolation operation is performed to complete the tracking. The application solves the problem of deploying a twin network on an MCU device, greatly reduces the deployment cost on the basis of sacrificing a small part of performance level, and further improves the protection of user privacy data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target tracking, and in particular to a single-target tracking system and method based on MCU. BACKGROUND

[0002] Target tracking is an important technology that has emerged in recent years in the fields of security, entertainment, and industry. Single-target tracking is an important application thereof.

[0003] Existing single-target tracking algorithms are usually implemented using deep learning models, and the model used is usually a twin network model. Most application scenarios require the deployment of a neural network to an edge device, such as a smartphone or a server terminal. However, these edge devices require considerable energy consumption and cost. In addition, the risk of user information leakage is also a major problem for such devices. MCU is inexpensive and difficult to crack, so in scenarios that focus on cost and privacy, MCU becomes the first choice. Therefore, a corresponding solution is needed for the problem of deploying a twin network, a neural network, on a MCU device with extremely limited resources. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a single-target tracking system and method based on MCU, which aims to solve the problem of deploying a twin network on a MCU device, so as to greatly reduce the deployment cost while sacrificing a small part of the performance level, and further improve the protection of user privacy data.

[0005] To solve the above technical problems, the technical scheme adopted by the present application is: a single-target tracking system based on MCU, comprising a camera device and a user interaction device, a plurality of MCUs are connected to the camera device, a neural network is arranged on the plurality of MCUs, the neural network comprises an image feature fusion network, a template image feature extraction network and an instance image feature extraction network are connected to the input layer of the image feature fusion network, and a confidence output head and a position information output head are connected to the output layer of the image feature fusion network.

[0006] The further improvement of the technical scheme of the present application is that the plurality of MCUs are algorithm running MCUs and a master MCU connected in parallel with the camera device, the user interaction device is connected to the master MCU, the algorithm running MCUs are deployed with a template image feature extraction network, an instance image feature extraction network, and an image feature fusion network connected to the two extraction networks through an input layer, and the master MCU is deployed with a confidence output head and a position information output head connected to the output layer of the image feature fusion network.

[0007] Further improvement of the technical scheme of the application is that the module image feature extraction network and the instance image feature extraction network have the same structure, and are both light-weight designed convolutional neural networks.

[0008] Further improvement of the technical scheme of the application is that the image feature fusion network is two parallel convolution attention CBAM modules, each CBAM module first uses two independent convolution layers to further extract the features of the template image and the instance image, after the further extraction, the three-dimensional image features are compressed into a two-dimensional image feature matrix through compression of two dimensions, then the feature matrices of the template image and the instance image are multiplied and restored into three-dimensional image features, after obtaining the dimensions of each feature, the features are sequentially passed through an average pooling layer, two convolution layers and sigmoid operation to obtain the weight of each feature, and finally the channels are weighted to obtain the final operation result.

[0009] Further improvement of the technical scheme of the application is that the confidence output head and the position information output head are both composed of a plurality of convolution blocks, and each convolution block is composed of two convolution layers and a standard layer.

[0010] A single-target tracking method based on an MCU includes the following specific steps:

[0011] Step 1: Obtain a template image and perform data preprocessing on the template image; after the preprocessing, input the template image into a template image feature extraction network to output template image features;

[0012] Step 2: Obtain an instance image and perform data preprocessing on the instance image; after the preprocessing, input the instance image into an instance image feature extraction network to output instance image features;

[0013] Step 3: Input the template image features and the instance image features into an image fusion network to obtain unprocessed prediction data;

[0014] Step 4: Perform confidence data post-processing;

[0015] Step 5: Perform position data post-processing to obtain a predicted bounding box, and determine a next frame of instance image according to the new bounding box, so as to complete the tracking of the target through a cycle;

[0016] Step 6: Perform an interpolation operation.

[0017] Further improvement of the technical scheme of the application is that step 1 includes the following specific steps:

[0018] Step 1.1: Manually frame a bounding box, determine the range of the image to be cropped according to the range of the given bounding box, the cropping area is a square, and the side length of the cropping area is:

[0019]

[0020] wherein L is the side length of the square of the cutting region, w is the width of the bounding box, and h is the height of the bounding box;

[0021] Step 1.2: Determine the specific region of the cutting region, and set the center position of the bounding box as (x0, y0), so that the final cutting region is a square region with (x0, y0) as the center and L as the side length;

[0022] Step 1.3: Color filling is performed on the part exceeding the image boundary, and the color used is the average color of the image, and the calculation method of the average color is to calculate the average value of each color channel separated from the RGB color channel of the image, and finally combine the colors back to the RGB color channel;

[0023] Step 1.4: Adjust the size of the cut-out region to a uniform size, and the size of the template image is adjusted to 127x127.

[0024] Further improvement of the technical scheme of the application is that step 2 has the following specific steps:

[0025] Step 2.1: According to the range of the given bounding box, determine the range of the image to be cut, and the cutting region is a square, if the example image is the first frame image, the given bounding box is the bounding box manually selected, and if it is not the first frame image, the bounding box is the result predicted by the last frame example image; cutting the example image, and the side length of the cutting region is:

[0026]

[0027] wherein L is the side length of the square of the cutting region, w is the width of the bounding box, and h is the height of the bounding box;

[0028] Step 2.2: Determine the specific region of the cutting region, and set the center position of the bounding box as (x0, y0), so that the final cutting region is a square region with (x0, y0) as the center and L as the side length;

[0029] Step 2.3: Color filling is performed on the part exceeding the image boundary, and the color used is the average color of the image, and the calculation method of the average color is to calculate the average value of each color channel separated from the RGB color channel of the image, and finally combine the colors back to the RGB color channel;

[0030] Step 2.4: Adjust the size of the cut-out region to a uniform size, and the size of the example image is adjusted to 255x255.

[0031] Further improvement of the technical scheme of the application is that step 4 has the following specific steps:

[0032] Step 4.1: divide anchor points on the pre-processed instance image, and label the anchor points; each anchor point contains confidence data and position information data;

[0033] Step 4.2: the confidence output data is 256x2 two-channel data, the confidence data is the data of the second channel after the softmax function processing, and the confidence of the i-th anchor point is denoted as c i .

[0034] Further improvement of the technical scheme of the application is that: step 5.1: the position of the calibration frame in the pre-processed instance image is obtained by calculating the data channel data of each anchor point, and the calculation formula is as follows:

[0035]

[0036] Where x1, y1 are the horizontal and vertical coordinates of the predicted upper left corner coordinates of the calibration frame, x2, y2 are the horizontal and vertical coordinates of the right lower corner coordinates of the calibration frame, x a , y a are the horizontal and vertical coordinates of the anchor point position, v1, v2, v3, v4 are the data of the anchor point position channel respectively;

[0037] Step 5.2: calculate the confidence coefficient p of each anchor point in turn.

[0038] In order to facilitate expression, it is agreed that the sz(w, h) operation and the change(x) operation are calculated as follows:

[0039]

[0040]

[0041] The calculation formula of the confidence coefficient p is as follows:

[0042]

[0043] Where p is the confidence coefficient, p s is the frame confidence coefficient, p r is the proportional confidence coefficient, λ is the change coefficient obtained in the data pre-processing; w, h are the width and height of the calibration frame calculated in the last frame; p k is a modifiable hyperparameter;

[0044] Step 5.3: according to the confidence and the confidence coefficient of each anchor point, the modified confidence coefficient p' is calculated:

[0045] p i '=c i ×p i (1-lw )+w i ×l w

[0046] where c i is the confidence of the i-th anchor point, p i is the trust coefficient of the i-th anchor point, l w is the influence coefficient of the window, w i is the value of the corresponding position of the anchor point in the added two-dimensional matrix;

[0047] Step 5.4: Take the position information of the anchor point with the maximum modified trust parameter as the prediction result, and obtain the update coefficient according to the selected anchor point position. The update coefficient calculation formula is as follows:

[0048] γ = c b × p b × l r

[0049] where γ is the update coefficient, c b is the confidence of the selected anchor point, p b is the trust coefficient of the selected anchor point, l r is the update coefficient, which is a hyperparameter;

[0050] Step 5.5: Update the information of the calibration frame. The representation of the calibration frame in this method is the center and width and height of the calibration frame. The calibration frame update formula is as follows:

[0051]

[0052] where x c ', y c ', w', h' are the center point coordinates and width and height of the updated calibration frame, x c , y c , w, h are the coordinates before updating, γ is the update coefficient, x1, y1 are the horizontal and vertical coordinates of the top-left corner coordinates and the horizontal and vertical coordinates of the bottom-right corner coordinates of the calibration frame predicted by the selected anchor point;

[0053] Step 5.6: Map the predicted coordinates to project the template image coordinates in the predicted result back to the original image coordinates. The projection formula is as follows:

[0054]

[0055] where x cr ', y cr ', w r ', h r ' are the center point coordinates and width and height of the calibration frame mapped back to the original image, x c ', y cw', h' are the updated calibration frame center point coordinates and width and height, and λ is the change coefficient recorded in the data preprocessing.

[0056] Thanks to the above technical solutions, the present application has achieved the following technical progress: by means of block deployment, multiple MCUs are set up, the feature extraction network and the information output head are deployed on different MCUs, the deployment cost of the single-target algorithm is effectively reduced, and the computing power of the single-chip MCU is fully utilized. By means of light-weight improvement of the image feature extraction network, the computing power consumed by the network is reduced as much as possible while the accuracy of the model is ensured. By means of the frame insertion operation on the output target data, the tracking efficiency of the target is improved, the long processing time required for running the single-target tracking algorithm on the MCU device is solved, and the real-time performance of the deployed algorithm is improved. All programs run locally through the control device, data do not interact through the network, and the programs can be deployed in various environments, have good adaptability, and are highly safe. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, a brief introduction will be given below to the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings;

[0058] Figure 1 is a structural schematic diagram of the target tracking system of the present application;

[0059] Figure 2 is a neural network structure diagram deployed in the MCU of the present application;

[0060] Figure 3 is a structural schematic diagram of the image feature fusion network of the present application;

[0061] Figure 4 is a structural schematic diagram of the convolution block of the present application;

[0062] Figure 5 is a single-target tracking algorithm running flowchart of the present application;

[0063] Figure 6 is a distribution diagram of the anchor points on the instance image of the present application. DETAILED DESCRIPTION

[0064] The present application will be further described in detail below in combination with embodiments:

[0065] Embodiment One

[0066] As Figure 1As shown, it is a structure diagram of a single target tracking system based on MCU, including a camera device and a user interaction device, the camera device is provided with a plurality of MCUs, in this embodiment, two pieces, that is, the camera device is connected with an algorithm running MCU and a master control MCU in parallel. The camera device uses an OV2640 camera, and the master control MCU and the algorithm running MCU both use STM32 series chips. Since both MCU chips have the demand for image information acquisition, in order to reduce the delay, the camera device is connected with the two MCUs in parallel. The master control MCU is communicatively connected with the user interaction device and a motion control device, wherein the user interaction device uses a 7-inch LCD display screen, which has a touch function. In order to improve the operability of the user, the embodiment uses a TouchGFX framework to compile a GUI interface and perfect the user interaction function. The motion control device uses a two-degree-of-freedom holder, which is used to obtain the target position through the image fusion unit, control the camera device to move through the target position, and keep the target in the shooting range of the camera device. The single target tracking system based on MCU is controlled by an infrared receiving device, which is used to obtain the infrared signal sent by the user through a remote controller, decode the obtained infrared signal, and deliver the decoding result to the control device to respond to the user operation. Hx1838 and a remote controller matched therewith are used.

[0067] In order to realize single target tracking, a neural network is deployed on the two MCUs, as shown in Figure 2 As shown, the neural network is composed of a plurality of sub neural networks, including a template image feature extraction network, an instance image feature extraction network, an image feature fusion network, a confidence output head, and a position information output head. The image feature fusion network has two input layers connected with the template image feature extraction network and the instance image feature extraction network respectively, and two output layers connected with the confidence output head and the position information output head respectively. The template image feature extraction network is a neural network for extracting features of a first frame image selected by a person as an input to obtain template image features. The instance image feature extraction network is a neural network for extracting features of an instance image selected by a person after a certain frame as a subsequent input to obtain instance image features. The image feature fusion network is a neural network for realizing the single target tracking task by using the template image features and the instance image features. The confidence output head is a confidence for obtaining a prediction result after data post-processing. The position information output head is a specific next frame image information position obtained after data post-processing.

[0068] In order to deploy this neural network on MCU device, the method adopts multi-module separate deployment architecture and unique model compression technology. According to the characteristics of small running space and weak computing power of MCU, the model is quantized and deployed intelligently. The specific deployment method is as follows:

[0069] Step 1: Select and construct the neural network according to the computing power of the existing MCU

[0070] The module image feature extraction network and the instance image feature extraction network need to have the same structure. A light-weight convolutional neural network is usually selected. For example, MobileNet, ShuffleNet, SqueezeNet, Xception, and EfficientNet. In this embodiment, MobileNet network is used as a specific feature extraction network. It should be noted that no matter which network mentioned above is used, the data preprocessing of the image features before input is the same processing. In addition, the size of the image feature map obtained also needs to be consistent. The image feature fusion network is as shown in Figure 3 The main body is two parallel convolution attention module (CBAM) modules. Each CBAM module will first use two independent convolution layers to further extract the features of the template image and the instance image. After further extraction, the three-dimensional image features are compressed into a two-dimensional image feature matrix by compressing two dimensions. Then, the feature matrix of the template image and the instance image is multiplied and restored to a three-dimensional image feature. After obtaining the dimensions of each feature, it is sequentially passed through an average pooling layer, two convolution layers, and a sigmoid operation to obtain the weight of each feature. Finally, the channels are weighted to obtain the final operation result.

[0071] The confidence output head and the position information output head are both composed of several convolution blocks. The structure of each convolution block is as shown in Figure 4 The confidence output head uses 4 convolution blocks, and the position information output head uses 6 convolution blocks.

[0072] Step 2: Reasonably distribute the neural network model to the MCU.

[0073] Statistical operation of each neural network and each subnetwork, according to the actual situation, under the premise of ensuring the performance index as much as possible, the sub neural network is quantized, and the value in the sub model is converted from FP32 to INT8. At the same time, it is necessary to ensure that the operation amount on each MCU is as consistent as possible.

[0074] In this embodiment, only the instance feature network is quantized, and the allocation of the sub neural network is that the module image feature extraction network, the instance image feature extraction network and the image feature fusion network connected with the two extraction networks through the input layer are deployed on the algorithm running MCU, and the confidence output head and the position information output head connected with the output layer of the image feature fusion network are deployed on the master control MCU.

[0075] Step 3: Establish a data transmission channel between each MCU.

[0076] In this embodiment, the data communication between MCUs is in the FIFO data format. After deployment, the module image feature extraction network, the instance image feature extraction network and the image feature fusion network connected with the two extraction networks through the input layer are deployed on the algorithm running MCU, and the confidence output head and the position information output head connected with the output layer of the image feature fusion network are deployed on the master control MCU, that is, the algorithm running MCU is used to obtain the template image and the instance image, and extract the features of the template image and the instance image through the single target tracking algorithm, and then fuse the features to obtain the result. The master control MCU is used to schedule various data, including image feature data from the algorithm running device, target position data, and interactive data transmitted by the user through the user interactive device. Since the control is MCU, there is a problem of limited resources when facing the calculation amount required by the algorithm running transposition, therefore, the whole deployment system is written with the multi-threading and object-oriented idea, each type of sensor or other device connected in a MCU device is taken as an object, independent data interaction and control are realized through multi-threading, and the data is transmitted to multiple MCUs for operation and regulation through the FIFO data transmission method, so as to ensure the real-time performance of the whole system.

[0077] According to the above system, the single target tracking method based on MCU is realized, as shown in Figure 5 The method comprises the following steps:

[0078] Step 1: Obtain the template image and perform data preprocessing on the template image; after processing, input the template image into the template image feature extraction network to output the template image features;

[0079] Step 1.1: Manually frame the calibration box, determine the range of the image to be cropped according to the range of the given calibration box, the cropping area is a square, and the template image is cropped, so the side length of the cropping area is:

[0080]

[0081] Wherein L is the side length of the square cropping area, w is the width of the calibration box, and h is the height of the calibration box.

[0082] Step 1.2: Determine the specific area of the obtained cropping area, and assume that the center position of the calibration frame is (x0, y0). The final cropping area is a square area with (x0, y0) as the center and a side length of L.

[0083] Step 1.3: Color filling is performed on the part exceeding the image boundary, and the color used is the average color of the image. The calculation method of the average color is to calculate the average value of each color channel of the image separated from the RGB color channel, and finally combine the colors back to the RGB color channel.

[0084] Step 1.4: Adjust the size of the cropped area to a uniform size, and the size of the template image is adjusted to 127x127.

[0085] Step 2: Obtain the instance image and perform data preprocessing on the instance image; input the processed instance image into the instance image feature extraction network to output the instance image feature.

[0086] Step 2.1: Determine the range of the image to be cropped according to the given calibration frame range. The cropping area is a square. If the instance image is the first frame image, the given calibration frame is the calibration frame manually selected by a person. In addition, if it is not the first frame image, the calibration frame is the result predicted by the previous frame instance image. After cropping the instance image, the side length of the cropping area is:

[0087]

[0088] where L is the side length of the square cropping area, w is the width of the calibration frame, and h is the height of the calibration frame.

[0089] Step 2.2: Determine the specific area of the obtained cropping area, and assume that the center position of the calibration frame is (x0, y0). The final cropping area is a square area with (x0, y0) as the center and a side length of L.

[0090] Step 2.3: Color filling is performed on the part exceeding the image boundary, and the color used is the average color of the image. The calculation method of the average color is to calculate the average value of each color channel of the image separated from the RGB color channel, and finally combine the colors back to the RGB color channel.

[0091] Step 2.4: Adjust the size of the cropped area to a uniform size, and the size of the instance image is adjusted to 255x255.

[0092] Step 3: Input the template image feature and the instance image feature into the image fusion network to obtain the unprocessed prediction data.

[0093] Step 4: Perform confidence data post-processing.

[0094] Step 4.1: divide anchor points on the pre-processed instance image, and label the anchor points; each anchor point contains confidence data and position information data; as shown in the following figure Figure 6 The distribution of the depicted points is arranged from left to right and from top to bottom, with a left-right interval and an up-down interval of 16 pixel points. The anchor points are labeled from 0. There are a total of 256 anchor points

[0095] Step 4.2: the confidence output data is 256x2 two-channel data, and the confidence data is the second channel data processed by the softmax function. The confidence of the i-th anchor point is denoted as c i .

[0096] Step 5: perform post-processing on the position data to obtain the predicted bounding box, and determine the instance image of the next frame according to the new bounding box to complete the tracking of the target.

[0097] Step 5.1: the data channel data of each anchor point needs to be calculated to obtain the position of the bounding box in the pre-processed instance image. The calculation formula is as follows:

[0098]

[0099] where x1, y 1 are the horizontal and vertical coordinates of the upper left corner of the predicted bounding box, x2, y2 are the horizontal and vertical coordinates of the lower right corner of the bounding box, x a , y a are the horizontal and vertical coordinates of the anchor point position. v1, v2, v3, v4 are the data of the anchor point position channel.

[0100] Step 5.2: calculate the confidence coefficient p of each anchor point in turn.

[0101] In order to facilitate expression, it is agreed that the sz(w, h) operation and the change(x) operation are calculated as follows:

[0102]

[0103]

[0104] The calculation formula of the confidence coefficient p is as follows:

[0105]

[0106] where p is the confidence coefficient, p s is the bounding box confidence coefficient, p r is the proportional confidence coefficient, and λ is the change coefficient obtained in the data preprocessing. w, h are the width and height of the bounding box calculated in the previous frame. pk The super parameter is modifiable, and in this embodiment, is 0.15.

[0107] Step 5.3: According to the confidence of each anchor point and the confidence coefficient, the modified confidence coefficient p' is calculated:

[0108] p i '=c i ×p i (1-l w )+w i ×l w

[0109] where c i is the confidence of the i-th anchor point, p i is the confidence coefficient of the i-th anchor point, l w is the influence coefficient of the window, and is a super parameter, which in this embodiment is 0.495. w i is the value of the anchor point in the added two-dimensional matrix, and in this embodiment, the added two-dimensional window is a Hanning window with a window length of 16.

[0110] Step 5.4: The position information of the anchor point with the maximum modified confidence parameter is taken as the prediction result, and the update coefficient is obtained according to the selected anchor point position, and the update coefficient calculation formula is as follows:

[0111] γ=c b ×p b ×l r

[0112] where γ is the update coefficient, c b is the confidence of the selected anchor point, p b is the confidence coefficient of the selected anchor point, and l r is the update coefficient, which is a super parameter, and in this embodiment is 0.385.

[0113] Step 5.5: Update the information of the calibration frame, and the representation of the calibration frame in this method is the center and width and height of the calibration frame. The calibration frame update formula is as follows:

[0114]

[0115] where x c ', y c ', w', h' are the center coordinates and width and height of the updated calibration frame, x c , y c , w, h are the coordinates before updating, γ is the update coefficient, and x1, y1 are the horizontal and vertical coordinates of the top-left corner and the horizontal and vertical coordinates of the bottom-right corner of the calibration frame predicted by the selected anchor point.

[0116] Step 5.6: mapping the predicted coordinates, projecting the template image coordinates in the predicted results back to the original image coordinates, the projection formula is as follows:

[0117]

[0118] In the formula, x cr ', y cr ', w r ', h r ' are the mapping back to the original image calibration frame center point coordinates and width and height, x c ', y c ', w', h' are the updated calibration frame center point coordinates and width and height, and λ is the change coefficient recorded in the data preprocessing.

[0119] Step 6: after calculating the predicted coordinates, insert a calibration frame in the possible position in the next frame prediction process, the calibration frame position calculation formula is as follows:

[0120]

[0121] In the formula, x cp ', y cp ', w p ', h p ' are the inserted calibration frame center point coordinates and width and height, x cr ', y cr ', w r ', h r ' are the calibration frame center point coordinates and width and height in the original image of the current frame after updating, x cr ", y cr ", w r ", h r " are the calibration frame center point coordinates and width and height in the original image of the previous frame before updating, and Δt is the updating speed of the calibration frame.

[0122] The above-described embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by those of ordinary skill in the art should fall within the scope of protection determined by the claims of the present application.

Claims

1. A single target tracking system based on an MCU, comprising a camera device and a user interaction device, characterized in that: The camera device is connected to a plurality of MCUs, and neural networks are arranged on the plurality of MCUs. The neural networks include an image feature fusion network. The input layer of the image feature fusion network is connected to a template image feature extraction network and an instance image feature extraction network. The output layer of the image feature fusion network is connected to a confidence output head and a position information output head. The plurality of MCUs are an algorithm operation MCU and a main control MCU connected in parallel with the camera device. The main control MCU is connected to a user interaction device. The algorithm operation MCU is deployed with a module image feature extraction network, an instance image feature extraction network and an image feature fusion network connected to the two extraction networks through an input layer. The main control MCU is deployed with a confidence output head and a position information output head connected to the output layer of the image feature fusion network. The feature fusion network consists of two parallel convolutional attention CBAM modules. Each CBAM module first uses two independent convolutional layers to further extract the features of the template image and the instance image. After the further extraction is completed, the three-dimensional image features are compressed into a two-dimensional image feature matrix by compressing the latter two dimensions. Then, the feature matrices of the template image and the instance image are multiplied and restored to three-dimensional image features. After obtaining the dimensions of each feature, the weight of each feature is obtained by the average pooling layer, two convolutional layers, and sigmoid operation. Finally, the channels are weighted to obtain the final calculation result. The specific tracking method steps of the system are as follows: Step 1: Obtain the template image and perform data pre-processing on the template image; after processing, input the template image feature extraction network to output the template image features; Step 2: Obtain the instance image and perform data pre-processing on the instance image; after processing, input the instance image feature extraction network to output the instance image features; Step 3: Input the template image features and instance image features into the image fusion network for calculation to obtain unprocessed prediction data; Step 4: Post-process the confidence data; Step 4.1: Divide the instance image after data pre-processing into anchor points and label the anchor points; each anchor point contains confidence data and location information data; Step 4.2: The confidence output data is 256x2 dual-channel data. The confidence data is the data of the second channel after the softmax function is processed. The confidence on the i-th anchor point is recorded as c i Step 5: Perform post-processing on the position data to obtain the predicted calibration frame. Based on the new calibration frame, the instance image of the next frame is re-determined, thereby completing the tracking of the target in a loop. Step 5.1: By calculating the data channel data of each anchor point, the position of the calibration frame in the instance image after data pre-processing is obtained. The calculation formula is as follows: Among them, x1, y1 are the horizontal and vertical coordinates of the upper left corner of the predicted calibration frame, x2, y2 are the horizontal and vertical coordinates of the lower right corner of the calibration frame, and x a ,y a are the horizontal and vertical coordinates of the anchor point, and v1, v2, v3, and v4 are the data of the anchor point position channel respectively; Step 5.2: Calculate the credibility coefficient p of each anchor point in turn. For ease of expression, the sz(w,h) operation and the change(x) operation are calculated as follows: The calculation formula of the credibility coefficient p is as follows: Among them, p is the credibility coefficient, p s is the border credibility coefficient, p r is the proportional credibility coefficient, λ is the coefficient of variation obtained in data pre-processing; w, h are the width and height of the calibration frame calculated in the previous frame; p k is a modifiable hyperparameter; Step 5.3: Calculate the modified credibility coefficient p' based on the confidence and credibility coefficient of each anchor point: p i '=c i ×p i (1-l w )+w i ×l w Among them, c i is the confidence on the i-th anchor point, p i is the credibility coefficient on the i-th anchor point, l w is the influence coefficient of the window, w i is the value of the corresponding position of the anchor point in the added two-dimensional matrix; Step 5.4: Take the position information of the anchor point with the largest corrected credible parameter as the prediction result, and obtain the update coefficient based on the selected anchor point position. The update coefficient calculation formula is as follows: γ=c b ×p b ×l r Where γ is the update coefficient, c b is the confidence of the selected anchor point, p b is the credibility coefficient of the selected anchor point, l r is the update coefficient, which is a hyperparameter; Step 5.5: Update the calibration frame information. In this method, the calibration frame is represented by the center, width, and height of the calibration frame. The formula for updating the calibration frame is as follows: Where x c ',y c ', w', h' are the coordinates of the center point of the calibration frame after update and the width and height, x c ,y c , w, h are the coordinates before updating, γ is the updating coefficient, x1, y1 are the horizontal and vertical coordinates of the upper left corner coordinates and the lower right corner coordinates of the calibration box predicted by the selected anchor point; Step 5.6: Map the predicted coordinates and project the template image coordinates in the predicted results back to the original image coordinates. The projection formula is as follows: In the formula, x cr ',y cr ', w r ',h r ' is the coordinates of the center point and width and height of the calibration frame mapped back to the original image, x c ',y c ', w', h' are the coordinates of the center point and width and height of the calibration frame after update, and λ is the coefficient of change recorded in data pre-processing Step 6: Perform interpolation operation.

2. The MCU-based single target tracking system according to claim 1, characterized in that: The module image feature extraction network and the instance image feature extraction network have the same structure, both of which are lightweight convolutional neural networks.

3. The MCU-based single target tracking system according to claim 1, characterized in that: The confidence output head and the position information output head are both composed of several convolution blocks, and each convolution block is composed of two convolution layers and one standard layer.

4. The MCU-based single target tracking system according to claim 1, characterized in that: Step 1 The specific steps are as follows: Step 1.1: Manually select the calibration frame. According to the range of the given calibration frame, determine the range of the image to be cropped. The cropping area is a square. When cropping the template image, the side length of the cropping area is: Where L is the side length of the cropping area square, w is the width of the calibration box, and h is the height of the calibration box; Step 1.2: Determine the specific area of ​​the cropping area. Assume that the center position of the calibration frame is (x0, y0). The final cropping area is a square area with (x0, y0) as the center and a side length of L. Step 1.3: Fill the portion beyond the image boundary with the image's average color. The average color is calculated by separating the RGB color channels of the image and averaging the colors of each channel, then combining them back into the RGB color channels. Step 1.4: Resize the cropped area to a uniform size. The template image is resized to 127x127.

5. The MCU-based single target tracking system according to claim 1, characterized in that: Step 2: Step 2.1: Determine the range of the image to be cropped based on the range of the given calibration frame. The cropping area is a square. If the instance image is the first frame, the given calibration frame is a manually selected calibration frame. If it is not the first frame, the calibration frame is the result predicted by the previous frame instance image. When cropping the instance image, the side length of the cropping area is: Where L is the side length of the cropping area square, w is the width of the calibration box, and h is the height of the calibration box; Step 2.2: Determine the specific area of ​​the cropping area. Assume that the center position of the calibration frame is (x0, y0). The final cropping area is a square area with (x0, y0) as the center and a side length of L. Step 2.3: Fill the portion beyond the image boundary with the image's average color. The average color is calculated by separating the RGB color channels of the image and taking the average of the colors of each channel, and then combining them back into the RGB color channels. Step 2.4: Resize the cropped area to a uniform size. The instance image is resized to 255x255.

Images