Target tracking method for fully mechanized coal mining face personnel and equipment based on neural network and correlation filtering

Abstract

The invention provides a target tracking method for fully mechanized coal mining face personnel and equipment based on a neural network and correlation filtering, and the method comprises the steps: calculating a gray difference value based on a previous frame of video image and a current frame of video image for a non-first frame of video image, and filtering the position information of a foreground target according to a gray threshold; obtaining a target position response diagram of a tracking target in the current frame video image according to the foreground target position information, the current frame video image and the trained related filter template; generating a plurality of candidate frames by taking the central point of the target position response diagram as the center, and sending the previous frame of video image, the target frame position information, the current frame of video image and the position information of the plurality of candidate frames into an improved dual modulation target estimation network to obtain the position information of the tracking target; according to the invention, the target mask with the target position information is calculated by using the gray frame difference method, the number of iterations of a correlation filtering algorithm is reduced through filtering of the target mask, and the tracking speed is increased.

Description

Technical Field

[0001] The invention relates to a target tracking method, in particular to a target tracking method and device for a fully mechanized mining face based on a neural network and correlation filtering. Background Art

[0003] Traditional coal mines use personnel positioning methods such as personnel positioning cards and RFID radio frequency identification technology. This method is costly and easily affected by underground electromagnetic fields; and it is unable to track and identify equipment on the working face. With the development of computer vision, existing coal mine working faces are equipped with cameras to collect on-site images. By using visual algorithms to process the real-time collected on-site images, the personnel and equipment on the fully mechanized working face can be located and identified.

[0004] Target tracking methods used in coal mines include traditional target tracking and deep learning target tracking. Traditional target tracking relies on manually designed feature extractors and machine learning algorithms, which are only applicable to scenes with simple backgrounds and obvious features. For coal mine environments with uneven lighting and dust and fog, it is difficult to extract target features from complex backgrounds, and the detection effect is limited.

[0005] The deep learning target tracking algorithm uses deep learning models such as convolutional neural networks (CNN) to automatically learn image features and directly applies them to the correlation filter or Struck tracking framework to obtain better tracking results. Therefore, it can be applied to narrow and confined spaces such as underground coal mines to achieve target tracking of personnel and equipment in fully mechanized mining working faces.

[0006] In order to solve the above problems, people have been seeking an ideal technical solution. Summary of the invention

[0007] The purpose of the present invention is to address the deficiencies of the prior art and thus provide a method and device for fully mechanized mining face target tracking based on neural network and correlation filtering.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is: a target tracking method for personnel and equipment in a fully mechanized mining face based on a neural network and correlation filtering, comprising the following steps:

[0009] Obtain and annotate images of personnel and equipment in underground coal mines;

[0010] An improved dual-modulation target estimation network is trained based on the annotated coal mine personnel and equipment image set, and the dual-modulation target estimation network is used to predict the IOU value of the candidate box in the current frame image;

[0011] Obtain the surveillance video stream, determine the tracking target based on the first frame of the video image, and train the relevant filter template;

[0012] For a non-first frame of video images, a grayscale difference is calculated based on the previous frame of video images and the current frame of video images, and foreground target position information is filtered out according to the grayscale threshold;

[0013] According to the foreground target position information, the current frame video image and the trained correlation filter template, a target position response map of the tracked target in the current frame video image is obtained;

[0014] Generate multiple candidate frames with the center point of the target position response map as the center, send the previous frame video image, the target frame position information, the current frame video image and the position information of multiple candidate frames to the improved dual modulation target estimation network to obtain the position information of the tracked target;

[0015] Repeat the above steps until the monitoring video stream ends.

[0016] Furthermore, the target estimation network includes a reference branch and a test branch, wherein the reference branch calculates a channel modulation vector with target information according to the previous frame of video image and target frame position information;

[0017] The test branch includes ResNet18 residual blocks 1-4, Prpool pooling operation and IoU prediction module, wherein the ResNet18 residual blocks 1-4 are used to extract candidate frame features of the current frame video image, spatially modulate the candidate frame features of the current frame video image based on the target features of the previous frame video image, obtain the candidate frame features after spatial position information enhancement, and perform pooling operation in combination with the position information of the candidate frame; and use the channel modulation vector to perform channel modulation on the candidate frame features after the pooling operation to obtain the candidate frame features after channel information enhancement;

[0018] Based on the IoU prediction module, the candidate box features after channel information enhancement are processed to obtain the location information of the tracked target.

[0019] Furthermore, the specific steps of spatially modulating the candidate frame features based on the target features of the previous frame of video image to obtain the candidate frame features after the spatial position information is enhanced include:

[0020] The candidate box features After reconstruction, the generated Where |M|=W×H;

[0021] The target features of the previous frame of video image Perform global average pooling to generate To aggregate spatial features;

[0022] After Q and M are processed by two-norm regularization, matrix inner product operation is performed, and softmax normalization operation is performed to obtain the probability map of the pixel-level similarity between Q and M.

[0023] Perform matrix inner product operation on S and M to obtain P∈R C×1 ;

[0024] Perform matrix inner product operation on P and S to obtain the similarity degree U;

[0025] Reconstruct U to obtain the spatial dimension encoding matrix U1;

[0026] The corresponding elements of the spatial dimension encoding matrix U1 and the candidate box feature F are added to obtain the candidate box feature with enhanced spatial position information.

[0027] A second aspect of the present invention provides a fully mechanized mining face target tracking device based on correlation filtering, comprising:

[0028] The labeling module is used to obtain and label images of personnel and equipment in underground coal mines;

[0029] The target estimation network training module trains an improved dual-modulation target estimation network based on the labeled coal mine personnel and equipment image set, and the dual-modulation target estimation network is used to predict the IOU value of the candidate box in the current frame image;

[0030] A video stream acquisition module is used to acquire monitoring video streams;

[0031] An initialization module, used for determining a tracking target based on a first frame of video image and training a related filter template;

[0032] The position response map acquisition module is used to calculate the grayscale difference between the previous frame of video image and the current frame of video image for non-first frame of video image, and filter out the foreground target position information according to the grayscale threshold; obtain the target position response map of the tracked target in the current frame of video image according to the foreground target position information, the current frame of video image and the trained related filter template;

[0033] A candidate box generation module is used to generate multiple candidate boxes centered on the center point of the target position response map;

[0034] The tracking target position acquisition module is used to send the position information of multiple candidate boxes and the current frame video image to the improved dual modulation target estimation network to obtain the position information of the tracking target.

[0035] A third aspect of the present invention provides a data prediction device, the device comprising at least: a memory, a communication bus and a processor, wherein:

[0036] The memory is used to store the data prediction program;

[0037] The communication bus is used to realize the communication connection between the processor and the memory;

[0038] The processor is used to execute the target tracking program stored in the memory to implement the steps of the target tracking method for personnel and equipment in the comprehensive mining working face based on neural network and correlation filtering as described in the first aspect.

[0039] The fourth aspect of the present invention provides a computer storage medium, which stores a target tracking program. When the target tracking program is executed by the processor, it implements the steps of the target tracking method for personnel and equipment in a comprehensive mining face based on neural networks and correlation filtering as described in the first aspect.

[0040] The fifth aspect of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the target tracking method for personnel and equipment in a comprehensive mining face based on neural networks and correlation filtering as described in the first aspect.

[0041] Compared with the prior art, the present invention has outstanding substantive features and significant progress. Specifically, the present invention proposes a comprehensive mining face target tracking method that integrates correlation filtering, target spatial modulation and IoU modulation with frame difference enhancement; specifically, the frame difference method is used to calculate the target mask with target position information, and the number of iterations of the correlation filtering algorithm is reduced by filtering the target mask, thereby increasing the tracking speed; by calculating the similar area between the target with clear features in the previous frame and the target in the current frame, and using the similar spatial dimension to enhance the spatial position of the appearance features of the target in the current frame, the target appearance is reconstructed and the expression of the target appearance features is improved. This operation and the IoU modulation in the network together constitute a dual modulation mechanism, which constructs a complete coupling relationship between the current frame and the previous frame from the spatial dimension and the channel dimension, and can better cope with scenes such as target appearance loss, thereby improving the target tracking effect in the underground coal mine environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of the tracking method described in Example 1 of the present invention.

[0043] Figure 2 It is a schematic diagram of the process of obtaining the target response graph in Example 1 of the present invention.

[0044] Figure 3 It is a schematic diagram of the spatial modulation process in Example 1 of the present invention.

[0045] Figure 4 It is a flowchart of the tracking method corresponding to the second frame image of Example 1 of the present invention.

[0046] Figure 5 It is a principle block diagram of the tracking system described in Example 2 of the present invention.

[0047] Figure 6 These are the experimental results of the present invention on pedestrian videos and coal mining machine drums.

[0048] Figure 7 These are the experimental results of ATOM on pedestrian videos and coal mining machine drums. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] Environmental factors such as uneven lighting and dust and fog obstruction in coal mines make it difficult to extract target features from complex backgrounds, which makes it easy for target appearance features to be lost during target tracking. The present invention proposes a comprehensive mining face target tracking method that combines correlation filtering with frame difference enhancement, target space modulation and IoU modulation. It builds a complete coupling relationship between the current frame and the previous frame from the spatial dimension and channel dimension, which can better cope with scenarios such as target appearance loss and improves the target tracking effect in coal mine underground environments.

[0051] The technical solution of the present invention is further described in detail below through specific implementation methods.

[0052] Example 1

[0053] This embodiment provides a method for tracking targets of personnel and equipment in a fully mechanized mining face based on a neural network and correlation filtering, comprising the following steps:

[0054] Step 1: Obtain and annotate an image set of underground personnel and equipment in a coal mine. Specifically, annotate tracking targets such as coal mining machine drums, side guards, and operating personnel in the image.

[0055] Step 2: training an improved dual-modulation target estimation network based on the annotated coal mine personnel and equipment image set, wherein the dual-modulation target estimation network is used to predict the IOU value of the candidate box in the current frame image.

[0056] Specifically, the target estimation network includes a reference branch and a test branch, the reference branch includes ResNet18 residual blocks 1-4 and Prpool pooling operations, and a channel modulation vector with target information is calculated based on the previous frame of video image and target frame position information; in one embodiment, the reference branch includes ResNet18 residual blocks 1-4, Prpool pooling operations and an IoU modulation module, and output feature maps are extracted from residual blocks 3 and residual blocks 4 respectively, and PrPool pooling operations are performed together with the target frame position information in the previous frame to extract the depth information of the target, and the target depth information is sent to the IoU modulation module to calculate the channel modulation vector with the target information.

[0057] The test branch includes ResNet18 residual blocks 1-4, Prpool pooling operation and IoU prediction module. The ResNet18 residual blocks 1-4 are used to extract candidate frame features of the current frame video image, spatially modulate the candidate frame features of the current frame video image based on the target features of the previous frame video image, obtain the candidate frame features after spatial position information enhancement, and perform pooling operation in combination with the position information of the candidate frame; and use the channel modulation vector to perform channel modulation on the candidate frame features after the pooling operation to obtain the candidate frame features after channel information enhancement; based on the IoU prediction module, the candidate frame features after channel information enhancement are processed to obtain the position information of the tracking target.

[0058] Step 3, obtain the monitoring video stream, determine the tracking target based on the first frame of the video image and train the relevant filter template;

[0059] Specifically, the specific steps of determining the tracking target based on the first frame of video image and training the relevant filter template are: performing target detection from the first frame of video image, obtaining the target frame, and extracting the features of the target frame, and training the relevant filter template based on the features of the target frame.

[0060] Specifically, the tracking targets in coal mines include personnel and equipment in the fully mechanized mining face, where the equipment includes but is not limited to coal mining machine drums, side guards, etc.

[0061] According to the convolution theorem, the convolution operation of a function in the frequency domain is equal to the product in the frequency domain as shown in the formula:

[0062]

[0063] By converting the convolution operation of two signals in the time domain into a product operation in the frequency domain through the following formula, the amount of calculation can be greatly reduced and the operation speed can be accelerated:

[0064] G=F·H *

[0065] After simplified calculation, the results are as follows:

[0066]

[0067] Among them, H * It is the filter template to be found, and F is the Fourier transform of the original image.

[0068] Specifically, when training the relevant filter template, the L2 classification loss function is selected. The expression of the L2 classification loss function is as follows:

[0069]

[0070] Among them, y i is a Gaussian distribution centered on the first frame target, used as the training label. j ; w) is the confidence map of the target prediction of the current frame; γ j Used to control the influence of each training sample on the model; the degree of regularization is λ k To control, it is used to prevent the trained tracker from overfitting and improve the generalization ability of the tracker.

[0071] By minimizing the above formula through the steepest descent algorithm, we can train f(x j ; w), w is the parameter in the relevant filter template.

[0072] It should be noted that after obtaining the target position information of the tracked target in the current frame, it is necessary to extract the features of the target frame and update the relevant filter template based on the features of the target frame.

[0073] Step 4: for a non-first frame of video image, a grayscale difference is calculated based on the previous frame of video image and the current frame of video image, and foreground target position information is filtered out according to the grayscale threshold;

[0074] According to the foreground target position information, the current frame video image and the trained correlation filter template, a target position response map of the tracked target in the current frame video image is obtained.

[0075] The steps of obtaining the target position response map of the tracked target in the current frame video image according to the foreground target position information, the current frame video image and the trained correlation filter template are as follows: Figure 2 As shown, including:

[0076] Input the Rth frame video image and the R-1th frame video image, and grayscale the two frames of video images respectively, and make a difference between the two grayscale images, and use the grayscale threshold to filter out high-quality foreground target position information, as follows:

[0077] X t (x,y)=|X t (x,y)-Xt-1 (x,y)|

[0078]

[0079] T p is the grayscale difference; X t (x,y) is the grayscale value of the pixel (x,y).

[0080] In the tracking stage, based on the foreground target position information, the residual blocks 1-4 of the backbone network Resnet18 are used to extract features of the current frame video image, and the output feature x of the residual block 4 is sent to the trained correlation filter template to predict the confidence map of the target center position of the current frame image:

[0081] f(x;w)=φ2(w2*φ1(w1*x))

[0082] Where w = {w1, w2} is the correlation filter template parameter, φ1 and φ2 are activation functions; * represents multi-channel convolution; f(x; w) represents the position response map of the target in the next frame, that is, the prediction result of the target center point position.

[0083] Step 5, after obtaining the target position response map, generate multiple candidate frames with the center point of the target position response map as the center, send the previous frame video image, the target frame position information, the current frame video image and the position information of multiple candidate frames to the improved dual modulation target estimation network to obtain the position information of the tracked target.

[0084] Among them, multiple candidate boxes are generated by shifting the center point of the target position response map, such as moving up, down, left, and right. Since the size of the offset follows a Gaussian distribution, it is called Gaussian noise, or noise for short.

[0085] Specifically, the idea of ​​spatially modulating the candidate frame features based on the target features of the previous frame of video image to obtain the candidate frame features with enhanced spatial position information is as follows:

[0086] A deep learning network is used to extract target features in the first frame of video image and candidate frame features in the current frame of video image;

[0087] The similarity calculation is used to obtain the correlation between each candidate frame feature and the target feature in the previous frame of the video image, and the memory query is used to select the target feature with high correlation to enhance the candidate frame feature in the current frame.

[0088] The specific steps are:

[0089] The candidate frame features output from residual block 4 after the current frame video image is processed by ResNet18 Perform global average pooling to generate To aggregate spatial features;

[0090] The target features output from residual block 4 after the previous frame of video image is processed by ResNet18 After reconstruction, the generated Where |M|=W×H;

[0091] After Q and M are processed by two-norm regularization, matrix inner product operation is performed, and softmax normalization operation is performed to obtain the probability map of the pixel-level similarity between Q and M. The specific formula is as follows:

[0092]

[0093] Perform matrix inner product operation on S and M to obtain P∈R C×1 ;

[0094] Perform matrix inner product operation on P and S to obtain the similarity degree U∈R C×HW ;

[0095] Reconstruct U to obtain the C×H×W spatial dimension encoding matrix U1;

[0096] The corresponding elements of the spatial dimension encoding matrix U1 and the candidate box feature F are added to obtain the candidate box feature with enhanced spatial position information.

[0097] The channel modulation vector is used to perform channel modulation on the candidate box features after the pooling operation to obtain the candidate box features after channel information enhancement; the candidate box features after channel information enhancement are processed based on the IoU prediction module to obtain the position information of the tracked target.

[0098] Repeat steps 3-6 until the monitoring video stream ends.

[0099] It should be noted that after processing the candidate box features after channel information enhancement based on the IoU prediction module, a preset number of candidate boxes with high scores are selected as the best candidate boxes, and the position coordinate information of the best candidate boxes is averaged, and the obtained position information average is used as the final predicted position of the tracking target. Preferably, N is 3, that is, the average of the three candidate boxes with the top three predicted scores is taken as the final prediction result of the target.

[0100] Or in another embodiment, after the IoU prediction module processes the candidate box features after the channel information is enhanced, the candidate box with the highest score is selected as the best candidate box, and the position information of the best candidate box is used as the final predicted position of the tracking target.

[0101] Compared with directly selecting the candidate box with the highest score as the best candidate box, selecting the average value of the position information of the top N candidate boxes can further reduce the error of the position information.

[0102] In order to verify the advantages of the proposed algorithm in the underground coal mine tracking scene, ATOM and the method proposed in this paper were used to conduct a comparative experiment on the tracking algorithm in the underground image set, and two tracking sequence results were selected for visualization. The experimental results of this method on pedestrian videos and videos of coal mining machine drums are shown in Figure 2. Figure 6 The experimental results of ATOM on pedestrian videos and videos of coal mining machine drums are shown in Figure 7 shown.

[0103] Experiments show that the proposed algorithm can improve the tracking effect of the model in underground environments, and performs well in comparative tests of underground test video sequences and other traditional tracking methods.

[0104] Example 2

[0105] This embodiment provides a fully mechanized mining face target tracking device based on correlation filtering, comprising:

[0106] The labeling module is used to obtain and label images of personnel and equipment in underground coal mines;

[0107] The target estimation network training module trains an improved dual-modulation target estimation network based on the labeled coal mine personnel and equipment image set, and the dual-modulation target estimation network is used to predict the IOU value of the candidate box in the current frame image;

[0108] A video stream acquisition module is used to acquire monitoring video streams;

[0109] An initialization module, used for determining a tracking target based on a first frame of video image and training a related filter template;

[0110] The position response map acquisition module is used to calculate the grayscale difference between the previous frame of video image and the current frame of video image for non-first frame of video image, and filter out the foreground target position information according to the grayscale threshold; obtain the target position response map of the tracked target in the current frame of video image according to the foreground target position information, the current frame of video image and the trained related filter template;

[0111] A candidate box generation module is used to generate multiple candidate boxes centered on the center point of the target position response map;

[0112] The tracking target position acquisition module is used to send the position information of multiple candidate boxes and the current frame video image to the improved dual modulation target estimation network to obtain the position information of the tracking target.

[0113] Specifically, the working steps of the comprehensive mining working face target tracking device based on correlation filtering are shown in the figure.

[0114] Example 3

[0115] This embodiment provides a data prediction device, the device at least comprising: a memory, a communication bus and a processor.

[0116] in:

[0117] The memory is used to store the data prediction program;

[0118] The communication bus is used to realize the communication connection between the processor and the memory;

[0119] The processor is used to execute the target tracking program stored in the memory to implement the steps of the target tracking method for personnel and equipment in the comprehensive mining working face based on neural network and correlation filtering as described in Example 1.

[0120] Example 4

[0121] This embodiment provides a computer storage medium, which stores a target tracking program. When the target tracking program is executed by the processor, the steps of the target tracking method for personnel and equipment in a comprehensive mining face based on neural network and correlation filtering as described in Example 1 are implemented.

[0122] Example 5

[0123] This embodiment provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the target tracking method for personnel and equipment in a comprehensive mining face based on neural networks and correlation filtering as described in Example 1.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present invention, which should be included in the scope of the technical solution for protection of the present invention.

Claims

1. A target tracking method for personnel and equipment in a fully mechanized mining face based on neural network and correlation filtering, characterized in that: The following steps are involved: Obtain and annotate images of personnel and equipment in underground coal mines; An improved dual-modulation target estimation network is trained based on the annotated coal mine personnel and equipment image set, and the dual-modulation target estimation network is used to predict the IOU value of the candidate box in the current frame image; Obtain the surveillance video stream, determine the tracking target based on the first frame of the video image, and train the relevant filter template; For a non-first frame of video images, a grayscale difference is calculated based on the previous frame of video images and the current frame of video images, and foreground target position information is filtered out according to the grayscale threshold; Obtain a target position response map of the tracked target in the current frame video image according to the foreground target position information, the current frame video image and the trained correlation filter template; Generate multiple candidate frames with the center point of the target position response map as the center, send the previous frame video image, the target frame position information, the current frame video image and the position information of multiple candidate frames to the improved dual modulation target estimation network to obtain the position information of the tracked target; Repeat the above steps until the monitoring video stream ends.

2. According to claim 1, a method for tracking targets of personnel and equipment in a fully mechanized mining face based on a neural network and correlation filtering, characterized in that: The target estimation network includes a reference branch and a test branch, wherein the reference branch calculates a channel modulation vector with target information according to the previous frame of video image and target frame position information; The test branch includes ResNet18 residual blocks 1-4, Prpool pooling operation and IoU prediction module, wherein the ResNet18 residual blocks 1-4 are used to extract candidate frame features of the current frame video image, spatially modulate the candidate frame features of the current frame video image based on the target features of the previous frame video image, obtain the candidate frame features after spatial position information enhancement, and perform pooling operation in combination with the position information of the candidate frame; and use the channel modulation vector to perform channel modulation on the candidate frame features after the pooling operation to obtain the candidate frame features after channel information enhancement; Based on the IoU prediction module, the candidate box features after channel information enhancement are processed to obtain the location information of the tracked target.

3. According to claim 2, a method for tracking targets of personnel and equipment in a fully mechanized mining face based on a neural network and correlation filtering is characterized in that: The specific steps of spatially modulating the candidate frame features based on the target features of the previous frame video image to obtain the candidate frame features after the spatial position information is enhanced include: The candidate box features After reconstruction, the generated Where |M|=W×H; The target features of the previous frame of video image Perform global average pooling to generate To aggregate spatial features; After Q and M are processed by two-norm regularization, matrix inner product operation is performed, and softmax normalization operation is performed to obtain the probability map of the pixel-level similarity between Q and M. Perform matrix inner product operation on S and M to obtain P∈R C×1 ; Perform matrix inner product operation on P and S to obtain the similarity degree U; Reconstruct U to obtain the spatial dimension encoding matrix U1; The corresponding elements of the spatial dimension encoding matrix U1 and the candidate box feature F are added to obtain the candidate box feature with enhanced spatial position information.

4. A method for tracking targets of personnel and equipment in a fully mechanized mining face based on a neural network and correlation filtering according to claim 1 or 2, characterized in that: The specific steps of generating multiple candidate frames with the center point of the target position response map as the center include: adding Gaussian noise based on the target position coordinates of the previous frame of video image with the center point of the target position response map as the center to generate multiple candidate frames.

5. A method for tracking targets of personnel and equipment in a fully mechanized mining face based on a neural network and correlation filtering according to claim 1 or 2, characterized in that: After processing the candidate box features after channel information enhancement based on the IoU prediction module, a preset number of candidate boxes with high scores are selected as the best candidate boxes, and the position coordinate information of the best candidate boxes is averaged, and the obtained position information mean is used as the final predicted position of the tracking target.

6. A method for tracking targets of personnel and equipment in a fully mechanized mining face based on a neural network and correlation filtering according to claim 1 or 2, characterized in that: After processing the candidate box features after channel information enhancement based on the IoU prediction module, the candidate box with the highest score is selected as the best candidate box, and the position information of the best candidate box is used as the final predicted position of the tracking target.

7. A fully mechanized mining face target tracking device based on correlation filtering, characterized in that: include: The labeling module is used to obtain and label images of personnel and equipment in underground coal mines; The target estimation network training module trains an improved dual-modulation target estimation network based on the labeled coal mine personnel and equipment image set, and the dual-modulation target estimation network is used to predict the IOU value of the candidate box in the current frame image; A video stream acquisition module is used to acquire monitoring video streams; An initialization module, used for determining a tracking target based on a first frame of video image and training a related filter template; A position response map acquisition module is used to calculate the grayscale difference between the previous frame of video image and the current frame of video image for a non-first frame of video image, and filter out the foreground target position information according to the grayscale threshold; According to the foreground target position information, the current frame video image and the trained correlation filter template, a target position response map of the tracked target in the current frame video image is obtained; A candidate box generation module is used to generate multiple candidate boxes centered on the center point of the target position response map; The tracking target position acquisition module is used to send the position information of multiple candidate boxes and the current frame video image to the improved dual modulation target estimation network to obtain the position information of the tracking target.

8. A data prediction device, characterized in that: The device at least includes: a memory, a communication bus and a processor. in: The memory is used to store the data prediction program; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the target tracking program stored in the memory to implement the steps of the target tracking method for personnel and equipment in a comprehensive mining face based on a neural network and correlation filtering as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores a target tracking program, and when the target tracking program is executed by the processor, the steps of the target tracking method for personnel and equipment in a comprehensive mining face based on a neural network and correlation filtering are implemented as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the target tracking method for personnel and equipment in a comprehensive mining face based on a neural network and correlation filtering as described in any one of claims 1 to 6 are implemented.

Images