Multi-target constructor tracking method, system and equipment for power grid field operation, and medium

Abstract

The invention relates to the technical field of multi-target tracking, and discloses a power grid field operation-oriented multi-target constructor tracking method, system equipment and medium, and the method comprises the steps: predicting the position of a constructor in a next frame through a coordinate prediction function based on a time sequence, and reducing the calculation time complexity; by setting a detection frame-appearance feature extraction module, features of different levels can be extracted, and a feature map which is small in calculation volume, high in expression ability and rich in feature is obtained; and through a subsequent appearance feature-mixed attention mechanism module, feature extraction is enhanced by using a channel attention mechanism, background noise interference is effectively suppressed through a space attention mechanism, and finally, a feature map with higher discrimination is obtained by fusing feature maps of various scales, so that the characterization capability of the appearance features is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of power operation tracking technology, and in particular to a method, system, equipment and medium for tracking multi-target construction personnel in power grid field operations. Background Technology

[0002] Power grid operation sites are typically complex environments with dense populations and dynamic factors such as personnel movement over time. Traditional safety management methods based on manual monitoring and static cameras are insufficient to meet the needs of efficient safety management in modern power grid operation sites. Therefore, multi-target construction personnel tracking methods based on computer vision and deep learning technologies have gradually become a research hotspot.

[0003] Currently, multi-target construction worker tracking refers to the simultaneous detection and tracking of multiple target personnel from a video sequence using computer vision technology. Unlike single-target tracking, multi-target tracking requires solving complex problems such as data association between targets and occlusion handling. By combining target detection, data association (such as the Hungarian algorithm), and trajectory management techniques, multi-target tracking can generate continuous motion trajectories for each target. The technological development of multi-target tracking has evolved from traditional methods to deep learning. Traditional methods mainly rely on Kalman filtering and the Hungarian algorithm; Kalman filtering is used to predict the position and velocity of the target, while the Hungarian algorithm is used for data association. With the rise of deep learning, detection-by-detection has become the mainstream method. By combining target detection models (such as YOLO and Faster R-CNN) and appearance feature extraction models (such as ReID), tracking performance has been significantly improved. In recent years, methods based on attention mechanisms and multi-task learning have further improved the accuracy and robustness of multi-target tracking, driving the development of this field.

[0004] However, current technology is highly dependent on the physical features of construction workers in images. When the physical features of construction workers change significantly, the extracted physical features also change significantly, leading to a decrease in tracking performance. At the same time, existing technology does not perform well in resisting interference when performing multi-target tracking, has low robustness, and is prone to the problem of losing targets. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, this invention provides a multi-target construction worker tracking method and system for power grid field operations, which solves the problems of current technology being highly dependent on the physical features of construction workers in images, resulting in poor tracking performance, poor anti-interference performance, and low robustness.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, the present invention provides a method for tracking multi-target construction personnel in power grid field operations, including: acquiring video data of power grid operation sites;

[0009] A raw image queue is constructed by extracting frame images from video data;

[0010] The original image queue is preprocessed to obtain a second image queue containing the detection boxes of construction workers;

[0011] The next frame prediction box for all construction worker detection boxes in the second image queue is generated by a time-based coordinate prediction function, thus obtaining the third image queue.

[0012] The images of construction workers in the detection boxes and prediction boxes in the third image queue are input into the feature extraction network to generate multiple feature maps of the construction workers. After being fused through the attention mechanism, the corresponding fused feature map of the construction workers is obtained. After compression, the shape feature vector of the construction workers is obtained.

[0013] By calculating the geometric and shape similarity between the predicted and detected bounding boxes in the third image queue, a cost matrix is ​​constructed, and multiple optimal matching pairs are selected for target matching of multiple construction workers; among them, the shape similarity is calculated based on the shape feature vector.

[0014] Based on the best matching pair obtained in the current frame, the center position of the detection box corresponding to each construction worker in the third image queue is obtained, and the corresponding movement sequence is constructed by the center positions of each construction worker in consecutive frames to obtain the movement trajectory of each construction worker.

[0015] As a preferred embodiment of the multi-target construction worker tracking method for power grid field operations described in this invention, a third image queue is obtained by generating the next frame prediction boxes of all construction worker detection boxes in the second image queue using a time-based coordinate prediction function, including:

[0016] For any frame in the second image queue, obtain the position of the detection box for the construction worker and get the coordinate information of each corner of the detection box;

[0017] The time-based coordinate prediction function is expressed as:

[0018]

[0019] Among them, c j c represents the coordinates of a corner of the detection box for a construction worker in the j-th frame of the image; i -c i-1 k represents the difference between two coordinates. base This represents the set baseline value; α is the weighting coefficient; |cj -c0| indicates c j The straight-line distance between the two coordinates c0 and c0; Indicates taking α|c j The integer less than -c0|; T(c) represents the weighting coefficient. j ) represents the predicted coordinates of a corner of a construction worker's detection frame in frame j+1; k represents the exponential parameter that controls the distribution of displacement weights in historical frames.

[0020] The predicted coordinates of the four corners of a detection box are calculated using the coordinate prediction function and then connected to obtain the predicted box of the next frame.

[0021] The prediction boxes for the next frame are generated by predicting all detection boxes in the second image queue, resulting in the third image queue.

[0022] As a preferred embodiment of the multi-target construction worker tracking method for power grid field operations described in this invention, the images of construction workers within the detection boxes and prediction boxes in the third image queue are input into a feature extraction network to generate multiple feature maps of the construction workers, including:

[0023] The construction worker image within a frame in the third image queue is input into the feature extraction network. After passing through a two-dimensional convolutional layer, a first feature map G1 of the construction worker is obtained.

[0024] Using the first feature map G1 as input, after passing through the bounding box-shape feature extraction module, the nth feature map G of the construction worker is obtained. n .

[0025] As a preferred embodiment of the multi-target construction worker tracking method for power grid field operations described in this invention, the feature extraction network includes a bounding box-shape feature extraction module, and the extraction process specifically includes:

[0026] The first feature map G1 is used as input and passed through a two-dimensional convolutional layer to obtain the multi-scale feature map P1 of the construction workers.

[0027] After passing the multi-scale feature map P1 through the maximum pooling layer, the maximum pooling feature map P2 of the construction personnel is obtained.

[0028] After passing the multi-scale feature map P1 through a hollow convolutional layer, we obtain the hollow convolutional feature map P5 of the construction workers.

[0029] After passing the maximum pooling feature map P2 of the construction workers through a shuffling convolutional neural network, the first shuffling convolutional feature map P3 of the construction workers is obtained.

[0030] After passing the construction worker vacuous convolutional feature map P5 through a shuffling convolutional neural network, we obtain the second construction worker shuffling convolutional feature map P6.

[0031] After batch normalization of P3 and P6, the first normalized feature map P4 and the second normalized feature map P7 of the construction personnel are obtained respectively.

[0032] By splicing the first normalized feature map P4 and the second normalized feature map P7, the spliced ​​feature map P8 of the construction workers is obtained.

[0033] After performing a transpose convolution operation on the spliced ​​feature map P8 of the construction workers, we obtain the transpose convolution feature map P9 of the construction workers.

[0034] The construction worker transposed convolutional feature map P9 is residually connected with the multi-scale feature map P1 to obtain the construction worker second feature map G2.

[0035] The second feature map G2 is used as input, and after passing through the second bounding box-shape feature extraction module, the construction worker shape extraction feature map G3 is obtained.

[0036] Repeat the extraction operation until the feature map G is obtained. n-1 After inputting into the (n-1)th bounding box-shape feature extraction module, the extracted feature map G of the construction worker's shape is obtained. n This forms the first feature map sequence S = {G1, G2, G3, ..., G...} n}; where n-1 is the number of the structures.

[0037] As a preferred embodiment of the multi-target construction worker tracking method for power grid field operations described in this invention, the step of obtaining the corresponding construction worker fusion feature map after fusion through an attention mechanism includes:

[0038] The feature map sequence S is input into the shape feature-hybrid attention mechanism module;

[0039] For any feature map G in the feature map sequence S, input the arbitrary feature map G into the compression and activation submodule in the attention mechanism to obtain the first channel feature map G′1 after channel attention weighting;

[0040] Inputting any feature map G into the efficient channel attention network in the attention mechanism yields the second channel feature map G′2 after efficient channel attention weighting.

[0041] By inputting any feature map G into the nonlocal attention submodule of the attention mechanism, the third channel feature map G′3 after spatial attention weighting is obtained.

[0042] The three channel feature maps are weighted and fused to obtain a feature map G′ of construction workers enhanced by feature fusion.

[0043] All the mixed feature maps generated by the first feature map sequence S constitute the second feature map sequence S′;

[0044] The feature map with the highest resolution in the second feature map sequence S′ is selected as the upsampling target. After upsampling, a third feature map sequence S″ with the same resolution is obtained.

[0045] Multi-scale feature fusion is performed on all feature maps in the third feature map sequence S″ to obtain the fused feature map G of construction workers. n+1 .

[0046] As a preferred embodiment of the multi-target construction worker tracking method for power grid field operations described in this invention, the compressed external feature vector of the construction worker includes:

[0047] Integrate the feature map of construction workers G n+1 After being input into global average pooling compression, the pooled feature vector E1 of the construction workers is obtained;

[0048] The pooled feature vector E1 is input into the fully connected layer to obtain the fully connected feature vector E2 of the construction workers;

[0049] After inputting the fully connected feature vector E2 into the L2 normalization layer, the shape feature vector E3 of the construction worker is obtained.

[0050] As a preferred embodiment of the multi-target construction worker tracking method for power grid field operations described in this invention, a cost matrix is ​​constructed by calculating the geometric and shape similarities between the predicted bounding boxes and detection boxes in the third image queue, and multiple optimal matching pairs are selected, including:

[0051] For any frame a in the third image queue q If q = 0, then a q For the 0th frame image, at this time, a q All construction workers within the detection boxes are marked as tracking personnel;

[0052] If q≠0, based on the prediction box of a certain tracker and a q For a certain detection box on the screen, the intersection-union ratio (IoU) of the two boxes and the distance to the center point are calculated, thus obtaining the geometric similarity between a certain tracking person and the construction personnel within the detection box;

[0053] The similarity of the shapes of construction workers within two boxes is obtained by calculating the shape feature vectors using cosine distance.

[0054] Combining geometric similarity and shape similarity, a two-dimensional cost matrix C is constructed, where the number of rows in the matrix represents the number of trackers and the number of columns represents the number of detection boxes;

[0055] Based on the cost matrix C, cost pairs are selected using the Jonker-Volgenant linear allocation optimization algorithm to obtain multiple optimal matching pairs (t,i).

[0056] Secondly, the present invention provides a multi-target construction worker tracking system for power grid field operations, comprising:

[0057] The acquisition module is used to acquire video data from power grid operation sites.

[0058] The first extraction module is used to construct an original image queue by extracting frame images from video data;

[0059] The preprocessing module is used to preprocess the original image queue to obtain a second image queue containing the detection boxes of construction workers;

[0060] The generation module is used to generate the next frame prediction box of all construction worker detection boxes in the second image queue through a time-based coordinate prediction function, so as to obtain the third image queue.

[0061] The second extraction module is used to input the images of construction workers within the frame in the third image queue and the second image queue into the feature extraction network respectively, generate multiple feature maps of the construction workers, and obtain the corresponding fused feature maps of construction workers in each queue through the attention mechanism. After compression, the shape feature vectors of construction workers in each queue are obtained.

[0062] The matching module constructs a cost matrix by calculating the geometric and shape similarity between the predicted bounding boxes and the detection boxes in the third image queue, and selects multiple optimal matching pairs for target matching of multiple construction workers; among them, the shape similarity is calculated based on the shape feature vector.

[0063] The trajectory generation module is used to obtain the center position of the detection box corresponding to each construction worker in the third image queue based on the best matching pair obtained in the current frame, and to obtain the movement trajectory of each construction worker by constructing the corresponding movement sequence through the center positions of each construction worker in consecutive frames.

[0064] Thirdly, the present invention provides a computer device, comprising:

[0065] Memory and processor;

[0066] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of a multi-target construction personnel tracking method for power grid field operations.

[0067] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the multi-target construction personnel tracking method for power grid field operations.

[0068] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention predicts the position of construction workers in the next frame using a time-based coordinate prediction function, reducing the computational complexity of prediction time; by setting a detection box-shape feature extraction module, it can extract features at different levels, resulting in a shape feature map with low computational consumption, high expressive power, and rich features; furthermore, through the subsequent shape feature-hybrid attention mechanism module, it uses channel attention mechanism to enhance feature extraction and spatial attention mechanism to effectively suppress background noise interference; finally, by fusing feature maps at various scales, it obtains a more discriminative feature map, significantly improving the representational ability of shape features. Attached Figure Description

[0069] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a schematic diagram of the overall process of a multi-target construction personnel tracking method for power grid field operations according to an embodiment of the present invention.

[0071] Figure 2 This is a schematic diagram of the network structure for extracting the physical features of construction workers in a multi-target construction worker tracking method for power grid field operations according to an embodiment of the present invention.

[0072] Figure 3 This is a schematic diagram of the BB-AFE module in the network structure of the multi-target construction worker tracking method for power grid field operations according to an embodiment of the present invention.

[0073] Figure 4 This is a schematic diagram of the AC-HAM process in the network structure for extracting the physical features of construction workers in a multi-target construction worker tracking method for power grid field operations according to an embodiment of the present invention. Detailed Implementation

[0074] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0075] Example 1, referring to Figure 1 As an embodiment of the present invention, a multi-target construction worker tracking method for power grid field operations is provided, comprising:

[0076] S100: Acquire video data from power grid operation sites;

[0077] S200: Constructs a raw image queue by extracting frame images from video data;

[0078] S300: Preprocess the original image queue to obtain a second image queue containing the detection boxes of construction workers;

[0079] S400: Generate the next frame prediction box of all construction worker detection boxes in the second image queue using a time-based coordinate prediction function to obtain the third image queue;

[0080] S500: Input the images of construction workers in the detection boxes and prediction boxes in the third image queue into the feature extraction network to generate multiple feature maps of the construction workers. After being fused through the attention mechanism, the corresponding fused feature map of the construction workers is obtained. After compression, the shape feature vector of the construction workers is obtained.

[0081] S600: By calculating the geometric similarity and shape similarity between the predicted bounding box and the detection bounding box in the third image queue, a cost matrix is ​​constructed, and multiple optimal matching pairs are selected for target matching of multiple construction workers; among them, the shape similarity is calculated based on the shape feature vector.

[0082] S700: Based on the best matching pair obtained in the current frame, obtain the center position of the detection box corresponding to each construction worker in the third image queue, and obtain the movement trajectory of each construction worker by constructing the corresponding movement sequence through the center positions of each construction worker in consecutive frames.

[0083] It should be noted that the present invention preprocesses the on-site image data queue in steps S100-S700 to obtain an image data queue containing detection boxes; constructs a deep learning-based multi-target construction worker tracking model for tracking multiple construction workers, wherein a construction worker position prediction algorithm is set to predict the position of the construction worker in the next frame image; and constructs a construction worker shape feature extraction network to extract the shape features of the construction workers and obtain shape feature vectors; based on the above steps, target matching of construction workers is performed, and the trained multi-target construction worker tracking model is used to calculate and solve for the movement trajectory of multiple construction workers. The whole process can reduce the amount of computation in the feature extraction process and improve the representation ability of shape features.

[0084] Example 2, refer to Figures 1-4 As an embodiment of the present invention, based on the above embodiment, a multi-target construction personnel tracking method for power grid field operations is provided.

[0085] In this embodiment of the invention, step S100 involves acquiring video data of the power grid operation site;

[0086] Specifically, surveillance cameras can be installed at power grid operation sites to collect video data from the operation sites.

[0087] In this embodiment of the invention, step S200 involves constructing an original image queue by extracting frame images from video data;

[0088] Specifically, each frame of the video data can be used as an element of the image data queue to obtain an original image queue A1.

[0089] S300: Preprocess the original image queue to obtain a second image queue containing the detection boxes of construction workers;

[0090] Specifically, preprocessing can be performed on each frame of the original image queue A1 using a pre-trained object detection model to add a detection box for each construction worker in the image, resulting in a second image queue A2 containing the detection boxes of the construction workers.

[0091] Among them, the pre-trained object detection model is an existing object detection model, such as YOLO (You Only Look Once) object detection model, which is obtained by pre-training on existing datasets (such as COCO dataset).

[0092] Among them, the detection box is the bounding box output by the target detection model, which represents the position and range of the detected target in the image. The detection box mainly serves as the basic setting for subsequent target tracking.

[0093] In this embodiment of the invention, step S400 generates the next frame prediction box of all construction worker detection boxes in the second image queue using a time-based coordinate prediction function to obtain the third image queue, including the following steps A1-A4:

[0094] A1: For any frame in the second image queue A2, obtain the position of the construction worker detection box and get the coordinate information of each corner of the detection box;

[0095] Specifically, in step A1, a two-dimensional coordinate system can be established in the second image, with the lower left corner of the image as the origin, the width of the image as the x-axis, and the length of the image as the y-axis.

[0096] For example, for any frame image a j ∈A2, where j indicates that the image is the j-th frame image in the second image queue, and a is obtained. j The location information of the inspection frame by the construction workers is used to obtain the coordinates of each corner of the inspection frame.

[0097] A2: Design a time-based coordinate prediction function T. Based on the coordinates of each corner of the construction worker detection box in frames 0 to j, apply T to predict the position of the corner in frame j+1, and then obtain the construction worker prediction box in the image of frame j+1.

[0098] The time-based coordinate prediction function T is expressed as:

[0099]

[0100] Among them, c j c represents the coordinates of a corner of the detection box for a construction worker in the j-th frame of the image; i -c i-1 k represents the difference between two coordinates. base This represents the set baseline value; α is the weighting coefficient; |c j -c0| indicates c j The straight-line distance between the two coordinates c0 and c0; Indicates taking α|c j The integer less than -c0|; T(c) represents the weighting coefficient. j ) represents the predicted coordinates of a corner of a construction worker's detection frame in frame j+1; k represents the exponential parameter that controls the distribution of displacement weights in historical frames.

[0101] It should be noted that the aforementioned time-series-based coordinate prediction function is used to predict the position of construction workers in the next frame, which can solve the problem of high computational cost during prediction and reduce the computational time complexity.

[0102] A3: Calculate the predicted coordinates of the four corners of a detection box using a coordinate prediction function, and then connect them to obtain the prediction box for the next frame;

[0103] A4: Predict the next frame prediction box for each of the detection boxes in the second image queue, and obtain the third image queue A3.

[0104] For example, steps A1-A4 above can be represented as follows:

[0105] Taking the third frame of the surveillance video, a3∈A2, as an example: Obtain the coordinates of the top-left corner of a construction worker's detection frame (a rectangle): (50, 50). The coordinates of this corner in frames 0-2 are (17, 20), (32, 30), and (44, 40), respectively. Then, k... base Let α be 2, and α be 1 / 50. |(50,50)-(17,20)|≈44.6, so k=2.

[0106] The predicted coordinates of the aforementioned angle in the 4th frame are calculated using the coordinate prediction function T.

[0107] The calculation process is as follows:

[0108] At this time, j=3. When i=1, the calculated value is 1 / 14*(32-17,30-20)=(15 / 14,10 / 14);

[0109] When i = 2, the calculated value is 4 / 14*(44-32,40-30) = (48 / 14,40 / 14);

[0110] When i = 3, the calculated value is 9 / 14*(50-44,50-40) = (54 / 14,90 / 14);

[0111] The value is: (15 / 14, 10 / 14) + (48 / 14, 40 / 14) + (54 / 14, 90 / 14) = (117 / 14, 140 / 14);

[0112] The predicted coordinates of the aforementioned angle in the 4th frame are then calculated using the coordinate prediction function T:

[0113] (117 / 14, 140 / 14) + (50, 50) = (817 / 14, 60);

[0114] Similarly, by using the coordinate prediction function T to predict the coordinates of the remaining corners in the detection frame and connecting the points corresponding to each corner in the coordinate system, the predicted frame of the construction worker in the 4th frame can be obtained.

[0115] After performing the above operations on the second image queue A2, the third image queue A3 is obtained.

[0116] In an embodiment of the present invention, reference is made to Figure 2 In step S500, the images of construction workers within the detection boxes and prediction boxes in the third image queue are input into the feature extraction network to generate multiple feature maps of the construction workers, including the following steps B1-B2:

[0117] B1: Input the construction worker image within a frame in the third image queue into the feature extraction network. After passing through a two-dimensional convolutional layer, a first feature map G1 of the construction worker is obtained.

[0118] B2: Taking the first feature map G1 as input, after passing through the Bounding Box-Appearance Feature Extraction (BB-AFE) module, the nth feature map G of the construction worker is obtained. n .

[0119] In an embodiment of the present invention, in step B2, as follows: Figure 3 As shown, the feature extraction network includes a bounding box-shape feature extraction module (BB-AFE), and the extraction process specifically includes B2.1-B2.11:

[0120] B2.1: Take the first feature map G1 as input, pass it through a two-dimensional convolutional layer (Conv2D) to obtain the multi-scale feature map P1 of the construction workers;

[0121] Specifically, the kernel size of a two-dimensional convolutional layer (Conv2D) can be 3×3 with a stride of 2;

[0122] The calculation process of the multi-scale feature map P1 is as follows:

[0123] P1=PReLU(Conv2D(G1,kenel=3×3,stride=2))

[0124] Where PReLU represents the Parametric ReLU activation function, Conv2D represents two-dimensional convolution, kernel is the kernel size, and stride is the stride.

[0125] B2.2: After passing the multi-scale feature map P1 through the maximum pooling layer, the maximum pooling feature map P2 of the construction personnel is obtained;

[0126] B2.3: After passing the multi-scale feature map P1 through a hollow convolutional layer, we obtain the hollow convolutional feature map P5 of the construction workers;

[0127] Specifically, the kernel size of the dilated convolutional layer can be 3×3, the stride is 2, and the dilation factor is 2;

[0128] The calculation process for the dilated convolution feature map P5 is as follows:

[0129] P5=DConv(P1,kernel=3×3,stride=2,dilation=2)

[0130] Where DConv represents dilated convolution, kernel is the kernel size, stride is the stride, and dilation is the dilation factor.

[0131] B2.4: After passing the construction worker max pooling feature map P2 through a shuffle convolutional neural network (ShuffleNet network), the first construction worker shuffle convolutional feature map P3 is obtained;

[0132] Specifically, the ShuffleNet network structure in B2.4 consists of local convolution, channel shuffling, and 1×1 convolution.

[0133] B2.5: Pass the construction worker sparse convolutional feature map P5 through a shuffle convolutional neural network (ShuffleNet network) to obtain the second construction worker shuffle convolutional feature map P6;

[0134] Specifically, the ShuffleNet network in B2.5 has the structure of downsampling, local convolution, channel shuffling, and 1×1 convolution.

[0135] Specifically, the resolution of P3 and P6 is the same.

[0136] B2.6: After batch normalization (BN) of P3 and P6, the first normalized feature map P4 and the second normalized feature map p7 of the construction personnel are obtained respectively;

[0137] B2.7: Concat the first normalized feature map P4 and the second normalized feature map P7 to obtain the construction worker splicing feature map P8;

[0138] Specifically, the calculation process for the spliced ​​feature map P8 is as follows:

[0139]

[0140] Here, Concat represents the concatenation operation, and axis=-1 indicates that the feature maps will be concatenated from the channel dimension.

[0141] B2.8: After performing a transposed convolution (TConv) operation on the spliced ​​feature map p8 of the construction workers, we obtain the transposed convolution feature map p9 of the construction workers;

[0142] Specifically, the calculation process of the transposed convolution feature map p9 is as follows:

[0143] p9 = TConv(p8)

[0144] Where TConv represents the transpose convolution operation.

[0145] B2.9: Perform a residual connection (ResCon) between the transposed convolutional feature map p9 of the construction workers and the multi-scale feature map p1 to obtain the second feature map G2 of the construction workers;

[0146] Specifically, the calculation process for the second feature map G2 is as follows:

[0147] G2 = ReLU(P1 + P9)

[0148] Where ReLU represents the ReLU activation function, and + indicates that the element-wise values ​​are added together.

[0149] It should be noted that in step S500, Conv2D can perform deeper feature extraction on the feature map; MaxPool can effectively reduce the size of the feature map, reduce the computational complexity of subsequent layers, and retain important features; DConv can improve the efficiency of the network, while expanding the receptive field and capturing a wider range of contextual information; ShuffleNet improves the expressive power of the feature map through channel rearrangement and depthwise separable convolution; BN can reduce internal covariance shift, making training more stable and accelerating convergence; Concat fuses feature information at multiple scales, enhancing the expressive power of the feature map; TConv can restore spatial details and generate high-quality features; ResCon can retain the original feature information and enable the information at the input end to be better transmitted and utilized.

[0150] For example, steps B2.1-B2.9 above can be embodied as follows:

[0151] The image of a construction worker within a bounding box in a frame a∈A3 is taken as the input to the construction worker shape feature extraction network. Let the size of this construction worker image be 512×512×3 (where 512 represents the height and width of the image, and 3 represents the number of image channels). After passing this construction worker image through a convolutional layer with a kernel size of 3×3 and a stride of 1, a construction worker feature map G1 with a size of 512×512×256 is obtained. The specific number of output channels will be determined during training.

[0152] Using G1 as input to the BB-AFE module, a 2D convolutional layer Conv2D with a kernel size of 3×3 and a stride of 2 is passed through to obtain a 256×256×256 multi-scale feature map P1 of construction workers. P1 is then passed through a max-pooling layer MaxPool to obtain a 128×128×256 max-pooled feature map P2 of construction workers. Finally, P1 is passed through a DConv layer with a kernel size of 3×3, a stride of 2, and a dilation factor of 2 to obtain a 256×256×256 dilated convolutional feature map P5 of construction workers. Passing P2 through a ShuffleNet network yields a 128×128×128 construction worker shuffling convolutional feature map P3, where the ShuffleNet network structure includes local convolution, channel shuffling, and 1×1 convolution. Passing P5 through a ShuffleNet network yields a 128×128×128 construction worker shuffling convolutional feature map P6, where the ShuffleNet network structure includes downsampling, local convolution, channel shuffling, and 1×1 convolution. Applying Batch Normalization (BN) to P3 and P6 yields a 128×128×128 feature map P4 and a 128×128×128 normalized construction worker feature map P7, respectively. Concatenating P4 and P7 yields a 128×128×256 construction worker stitched feature map P8. Next, P8 is processed through a TConv loop to obtain a 256×256×256 transposed convolutional feature map of construction workers, P9. Finally, P9 and P1 are subjected to ResCon conversion to obtain a 256×256×256 feature map of construction worker outline extraction, G2.

[0153] B2.10: The second feature map G2 is used as input. After passing through the second bounding box-shape feature extraction module, the construction worker shape extraction feature map G3 is obtained.

[0154] B2.11: Repeat the extraction operation until the feature map G is extracted. n-1 After inputting into the (n-1)th bounding box-shape feature extraction module, the extracted feature map G of the construction worker's shape is obtained. n This forms the first feature map sequence S = {G1, G2, G3, ..., G...} n}; where n-1 is the number of structures.

[0155] For example, steps B2.10-B2.11 can be embodied as follows:

[0156] If n-1=4, then taking G2 as input, after passing through the second BB-AFE module, we obtain a construction worker appearance feature map G3 with a size of 128×128×256. Taking G3 as input, after passing through the third BB-AFE module, we obtain a construction worker appearance feature map G4 with a size of 64×64×256. Taking G4 as input, after passing through the fourth BB-AFE module, we obtain a construction worker appearance feature map G5 with a size of 32×32×256. G1, G2, G3, G4, and G5 form a feature map sequence S = {G1, G2, G3, G4, G5}.

[0157] It should be noted that the BB-AFE module in the above steps is used to extract multi-scale features of the construction worker's appearance. This module uses multiple deep convolutional layers, pooling layers, and other feature extraction techniques; among which:

[0158] Two-dimensional convolutional layers (Conv2D) are core components in convolutional neural networks (CNNs) used to process two-dimensional data (such as images). 2D convolutional layers feature parameter sharing and local connectivity, effectively reducing the number of parameters while preserving spatial information. Max pooling is a downsampling operation in CNNs used to reduce the spatial size of feature maps while retaining the most important features. It achieves dimensionality reduction by taking the maximum value within a local region of the input feature map, thereby reducing computational complexity and enhancing the model's robustness to translation and deformation. Dilated convolutional layers (DConv) are an improved convolutional operation that expands the receptive field by introducing "dilation" into the convolutional kernel without increasing the number of parameters or reducing the resolution of the feature map, enabling the capture of a wider range of contextual information. The ShuffleNet network significantly reduces computation and the number of parameters through grouped convolution and channel shuffling. Grouped convolution divides the input channels into multiple groups for separate processing, reducing computational complexity; channel shuffling rearranges the channels, promoting information interaction between different groups and avoiding information isolation. Batch normalization (BN) aims to accelerate the training of neural networks and improve model stability, reducing gradient vanishing or exploding, thus allowing for higher learning rates and faster convergence. Concat connects multiple tensors along a specified dimension, merging feature maps from different branches or layers along the channel dimension, thereby increasing the number of channels in the feature maps while preserving their spatial information. Transposed convolution (TConv) enlarges the spatial size of the input feature map, achieving upsampling. Residual connections (ResCon) aim to address the gradient vanishing and degradation problems in deep neural networks.

[0159] In embodiments of the present invention, such as Figure 4As shown, in step S500, the corresponding construction worker fusion feature map is obtained after fusion through the attention mechanism, including the following step B3 and its sub-steps:

[0160] B3: Input the feature map sequence S into the shape feature-hybrid attention mechanism (AC-HAM) module;

[0161] B3.1: For any feature map G in the feature map sequence S, input the arbitrary feature map G into the compression and activation submodule (SE Block) in the attention mechanism to obtain the first channel feature map G′1 after channel attention weighting;

[0162] By inputting any feature map G into the efficient channel attention network (ECA-Net) in the attention mechanism, we obtain the second channel feature map G′2 after efficient channel attention weighting.

[0163] By inputting any feature map G into the nonlocal attention submodule (NLA) in the attention mechanism, the spatial attention-weighted third channel feature map G′3 is obtained.

[0164] It should be noted that the SE Block module is mainly used to enhance the features of important channels and suppress the features of unimportant channels; ECA-Net is used to avoid the high computational cost of fully connected layers, capture the relationship between channels through local cross-channel interaction, reduce the number of parameters and computation, and maintain high performance; the NLA module generates global attention weights by calculating the similarity between each position in the feature map and other positions, thereby enhancing the features of key regions and suppressing unimportant regions, and can effectively capture global contextual information in the image.

[0165] B3.2: Perform weighted feature fusion on the three channel feature maps to obtain a feature-enhanced hybrid feature map of construction workers, G′;

[0166] Specifically, G′1, G′2, and G′3 are weighted and fused, and the calculation process of the mixed feature map G′ is as follows:

[0167] G′=(G′1⊙λ1)+(G′2⊙λ2)+(G′3·λ3)

[0168] Where λ1, λ2, and λ3 represent learnable weight coefficients, and λ1 + λ2 + λ3 = 1; ⊙ represents channel-wise multiplication; · represents element-wise multiplication; and + represents element-wise addition.

[0169] B3.3: All the mixed feature maps generated by the first feature map sequence S constitute the second feature map sequence S′;

[0170] Specifically, all of the above G′ constitute a feature map sequence S′.

[0171] B3.4: Select the feature map with the highest resolution in the second feature map sequence S′ as the upsampling target. After upsampling, a third feature map sequence S″ with the same resolution is obtained.

[0172] B3.5: Perform multi-scale feature fusion on all feature maps in the third feature map sequence S″ to obtain the fused feature map G of construction workers. n+1 ;

[0173] Specifically, the construction worker fusion feature map G n+1 The calculation process is as follows:

[0174] G n+1 =Conv2D(Concat(S″,axis=-1),kernel=1×1,stride=1)

[0175] Where Conv2D represents two-dimensional convolution; Concat(S″) represents concatenating all feature maps in S″; kernel represents the convolution kernel size; stride represents the stride; axis = -1 indicates that the feature maps will be concatenated along the channel dimension.

[0176] It should be noted that, as Figure 4 As shown, AC-HAM is a shape feature-hybrid attention mechanism module constructed in this invention, used to fuse feature maps of multiple different scales. The AC-HAM module uses a combination of channel attention and spatial attention mechanisms for weighting, and then fuses the feature maps to obtain a fused feature map of a construction worker. Among them, the compression and excitation submodule SE Block and the efficient channel attention network ECA-Net can reduce redundant information in the feature map and improve the representation ability of the features; the spatial attention mechanism NLA can capture long-distance spatial dependencies and also enhance the features of the target area through global information; the feature map after weighted feature fusion can improve feature diversity; the final multi-scale feature fusion operation can generate more discriminative features and improve feature expressiveness.

[0177] In this embodiment of the invention, step S500 further includes step B4 and its sub-steps:

[0178] B4: The compressed shape feature vector E3 is obtained from the construction worker.

[0179] B4.1: Integrate construction personnel into feature map G n+1 After being compressed by Global Average Pooling (GAP), the pooled feature vector E1 of the construction workers is obtained;

[0180] It should be noted that GAP does not introduce additional parameters, which can effectively reduce the amount of computation and prevent overfitting, while preserving global information between channels.

[0181] B4.2: Input the pooled feature vector E1 into the fully connected layer (FCL) to obtain the fully connected feature vector E2 of the construction workers;

[0182] It should be noted that FCL can compress high-dimensional feature vectors to low dimensions, reducing the dimensionality of feature vectors while retaining key information and enhancing feature representation capabilities.

[0183] B4.3: After inputting the fully connected feature vector E2 into the L2 normalization layer (L2N), the shape feature vector E3 of the construction worker is obtained;

[0184] It should be noted that L2 normalization can eliminate the scaling effect of vectors, making them more suitable for subsequent similarity calculations.

[0185] Specifically, the calculation process for the shape feature vector E3 is as follows:

[0186]

[0187] Where L2N represents the L2 normalized layer; ||E2||2 represents the L2 norm of E2;

[0188] It should be noted that L2N can accelerate the training process and enhance the robustness of the model.

[0189] It should also be noted that, such as Figure 2 As shown, the construction worker shape feature extraction network in B1-B4 above is mainly used to extract the shape features of construction workers within the bounding box (detection box + prediction box) and compress them into a single shape feature vector. The network takes the image of the construction worker within the bounding box as input, passes through multiple deep convolutional layers and other feature extraction techniques to obtain multiple feature maps, then fuses these feature maps using an attention mechanism, and finally compresses them to obtain a single shape feature vector. The overall structure design addresses the problem of obtaining a shape feature vector that simultaneously achieves high quality and reduced background noise in existing lightweight networks. It uses multiple feature extraction modules to enhance the representational ability of the features and finally converts the obtained features into a fixed-length shape feature vector.

[0190] For example, step B4 above can be specifically embodied as follows:

[0191] Input S into an AC-HAM module;

[0192] In the AC-HAM module, the changes in S are as follows:

[0193] For any G∈S, after weighting through the attention mechanism of SE Block, ECA-Net, and NLA, and then performing weighted feature fusion, a mixed feature map G′ of construction workers with the same size is generated. All G′ are combined to form a feature map sequence S′. The feature map with the highest resolution of 512×512×256 in S′ is selected as G1″ as the upsampling target. After upsampling all feature maps in S′, a feature map sequence S″ with all feature maps having a resolution of 512×512×256 is obtained. Then, multi-scale feature fusion is performed on all feature maps in S″ to obtain a fused feature map G of construction workers with a size of 512×512×256. n+1 ;

[0194] G n+1 As input, after compression by a global average pooling (GAP), a pooled feature vector E1 of construction workers with 256 dimensions is obtained;

[0195] Taking E1 as input, after passing through a fully connected layer (FCL), assuming that the output dimension of the fully connected layer is set to 128 after training, a fully connected feature vector E2 of construction workers with 128 dimensions is obtained.

[0196] Taking E2 as input, after passing through an L2 normalization layer (L2N), we obtain a 128-dimensional feature vector E3 of the construction worker's appearance, where the element values ​​of each dimension are normalized.

[0197] In this embodiment of the invention, step S600 involves calculating the geometric and shape similarities between the predicted bounding boxes and the detection boxes in the third image queue, constructing a cost matrix, and selecting multiple optimal matching pairs, including the following steps C1-C4:

[0198] It should be noted that geometric similarity refers to the similarity in size and position of two bounding boxes in an image, while shape similarity emphasizes the similarity in the shape feature vectors of two bounding boxes.

[0199] C1: For any frame a in the third image queue A3 q If q = 0, then a q For the 0th frame image, at this time, a q All construction workers within the detection boxes are marked as tracking personnel;

[0200] If q≠0, based on the prediction box of a certain tracker and a q For a certain detection box on the screen, the intersection-union ratio (IoU) of the two boxes and the distance to the center point are calculated, thus obtaining the geometric similarity between a certain tracking person and the construction personnel within the detection box;

[0201] Specifically, the distance between the center points of two bounding boxes is calculated using Euclidean distance. Euclidean distance measures the geometric distance between two points as a straight line, representing the shortest path length between them. In two-dimensional space, it is calculated as the square root of the sum of the squares of the coordinate differences between the two points, with the following formula:

[0202]

[0203] Specifically, the above IoU is calculated as follows:

[0204]

[0205] Where t represents the predicted bounding box of the t-th tracker; i represents the detected bounding box of the ith tracker; S cross (t,i) represents the area of ​​the intersection region of the two bounding boxes; S detect (i) represents the area of ​​the detection box.

[0206] C2: Calculate the shape feature vectors using cosine distance to obtain the shape similarity of the construction workers within the two boxes;

[0207] Specifically, the formula for calculating the cosine distance is as follows:

[0208]

[0209] Where t represents the predicted bounding box of the t-th tracker; i represents the detected bounding box of the ith tracker; f t f represents the shape feature vector of the construction worker in the prediction box of the t-th tracker; t ||f represents the shape feature vector of the construction worker in the i-th detection frame; t ||and||f i || represents the Euclidean norm of these two shape feature vectors.

[0210] C3: Combining geometric similarity and shape similarity, construct a two-dimensional cost matrix C, where the number of rows in the matrix represents the number of trackers and the number of columns represents the number of detection boxes;

[0211] Specifically, the formula for constructing the two-dimensional cost matrix C is as follows:

[0212] C t,i =λ4·IoU(t,i)+λ5·d o (t,i)+(1-λ4-λ5)·d f (t,i)

[0213] Among them, C t,i Let represent any element in C, referring to the cost between the predicted bounding box of the t-th tracker and the detection box of the i-th tracker; λ4 and λ5 are learnable weight coefficients; d o(t,i) represents the distance between the center point of the predicted bounding box of the t-th tracker and the center point of the ith detection box; d f (t,i) represents the cosine distance between the predicted bounding box of the t-th tracker and the detected bounding box of the i-th tracker.

[0214] C4: Based on the cost matrix C, cost pairs are selected using the Jonker-Volgenant linear allocation optimization algorithm to obtain multiple optimal matching pairs (t,i).

[0215] Specifically, the optimal matching pair (t,i) refers to the construction worker in the i-th detection box matching the t-th tracking worker under the current algorithm.

[0216] It should be noted that the Jonker-Volgenant algorithm is an optimization algorithm for efficiently solving the linear assignment problem. Its core objective is to find the minimum weight perfect match of the cost matrix in a bipartite graph matching problem. This algorithm significantly improves computational efficiency by combining the primal dual method and the shortest augmenting path strategy, especially when dealing with large-scale or sparse matrices. Through dynamic labeling, reducing redundant computation, and optimizing the augmenting path search mechanism, it reduces the time complexity to a more favorable constant factor level, making it suitable for real-time or high-dimensional scenarios.

[0217] For example, steps C1-C4 above can be embodied as follows:

[0218] Using the third frame image a3∈A3 as an example: if there are 15 detection boxes in a3 and 3 trackers (determined in the 0th frame image a0∈A3), then the 3 predicted boxes corresponding to the trackers can be obtained. By calculating the IoU and center distance between the predicted box and each detection box for each tracker, 45 different IoUs and center distances can be obtained.

[0219] By extracting the shape feature vectors from the detection boxes and prediction boxes respectively, 18 shape feature vectors can be obtained. The cosine distance between the shape feature vector corresponding to each prediction box and the shape feature vector corresponding to each detection box is calculated, resulting in 45 different cosine distances.

[0220] Based on the formula for calculating the cost matrix, substituting the obtained IoU, center distance, and cosine distance, we obtain a cost matrix. The Jonker-Volgenant algorithm is then used to calculate it. If the multiple optimal matching pairs obtained are (1,6), (2,2), and (3,13), it means that the first tracker matches the 6th detection box, the second tracker matches the 2nd detection box, and the third tracker matches the 13th detection box.

[0221] S700: Based on the best matching pair obtained in the current frame, obtain the center position of the detection box corresponding to each construction worker in the third image queue, and obtain the movement trajectory of each construction worker by constructing the corresponding movement sequence through the center positions of each construction worker in consecutive frames.

[0222] For example, in step S700, for any frame image a q For instance A3, based on the best matching pair obtained in the current frame, the center position of the detection box corresponding to each construction worker is obtained. Then, in consecutive frames, the center positions of the detection boxes of multiple construction workers are combined into a sequence to obtain the movement trajectory of the construction workers; finally, the above operation is repeated to obtain the movement trajectory of multiple construction workers.

[0223] It should also be noted that the functions and models involved in steps S400-S600 above need to be trained first. For example, before model training begins, all model parameters are initialized, and relevant hyperparameters are set, including the number of training epochs, batch size, optimizer selection (such as Adam or SGD), and learning rate. For the network extracting the physical features of construction workers, the loss function (such as Triplet Loss or Cross-Entropy Loss) and the dimension of the feature vectors also need to be set.

[0224] Video data from multiple power grid operation sites was collected, and multiple image data queues were constructed according to S100. These image data queues were then divided into training, validation, and test sets in a 6:2:2 ratio. The training and validation sets were then divided into multiple batches, with each batch of training data fed into the multi-target construction worker tracking model for training. For the construction worker appearance feature extraction network, a feature matching loss (e.g., Triplet Loss) was calculated. Model parameters were updated through backpropagation to progressively optimize model performance.

[0225] After all batches of the training set have completed one round of training, the validation set data is input into the multi-objective construction worker tracking model in batches to obtain the corresponding validation set loss (batch_loss). The validation set loss is used to monitor whether the model is overfitting and to adjust the training strategy as needed, such as terminating training early or dynamically adjusting the learning rate. Training ends when the model training reaches the point where the loss value tends to converge.

[0226] The pre-processed image data queue is used as input, and a trained multi-target construction worker tracking model is used to calculate and obtain the movement trajectories of multiple construction workers.

[0227] In summary, this invention constructs a deep learning-based multi-target construction worker tracking model. This model can predict the position of each construction worker in the next frame and generate a bounding box. Then, it extracts the shape features of the construction workers within the two bounding boxes to form a shape feature vector. The predicted bounding box and the shape feature vector are then applied to target matching to obtain the matching relationship between the construction worker and the bounding box (i.e., detection box + predicted box). Finally, based on the position of the bounding box corresponding to the construction worker, the movement trajectory of each construction worker in consecutive frames is obtained, reducing computational time complexity and obtaining a more discriminative feature map, thus achieving accurate positioning and tracking of multiple construction workers at power grid operation sites.

[0228] Example 3 illustrates a schematic scheme for a multi-target construction worker tracking method for power grid field operations. It should be noted that the technical solution of this multi-target construction worker tracking system for power grid field operations is based on the same concept as the aforementioned multi-target construction worker tracking method for power grid field operations. Details not described in detail in this embodiment can be found in the description of the aforementioned multi-target construction worker tracking method for power grid field operations.

[0229] This embodiment also provides a multi-target construction worker tracking system for power grid field operations, including:

[0230] The acquisition module is used to acquire video data from power grid operation sites.

[0231] The first extraction module is used to construct an original image queue by extracting frame images from video data;

[0232] The preprocessing module is used to preprocess the original image queue to obtain a second image queue containing the detection boxes of construction workers;

[0233] The generation module is used to generate the next frame prediction box of all construction worker detection boxes in the second image queue through a time-based coordinate prediction function, so as to obtain the third image queue.

[0234] The second extraction module is used to input the images of construction workers within the frame in the third image queue and the second image queue into the feature extraction network respectively, generate multiple feature maps of the construction workers, and obtain the corresponding fused feature maps of construction workers in each queue through the attention mechanism. After compression, the shape feature vectors of construction workers in each queue are obtained.

[0235] The matching module constructs a cost matrix by calculating the geometric and shape similarity between the predicted bounding boxes and the detection boxes in the third image queue, and selects multiple optimal matching pairs for target matching of multiple construction workers; among them, the shape similarity is calculated based on the shape feature vector.

[0236] The trajectory generation module is used to obtain the center position of the detection box corresponding to each construction worker in the third image queue based on the best matching pair obtained in the current frame, and to obtain the movement trajectory of each construction worker by constructing the corresponding movement sequence through the center positions of each construction worker in consecutive frames.

[0237] This embodiment also provides a computer device suitable for tracking multiple construction personnel in power grid field operations, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for tracking multiple construction personnel in power grid field operations as proposed in the above embodiment.

[0238] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the multi-target construction personnel tracking method for power grid field operations proposed in the above embodiments.

[0239] The storage medium proposed in this embodiment and the method for tracking multi-target construction personnel for power grid field operations proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0240] Based on the above description of the implementation methods, those skilled in the art will clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0241] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for tracking multi-target construction workers in power grid field operations, characterized in that, include: Acquire video data from power grid operation sites; A raw image queue is constructed by extracting frame images from video data; The original image queue is preprocessed to obtain a second image queue containing the detection boxes of construction workers; The next frame prediction box for all construction worker detection boxes in the second image queue is generated by a time-based coordinate prediction function, thus obtaining the third image queue. The images of construction workers in the detection boxes and prediction boxes in the third image queue are input into the feature extraction network to generate multiple feature maps of the construction workers. After being fused through the attention mechanism, the corresponding fused feature map of the construction workers is obtained. After compression, the shape feature vector of the construction workers is obtained. By calculating the geometric and shape similarity between the predicted and detected bounding boxes in the third image queue, a cost matrix is ​​constructed, and multiple optimal matching pairs are selected for target matching of multiple construction workers; among them, the shape similarity is calculated based on the shape feature vector. Based on the best matching pair obtained in the current frame, the center position of the detection box corresponding to each construction worker in the third image queue is obtained, and the corresponding movement sequence is constructed by the center positions of each construction worker in consecutive frames to obtain the movement trajectory of each construction worker.

2. The multi-target construction worker tracking method for power grid field operations as described in claim 1, characterized in that, The next frame prediction boxes for all construction worker detection boxes in the second image queue are generated using a time-based coordinate prediction function, resulting in a third image queue, which includes: For any frame in the second image queue, obtain the position of the detection box for the construction worker and get the coordinate information of each corner of the detection box; The time-based coordinate prediction function is expressed as: Among them, c j c represents the coordinates of a corner of the detection box for a construction worker in the j-th frame of the image; i -c i-1 k represents the difference between two coordinates. base This represents the set baseline value; α is the weighting coefficient; |c j -c0| indicates c j The straight-line distance between the two coordinates c0 and c0; Indicates taking α|c j The integer less than -c0|; Represents the weighting coefficient, t(c j ) represents the predicted coordinates of a corner of a construction worker's detection frame in frame j+1; k represents the exponential parameter that controls the distribution of displacement weights in historical frames; The predicted coordinates of the four corners of a detection box are calculated using the coordinate prediction function and then connected to obtain the predicted box of the next frame. The prediction boxes for the next frame are generated by predicting all detection boxes in the second image queue, resulting in the third image queue.

3. The multi-target construction worker tracking method for power grid field operations as described in claim 2, characterized in that, The images of construction workers within the detection and prediction bounding boxes in the third image queue are input into the feature extraction network to generate multiple feature maps of the construction workers, including: The construction worker image within a frame in the third image queue is input into the feature extraction network. After passing through a two-dimensional convolutional layer, a first feature map G1 of the construction worker is obtained. Using the first feature map G1 as input, after passing through the bounding box-shape feature extraction module, the nth feature map G of the construction worker is obtained. n .

4. The multi-target construction worker tracking method for power grid field operations as described in claim 3, characterized in that, The feature extraction network includes a bounding box-shape feature extraction module, and the extraction process specifically includes: The first feature map G1 is used as input and passed through a two-dimensional convolutional layer to obtain the multi-scale feature map P1 of the construction workers. After passing the multi-scale feature map P1 through the maximum pooling layer, the maximum pooling feature map P2 of the construction personnel is obtained. After passing the multi-scale feature map P1 through a hollow convolutional layer, we obtain the hollow convolutional feature map P5 of the construction workers. After passing the maximum pooling feature map P2 of the construction workers through a shuffling convolutional neural network, the first shuffling convolutional feature map P3 of the construction workers is obtained. After passing the construction worker vacuous convolutional feature map P5 through a shuffling convolutional neural network, we obtain the second construction worker shuffling convolutional feature map P6. After batch normalization of P3 and P6, the first normalized feature map P4 and the second normalized feature map P7 of the construction personnel are obtained respectively. By splicing the first normalized feature map P4 and the second normalized feature map P7, the spliced ​​feature map P8 of the construction workers is obtained. After performing a transpose convolution operation on the spliced ​​feature map P8 of the construction workers, we obtain the transpose convolution feature map P9 of the construction workers. The construction worker transposed convolutional feature map P9 is residually connected with the multi-scale feature map P1 to obtain the construction worker second feature map G2. The second feature map G2 is used as input, and after passing through the second bounding box-shape feature extraction module, the construction worker shape extraction feature map G3 is obtained. Repeat the extraction operation until the feature map G is obtained. n-1 After inputting into the (n-1)th bounding box-shape feature extraction module, the extracted feature map G of the construction worker's shape is obtained. n The first feature map sequence S = {G1, G2, G3, ..., G...} is formed. n }; where n-1 is the number of the structures.

5. A multi-target construction worker tracking method for power grid field operations as described in claim 4, characterized in that, The fusion feature map of the corresponding construction workers obtained after fusion through the attention mechanism includes: The feature map sequence S is input into the shape feature-hybrid attention mechanism module; For any feature map G in the feature map sequence S, inputting the arbitrary feature map G into the compression and activation submodule of the attention mechanism yields the first channel feature map G1 after channel attention weighting. ′ ; Inputting any feature map G into the efficient channel attention network in the attention mechanism yields the second channel feature map G2 after efficient channel attention weighting. ′ ; Inputting any feature map G into the nonlocal attention submodule of the attention mechanism yields the spatially attention-weighted third-channel feature map G3. ′ ; The three channel feature maps are weighted and fused to obtain a feature map G′ of construction workers enhanced by feature fusion. All the mixed feature maps generated by the first feature map sequence S constitute the second feature map sequence S′; The feature map with the highest resolution in the second feature map sequence S′ is selected as the upsampling target. After upsampling, a third feature map sequence S″ with the same resolution is obtained. Multi-scale feature fusion is performed on all feature maps in the third feature map sequence S″ to obtain the fused feature map G of construction workers. n+1 .

6. The multi-target construction worker tracking method for power grid field operations as described in claim 5, characterized in that, The compressed external feature vector of the construction worker includes: Integrate the feature map of construction workers G n+1 After being input into global average pooling compression, the pooled feature vector E1 of the construction workers is obtained; The pooled feature vector E1 is input into the fully connected layer to obtain the fully connected feature vector E2 of the construction workers; After inputting the fully connected feature vector E2 into the L2 normalization layer, the shape feature vector E3 of the construction worker is obtained.

7. The multi-target construction worker tracking method for power grid field operations as described in claim 6, characterized in that, By calculating the geometric and shape similarities between the predicted and detected bounding boxes in the third image queue, a cost matrix is ​​constructed, and several optimal matching pairs are selected, including: For any frame a in the third image queue q If q = 0, then a q For the 0th frame image, at this time, a q All construction workers within the detection boxes are marked as tracking personnel; If q≠0, based on a tracker's prediction box and a q For a certain detection box on the screen, the intersection-union ratio (IoU) of the two boxes and the distance to the center point are calculated, thus obtaining the geometric similarity between a certain tracking person and the construction personnel within the detection box; The similarity of the shapes of construction workers within two boxes is obtained by calculating the shape feature vectors using cosine distance. Combining geometric similarity and shape similarity, a two-dimensional cost matrix C is constructed, where the number of rows in the matrix represents the number of trackers and the number of columns represents the number of detection boxes; Based on the cost matrix C, cost pairs are selected using the Jonker-Volgenant linear allocation optimization algorithm to obtain multiple optimal matching pairs (t,i).

8. A multi-target construction worker tracking system for power grid field operations, using the method described in any one of claims 1-7, characterized in that, include: The acquisition module is used to acquire video data from power grid operation sites. The first extraction module is used to construct an original image queue by extracting frame images from video data; The preprocessing module is used to preprocess the original image queue to obtain a second image queue containing the detection boxes of construction workers; The generation module is used to generate the next frame prediction box of all construction worker detection boxes in the second image queue through a time-based coordinate prediction function, so as to obtain the third image queue. The second extraction module is used to input the images of construction workers within the frame in the third image queue and the second image queue into the feature extraction network respectively, generate multiple feature maps of the construction workers, and obtain the corresponding fused feature maps of construction workers in each queue through the attention mechanism. After compression, the shape feature vectors of construction workers in each queue are obtained. The matching module constructs a cost matrix by calculating the geometric and shape similarity between the predicted bounding boxes and the detection boxes in the third image queue, and selects multiple optimal matching pairs for target matching of multiple construction workers; among them, the shape similarity is calculated based on the shape feature vector. The trajectory generation module is used to obtain the center position of the detection box corresponding to each construction worker in the third image queue based on the best matching pair obtained in the current frame, and to obtain the movement trajectory of each construction worker by constructing the corresponding movement sequence through the center positions of each construction worker in consecutive frames.

9. A computer device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the multi-target construction personnel tracking method for power grid field operations as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the multi-target construction worker tracking method for power grid field operations as described in any one of claims 1 to 7.

Images