Equipment operation information high-precision identification method for complex scene test video

Abstract

The invention relates to the technical field of engineering equipment, and particularly discloses an equipment operation information high-precision identification method for a complex scene test video. The objective of the invention is to solve the problems of missing detection, false detection and inaccurate positioning of action information caused by multi-scale change, action shielding, strong noise interference and difficult time-space dynamic collaborative modeling in a complex equipment operation scene in a traditional target detection technology. Comprising the steps of video data preprocessing, multi-branch residual 3D convolution dynamic action feature extraction, path aggregation network 2D feature enhancement and decoupling, spatial-temporal feature fusion based on feature interactive modeling, and action information identification based on reconstruction of an anchor-free Center Net network. According to the method, the 2D pyramid spatial features, the 3D multi-branch residual spatial-temporal features and the channel coding attention mechanism are fused, and a path aggregation network and an improved anchor-free Center Net structure are combined, so that multi-scale dynamic modeling and high-precision anchor-free automatic detection of equipment operation action information in a complex scene test video are realized.

Description

A high-precision identification method for equipment operation information in test videos of complex scenarios Technical Field

[0001] This invention relates to the field of engineering equipment technology, and in particular to a high-precision identification method for equipment operation information in test videos of complex scenarios. Background Technology

[0002] During operational testing of engineering equipment, diverse operating environments and complex construction conditions lead to low quality video data in terms of resolution, frame rate, color saturation, and contrast. This is compounded by factors such as motion blur, camera shake, insufficient or overexposed lighting, adverse weather conditions (rain, snow, fog, haze), occlusion by mechanical components and operators, various non-target interference elements in the background, high-frequency noise, and abnormal pixel jumps. Traditional single-stage target detection methods based on pre-defined anchor frames (such as FasterYOLO) struggle to balance detection accuracy and real-time inference when faced with complex scenarios involving large and small targets, densely overlapping targets, rapidly changing dynamic environments, and inter-frame temporal contextual information. They also fail to accurately distinguish and locate key parts of small or partially occluded equipment movements. Furthermore, the lack of a sufficient fusion mechanism for multi-scale spatial information and temporal features results in significant issues of missed detections, false detections, and positioning errors. Therefore, there is an urgent need to propose a high-precision identification method for equipment operational information in complex scenario test videos. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a high-precision identification method for equipment operation information in test videos of complex scenarios. By integrating 2D pyramid spatial features, 3D multi-branch residual spatiotemporal features and channel coding attention mechanism, and combining path aggregation network and improved anchor-free CenterNet structure, multi-scale dynamic modeling and anchor-free automatic detection of equipment operation action information in test videos of complex scenarios can be achieved.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A high-precision identification method for equipment operation information in test videos of complex scenarios is proposed. This method acquires equipment operation action information from test video data under complex scene environments based on a dynamic spatiotemporal motion detection network model. Specifically, it includes:

[0006] Step 1, video data preprocessing: extract keyframes from the raw video data, feed them into the extended layer aggregation network to extract multi-level 2D features, and combine the multi-level 2D features with structural and semantic features through a feature pyramid to form 2D pyramid features.

[0007] Step 2, Multi-branch residual 3D convolutional dynamic action feature extraction: Input keyframes into the multi-branch 3D residual network to extract deep 3D features;

[0008] Step 3, 2D feature enhancement and decoupling of path aggregation network: Input 2D pyramid features into path aggregation network to further enhance the pyramid's representation ability, and use feature decoupling to decouple the position information and category information of the target in the key frame;

[0009] Step 4, Spatiotemporal feature fusion based on interactive feature modeling: Based on interactive modeling technology, in the decoupled fusion head, the positional and category features in the 3D spatiotemporal features are fused with the decoupled 2D spatial features, and the output features are obtained through the position predictor and the category predictor.

[0010] Step 5, Action Information Recognition Based on the Modified Anchor-Free CenterNet Network: In the output layer, based on the predicted value of the target position, the anchor box is predicted using the improved anchor-free CenterNet network, and the final target action information recognition result is generated using multi-level anchor-free and improved non-maximum suppression algorithms.

[0011] As a further aspect of the present invention, in step one, the extended layer aggregation network includes two shallow convolutional blocks. The shallow convolutional blocks are connected to a first extended layer aggregation module. The first extended layer aggregation module is connected to a first max pooling module. The first max pooling module is connected to a second extended layer aggregation module. The second extended layer aggregation module is connected to a second max pooling module. The second max pooling module is connected to a third extended layer aggregation module. The third extended layer aggregation module is connected to a third max pooling module. The third max pooling module is connected to a fourth extended layer aggregation module. The first extended layer aggregation module is connected to a first convolutional layer. The first convolutional layer is connected to a batch normalization layer.

[0012] As a further embodiment of the present invention, in step one, the feature pyramid network includes a second convolutional layer connected to the second extended aggregation module, a third convolutional layer connected to the third extended aggregation module, a pyramid pooling module connected to the fourth extended aggregation module, and a first splicing module connected to the first batch of normalization layers. The second convolutional layer is sequentially connected to the second batch of normalization layers, the second splicing module, the fifth extended aggregation module, and the fourth max pooling module. The third convolutional layer is sequentially connected to the third batch of normalization layers, the third splicing module, the sixth extended aggregation module, and the first upsampling convolutional module. The first upsampling convolutional module is connected to the second splicing module. The first splicing module is sequentially connected to the fourth convolutional layer, the fourth batch of normalization layers, and the second upsampling convolutional module. The pyramid pooling module is connected to the first splicing module, and the second upsampling convolutional module is connected to the third splicing module.

[0013] As a further aspect of the present invention, in step two, the multi-branch 3D residual network includes a fourth convolutional layer, the fourth convolutional layer is connected to a fifth max pooling layer, the fifth max pooling layer is connected to three first multi-branch residual blocks, each multi-branch residual block is connected to a first multi-branch downsampling residual block, the first multi-branch downsampling residual block is connected to three second multi-branch residual blocks, each second multi-branch residual block is connected to a second multi-branch downsampling residual block, the second multi-branch downsampling residual block is connected to 22 third multi-branch residual blocks, each third multi-branch residual block is connected to a third multi-branch downsampling residual block, and the third multi-branch downsampling residual block is connected to two fourth multi-branch residual blocks.

[0014] As a further embodiment of the present invention, in step three, the path aggregation network includes a first feature decoupling module connected to the fifth aggregation module, a fourth splicing module connected to both the sixth extended aggregation module and the fourth max pooling module, a fifth splicing module connected to the fourth batch normalization layer, the fourth splicing module connected to the seventh extended aggregation module, the seventh extended aggregation module connected to the fifth max pooling module and the second feature decoupling module, the fifth max pooling module connected to the fifth splicing module, the fifth splicing module also connected to the eighth extended aggregation module, and the eighth extended aggregation module connected to the third feature decoupling module.

[0015] As a further aspect of the present invention, in step four, the decoupling fusion head includes a first decoupling fusion head, a second decoupling fusion head, and a third decoupling fusion head.

[0016] As a further embodiment of the present invention, the first decoupling fusion head includes a multi-branch 3D convolutional dynamic motion feature receiving block, a first channel decoding module, and a second channel decoding module. The multi-branch 3D convolutional dynamic motion features of equipment operation actions and the equipment operation action category features are both input into two fifth convolutional layers through the first channel decoding module for convolutional feature extraction. The extracted features are then passed through a sixth convolutional layer to obtain the position prediction value of the equipment operation action. The multi-branch 3D convolutional dynamic motion features of equipment operation actions and the equipment operation action position features are input into two seventh convolutional layers through the second channel decoding module for convolutional feature extraction. The extracted features are then passed through an eighth convolutional layer to obtain the equipment operation category prediction value.

[0017] As a further embodiment of the present invention, in step four, the second decoupling fusion head is used to receive the equipment operation action category features and position features extracted by the second feature decoupling module, and at the same time receive the equipment operation action multi-branch 3D convolution dynamic action features; the third decoupling fusion head is used to receive the equipment operation action category features and position features obtained by the third feature decoupling module.

[0018] As a further aspect of the present invention, in step five, the improved anchor-free CenterNet network includes a center point heatmap generation module, a center point offset prediction branch, a target width and height prediction branch, and an anchor-free bounding box generation module.

[0019] As a further aspect of the present invention, step five specifically includes:

[0020] Step 51, Generate center point heatmap: The center point heatmap generation module applies a Gaussian distribution to the corresponding pointer on the feature map for each category and normalizes the pixel values ​​to generate an independent center point heatmap. The heatmap uses a Gaussian kernel function to allocate pixel values ​​to the target center.

[0021] Step 52, Center point offset prediction: The center point offset prediction branch predicts the offset relative to the feature map grid for each candidate center point through a convolutional layer.

[0022] Step 53, Target width and height prediction: The target width and height prediction branch predicts the target width and height for each center point and outputs width and height values ​​that are adapted to the feature map resolution;

[0023] Step 54, Generate Anchorless Boundary: The anchorless boundary generation module uses a mapping function to restore the coordinates of the upper left and lower right corners of the target rectangle on the original image at the original image scale, based on the center point coordinates, predicted offset, and width and height values.

[0024] The technical effects of this invention's high-precision recognition method for equipment operation information in complex scenario test videos are as follows: This invention achieves accurate acquisition of equipment operation action information from complex scenario test video data through a dynamic spatiotemporal motion detection network model. It extracts one frame from the original video sequence as a keyframe, effectively extracts static feature information of the equipment action information from the test video through 2D convolution, and deepens the model's understanding of motion through a multi-branch residual 3D convolutional network, addressing the impact of video jitter on detection from the network's knowledge capability level. It extracts keyframe graphic structure features of different granularities sequentially from the bottom up through a feature pyramid, and performs top-down convolutional upsampling within the pyramid. The backtracking feature transmits deep semantic features back to the bottom of the pyramid, anchoring concrete structural features with abstract semantic features. 2D convolutional features are decoupled from the features of the path aggregation network and converge with 3D convolutional features at the decoupling aggregation head to form output features. Combined with temporal and spatial features, dynamic and static information are effectively fused to help accurately capture rapid action changes and maintain a complete understanding of the target when the target is partially occluded or the action is slow, achieving comprehensive recognition of video actions. Anchor point regression and action classification are performed through the output features. After multi-stage regression and an improved non-maximum suppression algorithm, the final equipment type, action information, and time information are obtained. Attached Figure Description

[0025] Figure 1 is a technical roadmap for complex scene action information recognition and automatic acquisition according to the present invention;

[0026] Figure 2 is a diagram illustrating the core ideas for constructing the dynamic spatiotemporal action detection network model in complex scenarios according to the present invention.

[0027] Figure 3 is a schematic diagram of the dynamic spatiotemporal motion detection network model structure in complex scenarios according to the present invention;

[0028] Figure 4 is a schematic diagram of the structure of the feature pyramid of the present invention;

[0029] Figure 5 is a schematic diagram of the structure of the multi-branch residual block and the multi-branch downsampling residual block in the multi-branch 3D residual network of the present invention;

[0030] Figure 6 is a schematic diagram of the path aggregation network of the present invention;

[0031] Figure 7 is an analytical diagram of the internal component modules of the pyramid pooling module, max pooling module, extended layer aggregation module, feature decoupling module and upsampling convolution module in the path aggregation network of the present invention.

[0032] Figure 8 is a schematic diagram of the convolutional upsampling principle of the present invention;

[0033] Figure 9 is a schematic diagram of the channel number encoding structure of the decoupling fusion head of the present invention;

[0034] Figure 10 is a function graph of the Robust LeakyReLU activation function of the present invention. Detailed Implementation

[0035] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described content is only a part of this invention, not all of it. All other technical solutions obtained by those skilled in the art based on the content of this invention without inventive effort are within the scope of protection of this invention.

[0036] As shown in Figure 1, in the process of constructing datasets for complex scenarios, the acquisition of operational action information for engineering equipment faces the problem of target occlusion and truncation. Feature pyramids can capture feature information at different levels across multiple scales, enabling the model to still identify key features even when targets are partially occluded or truncated. Furthermore, the interaction between features at different scales allows for information sharing and dynamic collaboration between feature layers, enabling each layer to complement the others. This enhances the model's robustness to occlusion or truncation and improves its ability to identify incomplete targets. For small target detection, a multi-scale feature fusion method is used to optimize and combine feature levels at different scales, improving the model's sensitivity to small target detection. Furthermore, by optimizing the anchorless detection channel, the priority of small targets in the model processing can be ensured, thereby effectively reducing the problems of missed detection and false judgment of targets. Therefore, this invention integrates residual 3D features and 2D pyramid features, and through multi-dimensional feature fusion, attention mechanism and spatial regression, to achieve accurate recognition and localization of action categories and bounding boxes, enhancing the accuracy and reliability of complex scene operation action information recognition and bounding box localization in test data under complex scenes. Based on the above ideas, this invention proposes a high-precision recognition method for equipment operation information in test videos under complex scenes. It obtains equipment operation action information from test video data under complex scene environments based on a dynamic spatiotemporal action detection network model, specifically including:

[0037] Step 1, video data preprocessing: extract keyframes from the raw video data, feed them into the extended layer aggregation network to extract multi-level 2D features, and combine the multi-level 2D features with structural and semantic features through a feature pyramid to form 2D pyramid features.

[0038] Step 2, Multi-branch residual 3D convolutional dynamic action feature extraction: Input keyframes into the multi-branch 3D residual network to extract deep 3D features;

[0039] Step 3, 2D feature enhancement and decoupling of path aggregation network: Input 2D pyramid features into path aggregation network to further enhance the pyramid's representation ability, and use feature decoupling to decouple the position information and category information of the target in the key frame;

[0040] Step 4, Spatiotemporal feature fusion based on interactive feature modeling: Based on interactive modeling technology, in the decoupled fusion head, the positional and category features in the 3D spatiotemporal features are fused with the decoupled 2D spatial features, and the output features are obtained through the position predictor and the category predictor.

[0041] Step 5, Action Information Recognition Based on the Modified Anchor-Free CenterNet Network: In the output layer, based on the predicted value of the target position, the anchor box is predicted using the improved anchor-free CenterNet network, and the final target action information recognition result is generated using multi-level anchor-free and improved non-maximum suppression algorithms.

[0042] As shown in Figure 2, the construction approach for the dynamic spatiotemporal action detection network model in complex scenarios is as follows:

[0043] (1) Video data preprocessing: The input of this network model is an action sequence containing multiple frames, with a shape of 3×N×height×width, where N represents the number of frames. Each frame is a three-channel R, G, B image containing the spatial information of the current frame. Temporal information is expressed through the combination of consecutive frames. Such input fully reflects the two core features of the action sequence. Spatial and temporal information take into account the expression of spatiotemporal features, providing the network with a more comprehensive data foundation, enabling it to efficiently capture the complex characteristics in the action sequence.

[0044] (2) 3D convolutional layer extracts spatiotemporal features: The input multi-frame sequence image first enters the 3D convolutional layer. The 3D convolutional layer captures action features by processing the changes of consecutive frames in the time dimension and calculates intra-frame feature values ​​in the space dimension, thereby simultaneously extracting spatial and temporal information to obtain the spatiotemporal features of the video. The output tensor of such 3D convolution is usually 3×T×H×W. The extracted spatiotemporal features retain the dynamic information of the action of the input video frame changing over time, so as to better represent the dynamic features of the action;

[0045] (3) 2D convolutional layer extracts spatial features: While processing the entire frame sequence, the model extracts one frame from the video frame sequence as a key frame. These frames can effectively provide static feature information. The selection of key frames ensures the efficiency of the model in target detection and avoids the introduction of redundant information. These key frames are fed into the 2D convolutional layer separately. The 2D convolution focuses on extracting the spatial features of a single frame image, capturing static information such as edges and textures, without considering the time dimension, in order to extract frame-level detailed features and provide static feature information for subsequent action recognition.

[0046] (4) Output target location and action information: The fused features are passed to the classification network, and finally the results are output through the classification layer to complete the recognition or detection of action categories in the video. This layer outputs information such as action categories and target attributes through multi-level branches to provide output results for the final software display interface.

[0047] Through a dynamic spatiotemporal action detection network model, the synergistic effect of 2D and 3D feature extraction allows the network to deeply capture the spatial features of images in keyframes, such as texture, edge, and shape information, providing support for the static representation of the target. At the same time, 3D feature extraction captures dynamic changes between frames through temporal processing, effectively integrating the temporal features in the video. This fusion of temporal and spatial features enables the network to cope with different types of challenges in action detection, accurately capturing rapid action changes while maintaining a complete understanding of the target even when it is partially occluded or the action is slow, thus achieving comprehensive recognition of video actions.

[0048] As shown in Figure 3, the Dynamic Spatiotemporal Action Detection Network (D-STADNet) model is used in complex scenarios to accurately acquire video data of equipment operation actions in complex environments. D-STADNet further deepens the model's understanding of actions through multi-branch residual 3D convolution, mitigating the impact of video jitter on detection from the network's knowledge representation level. Through deep mining in the temporal dimension, the residual 3D convolution module enables the model to capture richer temporal features and improves sensitivity to dynamic changes in actions. A feature pyramid is used to extract multi-scale keyframe graphic structural features from the bottom up. This design captures spatial information of different granularities layer by layer, complementing both local details and overall semantics. By designing an upsampling, top-down backtracking path within the pyramid, deep semantic features are effectively transmitted back to shallower layers, achieving anchored fusion of concrete structural features and abstract semantic features. This bidirectional feature interaction enhances the model's ability to express target features in complex scenes. The spatial features extracted by 2D convolution are decoupled by the path aggregation network and converge with the temporal features extracted by 3D convolution at the decoupling fusion head. This stage of the design aims to fully leverage the advantages of 2D and 3D convolutions, ensuring that the final fused features can accommodate both static textures and dynamic actions. The fused features are used for anchor point regression and action classification. A multi-stage regression strategy progressively optimizes the accuracy of bounding box predictions, and an improved non-maximum suppression algorithm eliminates redundant detection structures to generate the final output. To clearly distinguish the names of each module in Figure 3, prefixes such as "first," "second," and "third" are used for specific descriptions, differentiating each module and its components.

[0049] In Figure 3, the extended layer aggregation network involved in step one includes two shallow convolutional blocks. The shallow convolutional blocks are connected to a first extended layer aggregation module. The first extended layer aggregation module is connected to a first max pooling module. The first max pooling module is connected to a second extended layer aggregation module. The second extended layer aggregation module is connected to a second max pooling module. The second max pooling module is connected to a third extended layer aggregation module. The third extended layer aggregation module is connected to a third max pooling module. The third max pooling module is connected to a fourth extended layer aggregation module. The first extended layer aggregation module is connected to a first convolutional layer. The first convolutional layer is connected to a batch normalization layer.

[0050] In Figure 3, the Feature Pyramid Network involved in step one includes a second convolutional layer connected to the second extended aggregation module, a third convolutional layer connected to the third extended aggregation module, a pyramid pooling module connected to the fourth extended aggregation module, and a first stitching module connected to the first batch of normalization layers. The second convolutional layer is sequentially connected to the second batch of normalization layers, the second stitching module, the fifth extended aggregation module, and the fourth max pooling module. The third convolutional layer is sequentially connected to the third batch of normalization layers, the third stitching module, the sixth extended aggregation module, and the first upsampling convolutional module. The first upsampling convolutional module is connected to the second stitching module. The first stitching module is sequentially connected to the fourth convolutional layer, the fourth batch of normalization layers, and the second upsampling convolutional module. The pyramid pooling module is connected to the first stitching module, and the second upsampling convolutional module is connected to the third stitching module. The Feature Pyramid Network (FPN) is a network structure designed to solve the problem of target scale variations in complex scenes. The design focus of Feature Pyramid Network (FPN) is to generate multi-scale feature maps to effectively handle targets of different sizes within the network, improving the accuracy and robustness of target detection. FPN extracts semantic information from low-resolution, deep features and passes it layer by layer to high-resolution, shallow feature maps, forming a top-down multi-scale feature pyramid. This structure allows the network to fully utilize deep semantic features to improve the detection capability for large targets when dealing with targets of various scales, while retaining high-resolution shallow features for detecting small targets. In complex engineering equipment operation scenarios, the upsampling process in the feature pyramid enhances the network's ability to perceive multi-scale targets. This process, through hierarchical upsampling and feature fusion operations, effectively combines low-resolution abstract features (such as contours and overall shapes) with high-resolution detailed features (such as the texture and edges of components), forming a comprehensive feature map containing information at different levels. This feature fusion not only empowers the network to learn more abstract and generalized semantic information but also preserves key detailed features in the equipment structure. It is of great significance for detecting small components or anomalies in complex backgrounds. With the help of multi-scale feature fusion strategies, deep learning object detection networks can effectively capture target information at different scales in engineering equipment operations, improving overall detection performance and accuracy, and providing more reliable technical support for the safe monitoring and maintenance of equipment. The FPN synthesizes information in bottom-up and top-down paths through operations such as convolution and pooling. In the bottom-up path, shallow feature maps are passed up layer by layer to extract high-resolution detailed information; in the top-down path, deep semantic features are passed down layer by layer and fused with the features of the previous layer, thereby generating a multi-scale feature map pyramid. This multi-scale feature pyramid effectively improves the network's performance in recognizing and locating targets at different scales. Especially in complex scenes, FPN enhances the network's performance in multi-scale detection tasks by integrating features at different levels.The structure of this process is shown in Figure 4, where feature maps of different scales represent the feature information of each layer of the network.

[0051] In Figure 3, the multi-branch 3D residual network involved in step two includes a fourth convolutional layer, which is connected to a fifth max pooling layer. The fifth max pooling layer is connected to three first multi-branch residual blocks. Each multi-branch residual block is connected to a first multi-branch downsampling residual block. Each first multi-branch downsampling residual block is connected to three second multi-branch residual blocks. Each second multi-branch residual block is connected to a second multi-branch downsampling residual block. Each second multi-branch downsampling residual block is connected to 22 third multi-branch residual blocks. Each third multi-branch residual block is connected to a third multi-branch downsampling residual block. Each third multi-branch downsampling residual block is connected to two fourth multi-branch residual blocks.

[0052] Multi-scale residual augmented convolution in simple scenarios leverages the advantages of residual connections to extract high-quality features from both within keyframes and between consecutive frames for action classification and regression. However, in complex scenarios, considering perturbation factors such as the small size of the excavator target, camera shake, and low visibility, the extraction and representation of spatiotemporal features are easily interfered with, making subsequent feature fusion not only ineffective but also introducing noise and causing performance degradation. To address this issue, this invention employs a multi-branch residual convolution module in models for complex scenarios and expands the feature extraction dimension of the network using the concept of "card count" to enhance the model's ability to capture complex features. "Card count" defines the number of parallel branches (i.e., branches) in the network. Each branch performs independent convolution operations on the input features, capturing feature details from different levels and perspectives. Finally, the outputs of each branch are converged into a unified feature representation through summation or concatenation operations. By increasing the number of cards, the network can improve the diversity and robustness of feature extraction without significantly increasing the number of parameters, effectively alleviating the problem of spatiotemporal feature failure in complex scenes. It can not only better cope with subtle changes and occlusions of the target, but also maintain high-performance feature expression in diverse environments. Its specific structure is shown in Figure 5. Each branch learns different feature patterns, capturing more diverse features and making the model more expressive and generalizable when processing complex data. During training, because shallow structural features are directly passed to deeper layers, multiple branches adaptively learn features at a finer granular level. Since there are many branches, and each branch has independent parameters, the model adaptively learns features from different "viewpoints" in different branches, making the entire network more capable of learning dynamic actions. Furthermore, as the representational power of 3D convolution is further enhanced, the learning pressure of 2D convolution on the internal structural features of keyframes is reduced, further improving the quality and representational power of structural and dynamic features.

[0053] In Figure 3, the path aggregation network involved in step three includes a first feature decoupling module connected to the fifth aggregation module, a fourth stitching module connected to both the sixth extended aggregation module and the fourth max pooling module, a fifth stitching module connected to the fourth batch normalization layer, a seventh extended aggregation module connected to the seventh extended aggregation module, a fifth max pooling module connected to the second feature decoupling module, a fifth max pooling module connected to the fifth stitching module, an eighth extended aggregation module connected to the third feature decoupling module. The Path Aggregation Network (PAN) is an optimized extension of the feature pyramid network, with the core objective of enhancing the expressive power of multi-scale features to improve the accuracy and robustness of target detection, especially in complex equipment operations. PAN further optimizes the network's detection capabilities for both small and large engineering equipment through improved feature fusion strategies, enabling the model to better integrate feature information at different levels. In target detection, features at different scales are crucial for accurately detecting small equipment components and large structures. Traditional Feature Pyramid Networks (FPNs) primarily fuse features via a top-down path. This approach mainly addresses the insufficient expressive power of high-level semantic information in small object detection. However, this unidirectional transmission mode can be limited in certain scenarios. Path Aggregation Networks (PANs) further optimize this by introducing a bottom-up path, enabling bidirectional transmission of multi-scale features and achieving more comprehensive feature fusion. This bidirectional feature transmission mechanism not only helps identify detailed features in equipment structures but also enhances the semantic understanding of the overall structure. The improved Path Aggregation Feature Pyramid structure is shown in Figure 6, providing stronger theoretical performance for motion detection tasks in equipment operations. PANs are not limited to improving feature fusion but also bring comprehensive optimization to multi-scale detection scenarios. In complex engineering equipment operation environments, they are particularly suitable for detecting fine components and small, hard-to-capture anomalies, while also accurately identifying the overall shape and large structures of equipment. Ultimately, through PAN's multi-scale feature fusion strategy, the performance of deep learning object detection models in complex scenarios is improved. Without Path Aggregation Networks (PANs), the model would be limited in multi-scale feature fusion, especially in the insufficient transmission and fusion of bottom-up detailed information and high-level semantic information. Traditional feature pyramid networks (such as FPN) can only achieve top-down feature propagation. This means that high-level semantic information can be passed to low-level features, but details in low-level features cannot be effectively passed back to high-level features. This leads to poor model performance when detecting small objects and complex scenes, because the rich details in low-level features are not fully utilized and integrated. The path aggregation network utilizes a pyramid pooling module, an upsampling convolution module, an extended aggregation module, and a feature decoupling module.The module structures used in the enhanced pyramid are shown in Figure 7. Max pooling effectively extracts contour and texture features from each frame while reducing data volume, making it an effective way to extract key information. In the upsampling convolution module, high-channel, low-size features in higher layers are first processed by 1×1 convolution to reduce the number of channels, and then the feature map is enlarged through convolutional upsampling to match the size of the lower-layer features. Convolutional upsampling is an operation that improves the spatial resolution of feature maps, aiming to map small-size feature maps to larger sizes. It is commonly used in deep networks to gradually reconstruct detailed information and enhance the expressive power of spatial features. Max pooling is used in both the pyramid pooling and max pooling modules, and this principle is basically consistent in simple scenes. It reduces the spatial dimension of the feature map by selecting the maximum value in a local region while preserving salient features, such as the contour and texture of the target. This process not only compresses the amount of data and reduces computational complexity, but also enhances the model's ability to focus on the salient features of the target, especially performing well when there are large changes between frames. In the upsampling convolution module, high-channel, low-size features at higher levels are first processed through 1×1 convolutions to reduce the number of channels. Then, convolutional upsampling is used to enlarge the feature map to match the size of the lower-level features. Convolutional upsampling is an operation that improves the spatial resolution of feature maps, aiming to map small-sized feature maps to larger sizes. It is commonly used in deep networks to progressively reconstruct detailed information and enhance the expressive power of spatial features. This process is particularly important in engineering equipment operations because it helps extract detailed features of engineering equipment structures in complex scenes. Convolutional upsampling is often combined with feature fusion operations, utilizing the integration of multi-level features to generate more expressive feature representations. In this process, the upsampled feature map can be concatenated or weighted with feature maps from the previous layer (or other scales) to form a comprehensive feature with rich multi-scale information. Through this combination of multi-level information, convolutional upsampling provides a solid foundation for the network to recognize targets of complex shapes or different scales (such as components and construction equipment), helping to improve the accuracy of tasks such as target detection and image segmentation. The calculation method for convolutional upsampling is shown in Figure 8. The effective application of this method further enhances the safety monitoring and maintenance capabilities in engineering equipment operations. By inserting zero values ​​into the smaller feature map and padding it with zeros to enlarge its size, an "expanded" feature map is formed. Then, the convolutional kernel slides across this expanded feature map, covering a local region at a time and calculating the dot product of the non-empty pixels to generate the output pixel value. Through this sliding and calculation process, the input feature map is gradually enlarged, generating a higher-resolution output feature map, achieving a spatial upsampling effect. Convolutional upsampling not only restores the spatial size but also introduces nonlinear transformations during the feature map enlargement process, thereby enriching the expressive power of the feature map. This process typically uses a fixed-size convolutional kernel to map each pixel of the feature map to a larger-sized output feature map.Therefore, convolutional upsampling is not only a spatial resolution restoration technique, but also enhances feature representation through nonlinear convolution operations, providing higher-quality input for subsequent feature fusion and multi-scale information integration. This enables the network to fully utilize multi-level feature information for accurate prediction in complex scenes. To further decompose the two-dimensional structural features of the equipment within the keyframe into category and location features, so that subsequent anchorless detection can predict location and category, a feature decoupling module is specifically designed within the network to decompose regression and classification features. This module processes the input features step by step through a series of convolution and activation operations. The module consists of three cascaded convolutional operations, each including a convolutional layer (3×3), a batch normalization layer, and a SiLU activation function. After the first convolutional layer, batch normalization and SiLU activation functions are connected for preliminary feature extraction. Then, the features enter the second layer, where they are divided into two parallel paths, each performing convolution, batch normalization, and SiLU operations respectively. Finally, in the third layer, these two paths output "regression features" and "classification features" respectively, with each path using the same convolutional structure to extract features for different tasks. The regression feature path is used for localization or bounding box regression, while the classification feature path is used for object classification. The SiLU activation function in the network module combines the advantages of ReLU and Sigmoid to maintain non-linear transformation and enhance gradient stability. The expansion layer aggregation module is the initial module for extracting 2D convolutional features, designed to extract features at different stages in a hierarchical manner and enhance the representational power of the current layer's features. This module enhances the effect of feature extraction and aggregation through a unique "expansion-then-aggregation" structure. The module starts with two parallel convolutional paths, each containing a 1×1 convolutional layer and a batch normalization layer, followed by the application of the SiLU activation function to extract local features. The outputs of these two paths are passed to the intermediate part, one of which undergoes multiple convolutional operations (each layer containing a 3×3 convolution, batch normalization, and SiLU activation function), repeated 4 times to progressively enrich the feature representation. The output of the intermediate part is combined with the features of the initial path in the "concatenation" step, thereby fusing information at different scales and depths. Finally, the concatenated features are passed through a 3×3 convolutional layer, a batch normalization layer, and the SiLU activation function to generate the module's output. This module is designed to combine shallow and deep features to enhance the model’s ability to express multi-scale information and to achieve stronger feature learning and spatiotemporal information integration during feature aggregation.

[0054] In Figure 3, the decoupling fusion head involved in step four includes a first decoupling fusion head, a second decoupling fusion head, and a third decoupling fusion head. The first decoupling fusion head includes a multi-branch 3D convolutional dynamic motion feature receiving block, a first-channel decoding module, and a second-channel decoding module. The multi-branch 3D convolutional dynamic motion features and equipment operation action category features are both input into two fifth convolutional layers through the first-channel decoding module for convolutional feature extraction. The extracted features are then passed through a sixth convolutional layer to obtain the predicted position value of the equipment operation action. The multi-branch 3D convolutional dynamic motion features and equipment operation action position features are input into two seventh convolutional layers through the second-channel decoding module for convolutional feature extraction. The extracted features are then passed through an eighth convolutional layer to obtain the predicted equipment operation category value. The second decoupling fusion head receives the equipment operation action category features and position features extracted by the second feature decoupling module, and simultaneously receives the multi-branch 3D convolutional dynamic motion features of the equipment operation action. The third decoupling fusion head receives the equipment operation action category features and position features obtained by the third feature decoupling module.

[0055] The design goal of the channel coding structure module is to achieve efficient fusion of 2D and 3D convolutional features, thereby enhancing the model's representation capabilities in complex equipment operation scenarios. The core idea of ​​this module is to combine the fine-grained perception capabilities of 2D convolutional features with the spatiotemporal dynamic capture capabilities of 3D convolutional features, enabling the network to more comprehensively capture the spatial information and dynamic motion features of engineering equipment structures in a multi-dimensional feature space. Specifically, 2D convolutional features can effectively extract detailed information about engineering equipment operations, such as the equipment's posture and the relative positions of personnel and equipment. This type of information is usually reflected in the spatial distribution of the image; therefore, 2D convolution performs well in processing texture, edges, and regional contrast in static images. Simultaneously, the high-resolution representation capability of 2D convolutional features can also help identify small but crucial components or texture features in mechanical structures. 3D convolutional features, by extending the convolution operation in the temporal dimension, can extract depth information and structural shape changes related to dynamic changes. For example, 3D convolution can capture the continuity of equipment movement trajectories during engineering equipment operations, as well as the dynamic characteristics of mechanical structures caused by the operation process. These features can effectively describe the overall motion patterns of engineering equipment and help the network model and understand time-series data. In the decoupled fusion head, the detection network for complex scenes, through the design of an innovative channel coding structure, can effectively convert given... Remodeling The channel encoder first connects the channels along the channel dimension, and then performs main channel fusion through two convolutional layers after batch normalization and activation functions. The channel encoding process is shown in Figure 9. Channel encoding includes channel fusion and channel self-attention mechanisms, both of which are used to fuse two-dimensional and three-dimensional features. The main process is as follows: First, the main purpose of the channel fusion module is to concatenate the spatiotemporal feature map (3D features) and the spatial feature map (2D features) along the channel dimension. Through this operation, the originally independent two-dimensional and three-dimensional features are mapped to the same feature space for subsequent fusion operations. To avoid excessive computational overhead due to too many channels after concatenation, a 1×1 convolution is used to compress the number of channels. This convolution operation reduces the dimensionality of the features through linear transformation while maintaining the integrity of the information. Then, a 3×3 convolution is used... The product layer further processes the compressed features, extracting and integrating the concatenated information. This step helps capture the local correlations between different features, thus achieving preliminary fusion. In the channel attention module, global contextual information needs to be extracted from the fused feature map. Specifically, the feature map of each channel is globally aggregated in the spatial dimension, and its average (mean pooling) and maximum (max pooling) are calculated to generate two channel vectors. These channel vectors represent the global feature distribution and extreme feature distribution of each channel, respectively, to effectively capture the global representation of features across the entire channel and retain the most representative contextual information. Secondly, a nonlinear transformation is performed on the generated channel vectors. To enhance expressive power, two fully connected layers are designed, combining ReLU and Sigmoid activation functions to achieve nonlinear mapping. The first fully connected layer is used to reduce the dimensionality of the channel vectors, compressing information and reducing computational overhead; the second fully connected layer restores the original number of channels and generates channel weight vectors through the Sigmoid function. This channel weight vector directly reflects the importance of each channel in a specific task. The introduction of nonlinear transformation enables the attention mechanism to better adapt to the task requirements of complex scenarios and capture the nonlinear relationships between channels. Finally, the generated channel weight vectors are applied to the fused original feature map. Specifically, the channel weights are multiplied one by one by each channel of the original feature map, thus weighting each channel. This weighting operation effectively highlights the expression of key channels within the network while suppressing less important channels, making the feature map representation more compact and efficient. For engineering equipment operation tasks, this mechanism can improve the perception of complex equipment structures, dynamic actions, and operational states by optimizing feature distribution. The significant advantage of channel attention lies in its ability to achieve refined feature selection by aggregating global feature information. In engineering equipment operations, this mechanism not only enables the model to utilize the spatial and channel information of the engineering equipment more effectively but also further improves the compactness and accuracy of feature representation.Furthermore, channel attention can be effectively combined with other attention mechanisms (such as spatial attention) to enhance the network's performance in various equipment motion detection tasks. This comprehensive strategy enables deep learning models to capture important information more efficiently and accurately and respond accordingly when handling complex engineering equipment structural condition monitoring.

[0056] It should be noted that in step five, the improved anchor-free CenterNet network includes a center point heatmap generation module, a center point offset prediction branch, a target width and height prediction branch, and an anchor-free bounding box generation module.

[0057] It should be noted that step five specifically includes:

[0058] Step 51, Generate center point heatmap: The center point heatmap generation module applies a Gaussian distribution to the corresponding pointer on the feature map for each category and normalizes the pixel values ​​to generate an independent center point heatmap. The heatmap uses a Gaussian kernel function to allocate pixel values ​​to the target center.

[0059] Step 52, Center point offset prediction: The center point offset prediction branch predicts the offset relative to the feature map grid for each candidate center point through a convolutional layer.

[0060] Step 53, Target width and height prediction: The target width and height prediction branch predicts the target width and height for each center point and outputs width and height values ​​that are adapted to the feature map resolution;

[0061] Step 54, Generate Anchorless Boundary: The anchorless boundary generation module uses a mapping function to restore the coordinates of the upper left and lower right corners of the target rectangle on the original image at the original image scale, based on the center point coordinates, predicted offset, and width and height values.

[0062] The CenterNet network transforms object detection into a center point prediction problem, representing objects by their center points and generating bounding boxes by predicting the center point offset and the object's width and height. Heatmaps are used to represent classification information, with each category corresponding to a separate heatmap. In each heatmap, when the center point of an object exists at a certain coordinate, a keypoint is generated at that coordinate and represented using a Gaussian plot. However, since the CenterNet base network is better suited for handling different object categories in its heatmap generation process, and single-frame object detection methods cannot obtain temporal motion context information, the CenterNet heatmap generation process needs to be modified to meet task requirements. Motion information in temporal space is extracted through multi-branch residual 3D convolution, and spatial information at different scales is extracted through feature pyramids. 2D information is fed into a path aggregation network to enhance its representation capabilities, while a feature deconstruction module separates the object information in keyframes. Specifically, the object's position and category information are decoupled to ensure that these two types of information do not interfere with each other, further enhancing the representation capabilities of the pyramid features and improving the clarity of feature representation. 3D spatiotemporal features are fused with decoupled 2D spatial features. The 3D features provide dynamic information, while the position and category features in the 2D features are processed by the position predictor and category predictor, respectively, outputting the center point of a single frame image to form a heatmap for that image. After the heatmap is generated, the value of each pixel can be regarded as the probability value that the location is the center point of the target. Gaussian circular annotations cause the area around the target center point to gradually decay, forming a smooth heat value distribution, which not only enhances the model's ability to locate the center point, but also effectively alleviates the problem of inaccurate target location annotation in the training data. Next, the network performs preliminary processing based on the heatmap scores. Specifically, the model extracts local extrema points from the heatmap, that is, the highest score points that may correspond to the target in each category. This step is usually combined with the non-maximum suppression (NMS) algorithm, which filters out points with low scores in the same local area, retaining only the highest score points as candidate center points. This step is crucial because the target may exist in multiple regions of the feature map. Without filtering, the network may output too many redundant boxes, reducing the efficiency and accuracy of subsequent predictions.

[0063] After initially identifying candidate center points, the model needs to further refine the specific locations of these center points. Since the heatmap is generated on a feature map, which is typically obtained by downsampling the original image multiple times, the resolution is significantly reduced. Therefore, the predicted center point coordinates at the feature map scale are often inaccurate and cannot be directly mapped to the original image. Thus, the network incorporates an offset branch to predict the offset of each candidate center point relative to its corresponding grid point. By correcting the offset, the center point coordinates can be accurately restored to the scale of the original image. This design effectively solves the problem of inaccurate localization caused by the low resolution of the feature map and also enhances the detection capability for small targets. Simultaneously, the location information predicted by the path aggregation network decouples and fuses the head and begins to play a role after generating the heatmap. This branch predicts the width and height of the target for each candidate center point; these values ​​are estimated at the feature map scale and then restored according to the resolution ratio of the feature map and the original image. The output of this branch, combined with the result of the offset branch, determines the rectangular bounding box corresponding to each candidate center point. The formulas for generating these bounding boxes are very straightforward. By combining the center point coordinates with the width and height values, the coordinates of the top left and bottom right corners of the rectangle are calculated, and the bounding box position of the target is finally output.

[0064] To address interference from complex environments and enhance the model's noise and interference resistance for application in challenging research and construction environments, this book designs an improved version of the LeakyReLU-based robust rectified linear unit activation function (Robust LeakyReLU). The Leaky ReLU-based robust rectified linear unit activation function is shown in the following formula.

[0065] ;

[0066] in It is a parameter that controls the slope of negative values, and is usually taken as a small value, between 0.01 and 0.1. and These are parameters that control the suppression effect of negative input and the noise reduction capability. Typically, they are... Take 10, Taking 1 as the input variable and x as the control input, the function behaves the same as the general ReLU when x is greater than 0. The function graph is shown in Figure 10. Traditional LeakyReLU introduces a fixed linear slope in the negative region, without distinguishing the amplitude of the input signal. Robust LeakyReLU activation function, on the other hand, introduces an exponential decay factor in the negative region that decreases rapidly as the absolute value of the input value increases. This quickly suppresses the influence of high-amplitude negative values ​​on the output, effectively suppressing potential high-frequency noise, while small-amplitude negative values ​​still retain a certain gradient, which helps to learn subtle features such as boundary features.

[0067] For consecutive frames in complex scenes, adjacent frames are usually similar in content, but may exhibit brief, rapid changes. Robust LeakyReLU's noise suppression properties can smooth these changes, further enhancing the network's learning of global spatiotemporal features. Noise interference, especially in complex scenes including small-scale rapid jitter and rain / snow, can also be a significant issue. Images in these videos often show similar overall content in adjacent frames, but rapid changes occur between frames. Therefore, activation functions with noise suppression capabilities can mitigate this type of noise interference introduced during shooting to some extent. The improved LeakyReLU activation function, based on a leaky robust rectified linear unit, effectively addresses high-frequency noise interference and input fluctuations by designing an exponential decay factor in negative value regions. This makes it suitable for use in dynamic scenes with strong noise interference, significantly improving the robustness of deep learning models. Combined with its excellent gradient propagation properties, this activation function is not only applicable to single scenes but also has broad application potential in scenarios with poor dataset quality and high-frequency noise interference.

[0068] In summary, the technical solution described in this invention achieves accurate acquisition of equipment operation action information from test video data in complex scenarios through a dynamic spatiotemporal motion detection network model. It extracts one frame from the original video sequence as a keyframe, effectively extracts static feature information of the equipment action information through 2D convolution, and deepens the model's understanding of motion through a multi-branch residual 3D convolutional network. This addresses the impact of video jitter on detection from the network's knowledge capability level. Furthermore, it extracts keyframe graphic structure features of different granularities sequentially from bottom to top through a feature pyramid, and backtracks features from top to bottom through convolutional upsampling within the pyramid, thus refining the deep-layer features. Semantic features are fed back to the bottom of the pyramid, anchoring concrete structural features with abstract semantic features. 2D convolutional features are decoupled from the features of the path aggregation network and converge with 3D convolutional features at the decoupling aggregation head to form output features. Combined with temporal and spatial features, dynamic and static information are effectively fused to help accurately capture rapid action changes and maintain a complete understanding of the target when the target is partially occluded or the action is slow, achieving comprehensive recognition of video actions. Anchor point regression and action classification are performed through the output features. After multi-stage regression and an improved nonmaximum suppression algorithm, the final equipment type, action information, and time information are obtained.

[0069] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0070] Finally: The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-precision identification method for equipment operation information in test videos of complex scenarios, characterized in that, The method for acquiring equipment operation action information from test video data in complex scene environments based on a dynamic spatiotemporal motion detection network model includes: Step 1, video data preprocessing: keyframes are extracted from the raw video data and fed into an extended layer aggregation network to extract multi-level 2D features. These multi-level 2D features are then combined with structural and semantic features through a feature pyramid to form 2D pyramid features. Step 2, multi-branch residual 3D convolutional dynamic motion feature extraction: keyframes are input into a multi-branch 3D residual network to extract deep 3D features. Step 3, path aggregation network 2D feature enhancement and decoupling: the 2D pyramid features are input into the path aggregation network to further enhance the pyramid representation. The process involves several steps: First, decoupling the position and category information of the target in the keyframe using feature decoupling. Second, spatiotemporal feature fusion based on interactive feature modeling: Using interactive modeling technology, the position and category features in the decoupled 2D spatial features are fused in the decoupling fusion head, and the output features are obtained through the position predictor and category predictor. Third, action information recognition based on the modified anchorless CenterNet network: In the output layer, anchor boxes are predicted based on the target position prediction value using the improved anchorless CenterNet network, and the final target action information recognition result is generated using multi-level anchorless and improved nonmaximum suppression algorithms.

2. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 1, characterized in that, In step one, the extended layer aggregation network includes two shallow convolutional blocks. The shallow convolutional blocks are connected to a first extended layer aggregation module. The first extended layer aggregation module is connected to a first max pooling module. The first max pooling module is connected to a second extended layer aggregation module. The second extended layer aggregation module is connected to a second max pooling module. The second max pooling module is connected to a third extended layer aggregation module. The third extended layer aggregation module is connected to a third max pooling module. The third max pooling module is connected to a fourth extended layer aggregation module. The first extended layer aggregation module is connected to a first convolutional layer. The first convolutional layer is connected to a batch normalization layer.

3. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 2, characterized in that, In step one, the feature pyramid network includes a second convolutional layer connected to the second extended aggregation module, a third convolutional layer connected to the third extended aggregation module, a pyramid pooling module connected to the fourth extended aggregation module, and a first splicing module connected to the first batch of normalization layers. The second convolutional layer is sequentially connected to the second batch of normalization layers, the second splicing module, the fifth extended aggregation module, and the fourth max pooling module. The third convolutional layer is sequentially connected to the third batch of normalization layers, the third splicing module, the sixth extended aggregation module, and the first upsampling convolutional module. The first upsampling convolutional module is connected to the second splicing module. The first splicing module is sequentially connected to the fourth convolutional layer, the fourth batch of normalization layers, and the second upsampling convolutional module. The pyramid pooling module is connected to the first splicing module, and the second upsampling convolutional module is connected to the third splicing module.

4. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 3, characterized in that, In step two, the multi-branch 3D residual network includes a fourth convolutional layer, which is connected to a fifth max-pooling layer. The fifth max-pooling layer is connected to three first multi-branch residual blocks. Each multi-branch residual block is connected to a first multi-branch downsampling residual block. Each first multi-branch downsampling residual block is connected to three second multi-branch residual blocks. Each second multi-branch residual block is connected to a second multi-branch downsampling residual block. Each second multi-branch downsampling residual block is connected to 22 third multi-branch residual blocks. Each third multi-branch residual block is connected to a third multi-branch downsampling residual block. Each third multi-branch downsampling residual block is connected to two fourth multi-branch residual blocks.

5. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 3, characterized in that, In step three, the path aggregation network includes a first feature decoupling module connected to the fifth aggregation module, a fourth splicing module connected to both the sixth extended aggregation module and the fourth max pooling module, a fifth splicing module connected to the fourth batch normalization layer, a seventh extended aggregation module connected to the seventh extended aggregation module, a fifth max pooling module connected to the fifth and second feature decoupling modules, a fifth max pooling module connected to the fifth splicing module, an eighth extended aggregation module connected to the eighth extended aggregation module, and a third feature decoupling module connected to the eighth extended aggregation module.

6. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 5, characterized in that, In step four, the decoupling fusion head includes a first decoupling fusion head, a second decoupling fusion head, and a third decoupling fusion head.

7. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 6, characterized in that, The first decoupled fusion head includes a multi-branch 3D convolutional dynamic motion feature receiving block, a first-channel decoding module, and a second-channel decoding module. The multi-branch 3D convolutional dynamic motion features and equipment operation action category features of the equipment operation actions are both input into two fifth convolutional layers through the first-channel decoding module for convolutional feature extraction. The extracted features are then passed through a sixth convolutional layer to obtain the position prediction value of the equipment operation action. The multi-branch 3D convolutional dynamic motion features and equipment operation action position features of the equipment operation actions are input into two seventh convolutional layers through the second-channel decoding module for convolutional feature extraction. The extracted features are then passed through an eighth convolutional layer to obtain the equipment operation category prediction value.

8. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 7, characterized in that, In step four, the second decoupling fusion head is used to receive the equipment operation action category features and position features extracted by the second feature decoupling module, and at the same time receive the equipment operation action multi-branch 3D convolution dynamic action features. The third decoupling fusion head is used to receive the equipment operation action category features and position features obtained by the third feature decoupling module.

9. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 8, characterized in that, In step five, the improved anchor-free CenterNet network includes a center point heatmap generation module, a center point offset prediction branch, a target width and height prediction branch, and an anchor-free bounding box generation module.

10. The high-precision identification method for equipment operation information in test videos of complex scenarios according to claim 9, characterized in that, Step 5 specifically includes: Step 51, generating a center point heatmap: The center point heatmap generation module applies a Gaussian distribution to the corresponding index on the feature map for each category and normalizes the pixel values ​​to generate an independent center point heatmap. The heatmap uses a Gaussian kernel function to allocate pixel values ​​to the target center; Step 52, center point offset prediction: The center point offset prediction branch predicts the offset relative to the feature map grid for each candidate center point through a convolutional layer; Step 53, target width and height prediction: The target width and height prediction branch predicts the target width and height for each center point and outputs width and height values ​​adapted to the feature map resolution; Step 54, generating an anchorless bounding box: The anchorless bounding box generation module restores the target rectangle's upper left and lower right corner coordinates on the original image at the original image scale using a mapping function based on the center point coordinates, predicted offset, and width and height values.

Images