A Deep Learning-Based Excavator Attitude Detection Method

Abstract

This invention belongs to the field of construction safety management and safe production, and discloses a deep learning-based excavator posture detection method. The method includes: acquiring excavator images at the construction site; performing image enhancement processing and key point annotation on the excavator images to obtain a training image dataset; constructing a target detection model; improving the target detection model based on a multi-scale feature fusion algorithm and a structure reparameterization algorithm to obtain an improved target detection model; performing module fusion improvement on the posture detection framework based on a human posture estimation network and the improved target detection model to obtain an excavator posture detection model; and inputting the image dataset with key point annotation into the excavator posture detection model to obtain detection data of excavator posture key points. This invention can improve the intelligence of construction site safety management and provide a smart management solution for safe production in the construction industry.

Description

Technical Field

[0001] This invention belongs to the field of construction safety management and safe production, and in particular relates to a deep learning-based method for excavator posture detection. Background Technology

[0002] The "Guiding Opinions on Promoting the Coordinated Development of Intelligent Construction and Building Industrialization" issued by the Ministry of Housing and Urban-Rural Development and other departments in July 2020 proposed to focus on the overall goal of high-quality development of the construction industry, drive development through digitalization and intelligentization, innovate and break through relevant core technologies, increase the application of intelligent construction in all aspects of engineering construction, and achieve the transformation, upgrading and sustainable healthy development of the construction industry. The "Sichuan Province 'Smart Construction Site' Construction Work Plan" issued by the Sichuan Provincial Department of Housing and Urban-Rural Development in August 2021 clearly pointed out that it is necessary to promote the informatization of safety production supervision of construction projects, utilize technologies such as intelligent video image recognition and information processing to support the collection and aggregation of safety information on on-site personnel behavior and machinery and equipment, and establish detection models using big data methods to promote the transformation of engineering project safety supervision towards digitalization and intelligence, shifting from post-event statistical analysis to pre-event detection and judgment, and realizing functions such as monitoring and early warning of high-risk sub-projects and the investigation and management of safety hazards. This shows that the intelligentization of engineering project construction has ushered in a major development opportunity. This research focuses on the intelligent safety detection system for construction excavators and workers, accurately grasping and implementing the relevant strategic decisions of the national and Sichuan provincial governments, and promoting the intelligent development of construction safety management.

[0003] With the booming development of my country's construction industry and the continuous expansion of construction scale, safety issues at construction sites are also on the rise. The high risk of construction sites stems from the dynamic and complex characteristics of construction-related entities, such as the movement and interaction of construction equipment and workers. Excavators play a crucial role in construction projects, but due to their complex operation and heavy weight, they often cause numerous safety problems, such as collisions with workers or other equipment during earthmoving activities.

[0004] There is an urgent need for a human-machine safety status identification design method that can solve the problem of excavator operation safety for construction workers in complex construction environments. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning-based excavator posture detection method to solve the problems existing in the prior art.

[0006] To achieve the above objectives, this invention provides a deep learning-based excavator posture detection method, comprising:

[0007] Acquire images of excavators at the construction site, perform image enhancement processing on the excavator images, and after the image enhancement processing is completed, annotate the core key points of the excavator images to obtain a training image dataset;

[0008] A YOLOv5 target detection model is constructed, and the YOLOv5 target detection model is improved based on the multi-scale feature fusion algorithm and the structure reparameterization algorithm to obtain the improved YOLOv5 target detection model.

[0009] The AlphaPose pose detection framework is improved by module fusion based on the FastPose human pose estimation network and the improved YOLOv5 target detection model to obtain the excavator pose detection model. The image dataset after the core key points are annotated is input into the excavator pose detection model to obtain the detection data of the excavator pose key points.

[0010] Optionally, the process of performing image enhancement processing on the image dataset includes: rotation, translation, flipping, and color conversion.

[0011] Optionally, the process of annotating the core key points of the image dataset includes:

[0012] The target detection bounding boxes and key points of the excavator in the image dataset were annotated using the image annotation software Labelme.

[0013] Optionally, the core key points include: the tail section, the cab, the boom, the bucket arm, the left end of the bucket, and the right end of the bucket.

[0014] Optionally, the YOLOv5 target detection model can be improved based on a multi-scale feature fusion algorithm and a structure reparameterization algorithm to obtain an improved YOLOv5 target detection model, specifically including:

[0015] A YOLOv5 object detection model is constructed, and a structural reparameterization algorithm is incorporated into the backbone network of the YOLOv5 object detection model. Furthermore, a bidirectional feature pyramid network is added to the neck network of the YOLOv5 object detection model. The backbone network and neck network of the YOLOv5 object detection model are improved and fused through the bidirectional feature pyramid network and the structural reparameterization algorithm to obtain an improved YOLOv5 object detection model.

[0016] Optionally, a structural reparameterization algorithm can be incorporated into the backbone network of the YOLOv5 object detection model, specifically including:

[0017] During the training of the YOLOv5 object detection model, a multi-branch model is used. During the inference process of the YOLOv5 object detection model, the multi-branch model is converted into a single-path model, so that the training and inference stages of the YOLOv5 object detection model use different network architectures to achieve a balance between accuracy and speed.

[0018] Optionally, a bidirectional feature pyramid network is further added to the neck network of the YOLOv5 object detection model, specifically including:

[0019] Different weights are assigned to each layer of the neck network in the YOLOv5 object detection model, and the weights are then fused into the corresponding neck network.

[0020] Optionally, the process of obtaining the excavator posture detection model includes: training the FastPose human posture estimation network and the improved YOLOv5 object detection model based on the image dataset, and performing module fusion improvement on the AlphaPose posture detection framework based on the trained FastPose human posture estimation network and the trained YOLOv5 object detection model to obtain the excavator posture detection model.

[0021] Optionally, the AlphaPose pose detection framework can be improved by module fusion based on the trained FastPose human pose estimation network and the trained YOLOv5 object detection model, specifically including:

[0022] An improved YOLOv5 target detection model is incorporated into the first stage of the AlphaPose posture detection framework, and an extended FastPose human pose estimation network is incorporated into the second stage. This completes the improvement and optimization of the CMOD module in the first stage and the SPPE module in the second stage of the AlphaPose posture detection framework, resulting in the excavator posture detection model.

[0023] Optionally, the image dataset after core key point annotation is input into the excavator posture detection model to obtain the detection data of the excavator posture key points, specifically including:

[0024] The image dataset after core key point annotation is used as the input image and input into the excavator posture detection model. Spatial transformation is performed on the input image, and the target detection box is obtained through the first stage CMOD module of the excavator posture detection model. The region of interest is extracted from the target detection box to realize automatic adjustment of the detection box.

[0025] The image feature extraction process is optimized by the SMPE module of the excavator posture detection model. A feature extraction layer with SEResNet deep residual network as the basic skeleton is established based on the SE block in the FastPose human posture estimation network. The input image is upsampled by the PixelShuffle subpixel convolutional layer and the DUC layer. Finally, the detection data of the excavator posture key points are output in the output layer.

[0026] The technical effects of this invention are as follows:

[0027] This invention provides a deep learning-based excavator posture detection method, comprising: acquiring excavator images at a construction site; performing image enhancement processing on the excavator images; annotating key points on the excavator images after image enhancement processing to obtain a training image dataset; constructing a YOLOv5 object detection model; improving the YOLOv5 object detection model based on a multi-scale feature fusion algorithm and a structure reparameterization algorithm to obtain an improved YOLOv5 object detection model; performing module fusion improvement on the AlphaPose posture detection framework based on the FastPose human pose estimation network and the improved YOLOv5 object detection model to obtain an excavator posture detection model; and inputting the image dataset with key point annotation into the excavator posture detection model to obtain detection data of excavator posture key points.

[0028] This invention establishes an excavator posture detection model by improving and optimizing the YOLOv5 target detection model and extending the AlphaPose posture estimation model. This method can provide timely warnings of potential unsafe behaviors and states of excavators to reduce construction safety risks for construction equipment and workers.

[0029] This invention provides a deep learning-based excavator posture detection method, which is beneficial to improving the work efficiency of construction site managers, enhancing the efficiency and intelligence of construction site safety management, and providing a smart management solution for safe production in the construction industry. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0032] Figure 1 A flowchart illustrating the improved fusion of the YOLOv5 target detection model in this embodiment of the invention;

[0033] Figure 2 This is a flowchart of the detection process in an embodiment of the present invention;

[0034] Figure 3 This is a diagram illustrating the construction process of the excavator posture detection model in an embodiment of the present invention. Detailed Implementation

[0035] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0036] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0037] Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. While only preferred methods have been described herein, any methods similar or equivalent to those described herein may be used in the implementation or testing of this invention. All references to this specification are incorporated by way of citation to disclose and describe the methods associated with those references. In the event of any conflict with any incorporated reference, the content of this specification shall prevail.

[0038] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.

[0039] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0040] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] Example 1

[0042] like Figures 1-3 As shown, this embodiment provides a deep learning-based excavator posture detection method, including:

[0043] Acquire images of excavators at the construction site, perform image enhancement processing on the excavator images, and after the image enhancement processing is completed, annotate the core key points of the excavator images to obtain a training image dataset;

[0044] A YOLOv5 target detection model is constructed, and the YOLOv5 target detection model is improved based on the multi-scale feature fusion algorithm and the structure reparameterization algorithm to obtain the improved YOLOv5 target detection model.

[0045] The AlphaPose pose detection framework is improved by module fusion based on the FastPose human pose estimation network and the improved YOLOv5 target detection model to obtain the excavator pose detection model. The image dataset after the core key points are annotated is input into the excavator pose detection model to obtain the detection data of the excavator pose key points.

[0046] The feasible approach involves collecting images of excavators at construction sites through both on-site and network data acquisition methods, generating an image dataset, and then expanding the dataset using data augmentation techniques such as rotation, translation, flipping, and color conversion. The core key points of the excavator are defined considering the machine component joints and overall contour. Referring to the annotation principles of human posture estimation, this invention annotates the key points of the excavator in an "object-centered" manner.

[0047] Multi-scale feature fusion and structural reparameterization algorithms are incorporated into the YOLOv5 object detection model to improve the accuracy of object detection.

[0048] The first stage of the AlphaPose posture detection framework incorporates an improved YOLOv5 algorithm, and the second stage incorporates the FastPose human posture estimation network to establish the excavator posture detection model YOLOv5-FastPose. This accurately detects the core key points of the excavator and connects all key points to obtain the working posture of each excavator. The technical solution of this invention can solve the safety problem of excavators operating in complex construction environments, as described above.

[0049] Feasible steps include establishing an image dataset and defining and labeling key points of the excavator. Images of excavators at the construction site are collected via network data acquisition to generate an image dataset, which is then expanded using data augmentation techniques such as rotation, translation, flipping, and color conversion. The core key points of the excavator are defined considering the machine's component joints and overall contour. Referring to the labeling principles of human posture estimation, the key points of the excavator are labeled using an object-centered approach.

[0050] A feasible approach is to improve the YOLOv5 target detection model and establish an excavator target detection algorithm. The backbone network and neck network will be improved and optimized to enhance the speed and accuracy of the YOLOv5 target detection model in automatically detecting excavators.

[0051] It is feasible to optimize the deep neural network training algorithm and establish a pose detection model. Improvements and optimizations were made to the two core modules, CMOD and SPPE, resulting in the improved model YOLOv5-FastPose.

[0052] STN adaptively performs various spatial transformations on the input image to improve classification accuracy, and extracts the region of interest from the excavator detection box to achieve automatic adjustment of the detection box.

[0053] The SMPE module constructs a FastPose pose estimation network to optimize the image feature extraction process, such as... Figure 3 As shown, this network integrates the SE block into ResNet-101, establishes a feature extraction layer with SEResnet as the basic skeleton, and performs upsampling operations through the PixelShuffle layer and DUC layer. Finally, it outputs 6 key points of the construction machinery in the output layer to effectively detect the key points of the construction machinery's posture.

[0054] The SDTN is located after the excavator attitude estimation network and is used to generate attitude proposals;

[0055] PGPG is used in the training of object detection and single-person pose estimation networks to augment existing data with pose guidance, increase training samples, achieve the purpose of data augmentation, and obtain better model performance.

[0056] PPNMS uses a parametric pose nonmaximum suppression method to solve the problem of pose redundancy detection. It eliminates redundant detection boxes by calculating pose similarity by defining pose distance. When the similarity is lower than a certain threshold, it is deleted as a redundant box. In other words, the pose with the highest confidence is used as a reference, and some poses that are close to it are eliminated by certain criteria.

[0057] The model training utilizes the mean squared error (MSE) loss function to improve the accuracy and robustness of pose recognition, and Adam is selected as the optimizer for the loss function in the model.

[0058] In FastPose pose estimation networks, SEResnet possesses powerful image feature extraction capabilities, enabling the training of networks with hundreds or even thousands of layers while maintaining excellent performance and strong image representation capabilities. The fundamental building block of SEResnet, "Squeeze-and-Excitation" (SE), focuses on channel relationships. This novel unit structure explicitly models the interdependencies between channels, adaptively recalibrating the channel feature responses.

[0059] Step S1: Establish an image dataset and define and label the key points of the excavator. Images of excavators at the construction site are collected via network data acquisition to generate an image dataset. Data augmentation techniques such as rotation, translation, flipping, and color conversion are used to expand the dataset. Considering the machine component joints and overall outline, the core key points of the excavator are defined, including: tail section, cab, boom, bucket arm, left end of bucket, and right end of bucket (6 key points). Referring to the labeling principles of human posture estimation, this design method labels the key points of the excavator in an "object-centered" manner.

[0060] Step S2: First, a Restructuring and Reparameterization (RepVGG) model is integrated into the backbone network. A multi-branch model is used during training, and converted to a single-branch model during inference. This allows the YOLOv5 model to use different network architectures for training and inference. Because the simpler single-branch structure is used during inference, fewer model parameters are required. Restructuring and Reparameterization allows for different network architectures for training and inference, prioritizing accuracy during training and speed during inference, achieving a better accuracy-speed balance. Second, a Bidirectional Feature Pyramid Network (BiFPN) is added to the neck network. Different weights are assigned to each layer of the neck network and fused, allowing the network to focus more on important layers, reducing unnecessary layer connections, enhancing the network's multi-scale feature fusion capability, and improving object detection accuracy.

[0061] The process of adding a bidirectional feature pyramid network to the neck network of the YOLOv5 model includes: Step S3, adopting the AlphaPose pose estimation algorithm based on a two-step framework of region pose estimation and introducing it into the field of construction safety monitoring. The native AlphaPose framework is suitable for human pose estimation. Considering the difference between the pose features of excavators and humans, the second-stage SPPE algorithm in the AlphaPose network model is improved. To improve the accuracy and robustness of pose recognition, the training process of AlphaPose is studied, and the error function or learning algorithm (such as Adam, Adagrad, etc.) is optimized to improve the efficiency and accuracy of excavator pose estimation. In order to extract the feature information of the excavator more accurately, the aforementioned improved YOLOv5 algorithm is integrated into the first stage of the AlphaPose framework, and the FastPose human pose estimation network is integrated into the second stage to establish an excavator pose detection model, thereby accurately detecting the core key points of the excavator, and connecting each key point in the order of tail, cab, boom, bucket arm, bucket-left end, and bucket-right end to obtain the whole body pose of the excavator. The specific steps include:

[0062] Step 3.1: STN adaptively performs various spatial transformations on the input image to improve the accuracy of classification, and extracts the region of interest from the human detection box to achieve the purpose of automatically adjusting the detection box;

[0063] Step 3.2: Optimize the image feature extraction process through the SMPE module of the excavator posture detection model. Integrate the SE block in the FastPose human pose estimation network into the excavator posture detection model, establish a feature extraction layer with SEResNet deep residual network as the basic skeleton, and perform upsampling operation on the image through PixelShuffle subpixel convolutional layer and DUC layer. Finally, output the detection of key points of excavator posture in the output layer.

[0064] The SMPE module constructs the FastPose human pose estimation network to optimize the image feature extraction process. This network integrates the SE block into ResNet-101, establishes a feature extraction layer with SEResnet as the basic skeleton, and performs upsampling operations through the PixelShuffle layer and DUC layer. Finally, it outputs 6 key points of the excavator in the output layer to effectively detect the key points of the excavator's pose.

[0065] Step 3.3: The SDTN is located after the excavator attitude estimation network and is used to generate attitude proposals;

[0066] Step 3.4: PGPG is used in the training of object detection and single-person pose estimation networks to augment existing data with pose guidance, increase training samples, achieve the purpose of data augmentation, and obtain better model performance.

[0067] Step 3.5: PPNMS uses a parameterized pose nonmaximum suppression method to solve the problem of pose redundancy detection. It eliminates redundant detection boxes by calculating pose similarity by defining pose distance. When the similarity is lower than a certain threshold, it is deleted as a redundant box. That is, the pose with the highest confidence is used as a reference, and some poses close to it are eliminated by certain criteria.

[0068] Step 3.6: Model training utilizes the mean squared error (MSE) loss function to improve the accuracy and robustness of pose recognition. Adam is selected as the optimizer for the loss function in the model.

[0069] Preferably, in step S3.6, the formula for calculating the loss function is:

[0070] Based on the research on construction machinery working posture recognition and human posture estimation methods, this invention proposes a design method and structure for human-machine safety status recognition based on excavator posture estimation.

[0071] This invention first uses the image annotation software Labelme to annotate the target detection bounding box and key point ground truth labels of the excavator in the image; it then integrates a multi-scale feature fusion algorithm into the YOLOv5 target detection model to improve the accuracy of target detection, and applies a structure reparameterization algorithm to the YOLOv5 target detection model to improve the inference speed of the model; finally, it improves and optimizes the two core modules, the CMOD module and the SPPE module, and establishes the YOLOv5-FastPose excavator posture detection model.

[0072] The beneficial effects of this invention are as follows: By improving and optimizing the YOLOv5 target detection model and the AlphaPose human posture estimation model, this invention establishes a YOLOv5-FastPose posture detection model for excavators. This method can provide timely early warnings for potential unsafe behaviors and states, thereby reducing construction safety risks for construction equipment and workers. This invention provides a deep learning-based excavator posture detection method, which is beneficial for improving the work efficiency of construction site managers, enhancing the efficiency and intelligence of construction site safety management, and providing a smart management solution for safe production in the construction industry.

[0073] Since construction projects typically involve a large amount of earthmoving, excavators are among the most frequently used construction machines. Therefore, excavators play a crucial role in completing construction projects. Furthermore, the operation of excavators is complex, with numerous posture changes, making it more difficult to identify the overall posture of excavators than other types of machinery. Experiments on the excavator dataset represent the most complex scenario among all common construction equipment; therefore, the effectiveness of this design method can be demonstrated using the publicly available excavator dataset. The Construction Equipment Poses Data Set (CEPD) is used for training and testing the excavator posture detection framework. Data augmentation operations, including translation, rotation, scaling, mirroring, blurring, brightening, darkening, and noise injection, are performed on the original CEPD, resulting in 61,488 annotated keypoints (i.e., 6 keypoints per image, with 10,248 images per keypoint category). The overall visibility distribution of this dataset is shown in Table 1.

[0074] Table 1

[0075] Overall distribution v＝1 v＝0 v＝-1 Sample size 51257 9525 706

[0076] The distribution of keypoints in terms of visibility also differs for each keypoint category, as shown in Table 2:

[0077] Table 2

[0078]

[0079] Following the general rule of deep learning research for most computer vision tasks, 70% (7173 images) of the dataset was used for training, 10% (1025 images) for validation, and 20% (2050 images) for testing. The training dataset was converted into an h5 file, which stored three types of data information: image name (ASCII code), object detection box coordinates of the label, and keypoint coordinates. The final output, including the excavator keypoint coordinates, is shown in Table 3.

[0080] Table 3

[0081]

[0082] Therefore, in summary, the human-machine safety status recognition design method based on excavator attitude estimation can effectively identify the key points of excavator attitude, which is conducive to improving the work efficiency of construction site management personnel, enhancing the efficiency and intelligence of construction site safety management, and providing a smart management solution for safe production in the construction industry.

[0083] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A deep learning-based excavator posture detection method, characterized by, The method comprises the following steps: An excavator image of a construction site is acquired, image enhancement processing is performed on the excavator image, core key point labeling is performed on the excavator image after the image enhancement processing is completed, and a training image dataset is obtained; A YOLOv5 target detection model is constructed, a structure reparameterization algorithm is integrated into a backbone network of the YOLOv5 target detection model, a bidirectional feature pyramid network is further added to a neck network of the YOLOv5 target detection model, the backbone network and the neck network of the YOLOv5 target detection model are improved and fused through the bidirectional feature pyramid network and the structure reparameterization algorithm, and an improved YOLOv5 target detection model is obtained; The structure reparameterization algorithm is integrated into the backbone network of the YOLOv5 target detection model, specifically including: a multi-branch model is used in a training process of the YOLOv5 target detection model, the multi-branch model is converted into a single-path model in an inference process of the YOLOv5 target detection model, different network architectures are used in a training stage and an inference stage of the YOLOv5 target detection model, and precision-speed balance is achieved; The improved YOLOv5 target detection model and the FastPose human body posture estimation network are trained based on the image dataset, the improved YOLOv5 target detection model is integrated into a first stage of an AlphaPose posture detection framework, the extended FastPose human body posture estimation network is integrated into a second stage, the first stage CMOD module and the second stage SPPE module in the AlphaPose posture detection framework are improved and optimized, and an excavator posture detection model is obtained; and the image dataset after the core key point labeling is input into the excavator posture detection model, and detection data of excavator posture key points is obtained.

2. The method of claim 1, wherein the method comprises: The image enhancement processing is performed on the excavator image, specifically including rotation, translation, flipping and color conversion. 3.The method of claim 1, wherein, The core key point labeling is performed on the excavator image, specifically including: The target detection frame and the core key points of the excavator in the image dataset are annotated by using an image annotation software Labelme.

4. The excavator posture detection method based on deep learning according to claim 3, characterized in that The core key points include: a tail, a cab, a boom, a bucket arm, a bucket left end and a bucket right end.

5. The method of claim 1, wherein the method is based on deep learning. The bidirectional feature pyramid network is further added to the neck network of the YOLOv5 target detection model, specifically including: Different weights are given to each layer of the neck network in the neck network of the YOLOv5 target detection model, and the weights are fused into the corresponding neck network.

6. The method of claim 1, wherein the method is based on deep learning. The image dataset after the core key point labeling is input into the excavator posture detection model, and the detection data of the excavator posture key points is obtained, specifically including: The image dataset after core key point labeling is input into the excavator posture detection model as an input image, spatial transformation is performed on the input image, a target detection box is obtained through a first stage CMOD module of the excavator posture detection model, a region of interest is extracted from the target detection box, and automatic adjustment of the detection box is realized; An image feature extraction process is optimized through an SMPE module of the excavator posture detection model, a feature extraction layer taking a SEResNet deep residual network as a basic skeleton is established based on an SE block in a FastPose human body posture estimation network, and an up-sampling operation is performed on the input image through a PixelShuffle sub-pixel convolution layer and a DUC layer, and finally detection data of excavator posture key points are output at an output layer.

Images