Elevator opening and closing detection method

Abstract

The invention discloses an elevator opening and closing detection method, which is operated in a GPU (Graphics Processing Unit), stores pictures shot by a monitoring camera, cleans and marks data, and divides the data into a training set, a verification set and a test set; and modifying a YOLO11 target detection model framework, pruning a training model by using a Network Slimming method, loading a detection model, and realizing elevator opening and closing state detection. The detection method based on the improved YOLO11 and the lightweight model is used for accurately detecting the opening and closing states of the elevator, the accuracy and efficiency of detection are improved, consumption of computing resources is remarkably reduced, and a detection system is more suitable for real-time and efficient industrial application scenes.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence applications related to oil drilling platforms, and in particular to a method for detecting the opening and closing of a jack. Background Technology

[0002] During drilling platform operations, tripping and jacking up are indispensable processes, and timely replacement of drill strings is a key factor in ensuring the smooth execution of these operations. Drilling platforms typically use chucks on the derrick to clamp the drill strings, thereby facilitating drill string replacement. Although current technology can achieve automatic lifting and lowering of chucks, there are still significant shortcomings in ensuring that the chucks are properly clamped to the drill strings.

[0003] Currently, the inspection of whether the auger is clamping the drill string mainly relies on manual inspection. This method not only seriously reduces the efficiency of tripping and running the drill string, but also is incompatible with the process requirements of automated production. It has become a bottleneck problem restricting the improvement of the level of automation in drilling operations and urgently needs to be effectively solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method for detecting the opening and closing of a hanging clamp, which solves the problems of existing technologies, improves the accuracy and efficiency of detection, and is more suitable for real-time and efficient industrial application scenarios.

[0005] To achieve the above technical objectives, this invention provides a hanging card opening and closing detection method, which runs on a GPU and includes: Step 1, saving images captured by a monitoring camera, cleaning and labeling the data, and dividing it into a training set, a validation set, and a test set; Step 2, modifying the YOLO11 object detection model framework; Step 3, using the Network Slimming method to prune and train the model; Step 4, loading the detection model to realize the opening and closing state detection of the hanging card.

[0006] This invention provides a method for detecting the opening and closing of drilling rig clamps, focusing on the core issue of the opening and closing of the clamps on the derrick. It proposes a detection method based on an improved YOLO11 network and a lightweight model to achieve accurate detection of the clamp opening and closing status. By optimizing the YOLO11 network structure and combining it with lightweight model technology, this method not only improves the accuracy and efficiency of detection but also significantly reduces the consumption of computational resources, making the detection system more suitable for real-time, high-efficiency industrial applications.

[0007] As a further improvement, in step one, the explosion-proof monitoring camera SDK is used to read the video footage and save the image in PNG format. In step two, the attention mechanism is coordinate attention (CA). The YOLO11 model consists of three key parts: a backbone network, a feature fusion network, and a prediction decoder network. The backbone network extracts multi-scale features of the image, and the feature fusion network effectively fuses the P3, P4, and P5 scale features extracted by the backbone network to enhance the expressive power and robustness of the features. The prediction decoder network is responsible for decoding the target detection box and the target category to achieve accurate target detection.

[0008] As a further improvement, P3, P4, and P5 extracted by the backbone network correspond to downsampling rates of 1 / 8, 1 / 16, and 1 / 32, respectively, while P2 adopts a fusion network with a downsampling rate of 1 / 4.

[0009] As a further improvement, in step one, the ROI region with the top-left corner coordinates (600,0) and a width and height of 640 is extracted from the original image and labeled. The labels in the image include: open, pattern1, pattern2, close, during, and pipe.

[0010] As a further improvement, in step one, the dataset is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0011] As a further improvement, step two specifically includes the following steps: S201 Modify the configuration model's YAML file, add a P2 scale to the feature fusion network module part of the YAML, fuse feature maps of different scales P3, P4, and P5, and then decode the coordinate position, category, and category confidence of the detected target through the decoding head part; S202 Register the attention mechanism module, add the attention mechanism to the model's YAML file, and add a new CA attention mechanism to the network in the feature fusion part.

[0012] As a further improvement, step three includes: S301: Modify the loss function of the YOLO11 model and add an L1 regularization function as follows: , in, This represents the input and target of the training. Indicates the parameters used in training. This represents the loss function used during normal model training. Represents the sparsity parameter, This represents the regularization strength coefficient; S302: Add the C3K2_pruned, SPPF_pruned, and Detect_pruned modules to the model file in the YOLO11s source code; S303: Train the normal detection model, set the parameter path of the training images, train using the YOLO11s model, and modify the width and depth values ​​of the s model to half of their original values; S304: Sparse training, load the detection model trained in step S303; S305: Model pruning training, load the model weights trained in step S304; S307: Use the torch.export interface function to load the trained pruned model and export the ONNX model, then use the tensorRT tool trtexec to convert the ONNX model into an engine model.

[0013] As a further improvement, step S306 is included after step S305: fine-tuning the pruned model, loading the pruned model file trained in step S305, and setting the training rounds to 200.

[0014] As a further improvement, in step S303, the number of training epochs is set to 1000, the L1 regularization parameter is set to 0, the SGD optimizer function is selected, and the training patience value of the model is 50 epochs; in step S304, the L1 regularization penalty coefficient is set to 0.01, the number of training epochs is set to 200, and the patience value is set to 201; in step S305, the pruning rates are set to 0.1, 0.2, 0.3, and 0.4, respectively.

[0015] As a further improvement, step four includes: S401: Loading the engine model file and obtaining the model detection category; S402: Determining whether the target information of the drill string is detected in the detection category; S403: Determining whether the center point coordinates of the drill string are within the detection frame of the jack; S404: Calculating the distance between pattern1 and pattern2 based on the width of the detected drill string, and normalizing the calculated distance to the distance of the drill string based on the width of the detected drill string; S405: Determining whether the relative distance is lower than threshold 1. If it exceeds threshold 1, the jack is determined to be in the open state; S406: Determining whether the relative distance is lower than threshold 2. If it is lower than threshold 2, the jack is determined to be in the closed state; otherwise, the jack is determined to be in the opening / closing process state.

[0016] This invention provides a method for detecting the opening and closing of a drilling clamp, which will provide strong technical support for the full automation of drilling operations. It is expected to significantly improve the efficiency and automation level of drilling operations, while reducing the errors and costs caused by manual inspection. Attached Figure Description

[0017] Figure 1This is a schematic diagram of step three of the present invention; Figure 2 This is a schematic diagram of step four of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figures 1 to 2 As shown, this invention provides a method for detecting the opening and closing of a hanging card, which runs on a GPU and includes: Step 1, saving images captured by a monitoring camera, cleaning and labeling the data, and dividing it into a training set, a validation set, and a test set; Step 2, modifying the YOLO11 object detection model framework; Step 3, using the Network Slimming method to prune and train the model; Step 4, loading the detection model to realize the detection of the opening and closing state of the hanging card.

[0020] This invention provides a method for detecting the opening and closing of drilling rig clamps, focusing on the core issue of the opening and closing of the clamps on the derrick. It proposes a detection method based on an improved YOLO11 network and a lightweight model to achieve accurate detection of the clamp opening and closing status. By optimizing the YOLO11 network structure and combining it with lightweight model technology, this method not only improves the accuracy and efficiency of detection but also significantly reduces the consumption of computational resources, making the detection system more suitable for real-time, high-efficiency industrial applications.

[0021] As a further improvement, in step one, the explosion-proof monitoring camera SDK is used to read the video footage and save the image in PNG format. In step two, the attention mechanism is coordinate attention (CA). The YOLO11 model consists of three key parts: a backbone network, a feature fusion network, and a prediction decoder network. The backbone network extracts multi-scale features of the image, and the feature fusion network effectively fuses the P3, P4, and P5 scale features extracted by the backbone network to enhance the expressive power and robustness of the features. The prediction decoder network is responsible for decoding the target detection box and the target category to achieve accurate target detection.

[0022] As a further improvement, P3, P4, and P5 extracted by the backbone network correspond to downsampling rates of 1 / 8, 1 / 16, and 1 / 32, respectively, while P2 adopts a fusion network with a downsampling rate of 1 / 4.

[0023] As a further improvement, in step one, the ROI region with the top-left corner coordinates (600,0) and a width and height of 640 is extracted from the original image and labeled. The labels in the image include: open, pattern1, pattern2, close, during, and pipe.

[0024] As a further improvement, in step one, the dataset is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0025] As a further improvement, step two specifically includes the following steps: S201 Modify the configuration model's YAML file, add a P2 scale to the feature fusion network module part of the YAML, fuse feature maps of different scales P3, P4, and P5, and then decode the coordinate position, category, and category confidence of the detected target through the decoding head part; S202 Register the attention mechanism module, add the attention mechanism to the model's YAML file, and add a new CA attention mechanism to the network in the feature fusion part.

[0026] As a further improvement, step three includes: S301: Modify the loss function of the YOLO11 model and add an L1 regularization function as follows: , in, This represents the input and target of the training. Indicates the parameters used in training. This represents the loss function used during normal model training. Represents the sparsity parameter, This represents the regularization strength coefficient; S302: Add the C3K2_pruned, SPPF_pruned, and Detect_pruned modules to the model file in the YOLO11s source code; S303: Train the normal detection model, set the parameter path of the training images, train using the YOLO11s model, and modify the width and depth values ​​of the s model to half of their original values; S304: Sparse training, load the detection model trained in step S303; S305: Model pruning training, load the model weights trained in step S304; S307: Use the torch.export interface function to load the trained pruned model and export the ONNX model, then use the tensorRT tool trtexec to convert the ONNX model into an engine model.

[0027] As a further improvement, step S306 is included after step S305: fine-tuning the pruned model, loading the pruned model file trained in step S305, and setting the training rounds to 200.

[0028] As a further improvement, in step S303, the number of training epochs is set to 1000, the L1 regularization parameter is set to 0, the SGD optimizer function is selected, and the training patience value of the model is 50 epochs; in step S304, the L1 regularization penalty coefficient is set to 0.01, the number of training epochs is set to 200, and the patience value is set to 201; in step S305, the pruning rates are set to 0.1, 0.2, 0.3, and 0.4, respectively.

[0029] As a further improvement, step four includes: S401: Loading the engine model file and obtaining the model detection category; S402: Determining whether the target information of the drill string is detected in the detection category; S403: Determining whether the center point coordinates of the drill string are within the detection frame of the jack; S404: Calculating the distance between pattern1 and pattern2 based on the width of the detected drill string, and normalizing the calculated distance to the distance of the drill string based on the width of the detected drill string; S405: Determining whether the relative distance is lower than threshold 1. If it exceeds threshold 1, the jack is determined to be in the open state; S406: Determining whether the relative distance is lower than threshold 2. If it is lower than threshold 2, the jack is determined to be in the closed state; otherwise, the jack is determined to be in the opening / closing process state.

[0030] This invention provides a method for detecting the opening and closing of a drilling clamp, which will provide strong technical support for the full automation of drilling operations. It is expected to significantly improve the efficiency and automation level of drilling operations, while reducing the errors and costs caused by manual inspection.

[0031] In a preferred embodiment of the present invention, a method for detecting the opening and closing of a hanging card based on an improved YOLO11 and a lightweight model is provided. This method runs on a GPU and includes: Step 1: Use the explosion-proof monitoring camera SDK to read the video footage, then save the images captured by the monitoring camera in PNG format, and clean and label the data. Due to the large field of view of the camera, the position of the gantry in the image changes little; the gantry always performs drill bit clamping operations at a specific position. Therefore, most of the original 1920*1080 resolution images do not need to be detected. To save inference speed for the model, this experiment extracts the ROI region with coordinates (600,0) and width and height of 640 in the upper left corner of the original image for image labeling. The labels in the image include "open", "pattern1", "pattern2", "close", "during", and "pipe". Subsequently, we randomly divide the dataset into training, validation, and test sets in an 8:1:1 ratio. Step two, modify the YOLO11 object detection model framework, specifically including the following steps: a) Modify the configuration model's YAML file, add a P2 scale to the feature fusion network module section of the YAML file, fuse feature maps of different scales P3, P4, and P5, and then decode the target's coordinates, category, and category confidence level through the decoding head.

[0032] The YOLO11 model consists of three key parts: the backbone, the feature fusion network, and the decoder head. The overall model comprises Conv, C3K2, SPPF, C2SPA, Concat, upsample, and Detect modules. The backbone primarily extracts multi-scale features from the image. The feature fusion network effectively fuses the P3, P4, and P5 scale features extracted by the backbone (corresponding to downsampling rates of 1 / 8, 1 / 16, and 1 / 32, respectively) to enhance feature expressiveness and robustness. The decoder head is responsible for decoding the target detection boxes and target categories, achieving accurate target detection. Because the excessively large receptive field of the P3 layer can introduce too much background noise in practical applications, affecting the detection of small targets, a P2 (1 / 4 downsampling rate) fusion network is introduced.

[0033] b) Register the Coordinate Attention (CA) attention mechanism module, add the CA attention mechanism to the yaml file that builds the model, and add the CA attention mechanism to the network in the feature fusion part.

[0034] Step 3: Use Network Slimming to prune and train the model. a) Modify the loss function of the YOLO11 model by adding an L1 regularization function, as shown in the formula: , in, This represents the input and target of the training. Indicates the parameters used in training. This represents the loss function used during normal model training. Represents the sparsity parameter, This represents the regularization intensity coefficient (hyperparameter); b) Add the C3K2_pruned module, SPPF_pruned module, and Detect_pruned module to the model file of the YOLO11s source code; c) Train a normal detection model, set the parameter path of the training image, train using the YOLO11s model, modify the width and depth values ​​of the s model to half of their original values, set the training epochs to 1000, set the L1 regularization parameter to 0, select the SGD optimizer function, and set the training patience value of the model to 50 epochs. d) Sparse training: Load the detection model trained in step c), set the L1 regularization penalty coefficient to 0.01, the number of training rounds to 200, and the patience value to 201; e) Model pruning training, setting pruning rates of 0.1, 0.2, 0.3, and 0.4 respectively, and loading the trained model weights (d). f) Fine-tune the pruned model, load the trained pruned model file (e) and set the training epochs to 200; g) Use the torch.export interface function to load the pruned model trained in f) and export the ONNX model. Then use the tensorRT tool trtexec to convert the ONNX model into an engine model.

[0035] Step 4; Load the detection model to realize the opening and closing status detection of the hanging clamp: a) Load the engine model file and obtain the categories detected by the model; b) Determine whether target information of the drill string has been detected in the detection category; c) Determine whether the center point coordinates of the drill string are within the detection frame of the jack; d) Since the distance between the lens and the jack is not absolutely static during the inspection process, there is a slight change in the distance between the camera and the jack. The distance between pattern1 and pattern2 is calculated based on the width of the drill string being inspected. The distance is then normalized to the distance of the drill string based on the width of the drill string being inspected. e) Determine if the relative distance is below threshold 1. If it exceeds threshold 1, then determine that the hanging card is in the open state. f) Determine if the relative distance is below threshold 2. If it is below threshold 2, determine that the hanger is in a closed state; otherwise, determine that the hanger is in a switching state.

[0036] This invention's technical solution innovatively extracts high-resolution feature maps at the P2 scale (1 / 4 downsampling) from the backbone network and fuses them across layers with multi-scale feature maps at P3, P4, and P5, significantly improving the confidence and localization accuracy of small target detection. This design solves the problem of small target feature loss caused by insufficient feature map resolution in traditional methods. A coordinate attention (CA) mechanism module is embedded at key locations in the feature fusion network to enhance feature representation capabilities through spatial location awareness, enabling the model to accurately capture the edge and corner features of small targets and improve the accuracy of detection box coordinates. A structured pruning method based on Network Slimming is used to optimize the improved model at the channel level, achieving reduced parameters and increased inference speed while maintaining high detection accuracy, meeting the real-time detection needs of industrial scenarios.

[0037] To verify the performance of this invention, this embodiment was conducted on a proprietary hanger dataset. Images were collected, ROI regions were cropped from the images, and processed images were annotated with Labellimg. All training processes in this experiment were performed on the same training set, and all test experiments were performed on the test set. During the experiment, the YOLOv11s model was used as the baseline model, and the following comparative experiments were conducted sequentially: P2-scale feature map fusion was added to the baseline model; based on the newly added P2-scale feature map fusion, a CA attention mechanism module was further added; finally, the model containing the CA attention mechanism module and P2-scale fusion was pruned.

[0038] In the model improvement experiments, the YOLO11s model was used as the benchmark, and the following comparative experiments were conducted sequentially: a new P2-scale feature map fusion was added to the benchmark model; based on the newly added P2-scale feature map fusion, a CA attention mechanism module was further added. The experimental results are shown in Table 1.

[0039] Table 1 Comparison of different improved models on the proprietary dataset

[0040] As shown in the table, compared to the baseline model YOLO11s, the model with the added P2 scale fusion significantly improves the accuracy of target detection, with precision (P) increasing by 2.3%, recall (R) by 1.8%, mAP50 by 3.3%, and mAP50:95 by 2.3%. Further adding the CA attention mechanism module further improves the model's precision (P) by 1.2%, recall (R) by 0.2%, and while mAP50 slightly decreases (by 0.1%), mAP50:95 increases by 0.5%. This indicates that by adding the P2 scale fusion and CA attention mechanism modules, the model achieves a significant improvement in target detection accuracy, particularly in precision and recall.

[0041] Based on the improved model, further pruning was performed to optimize the number of parameters and inference speed. Different pruning rates were applied to the model during the experiment: 10%, 20%, 30%, and 40%. The experimental results are shown in Table 2.

[0042] Table 2 Comparison of models with different pruning rates on our own dataset

[0043] As shown in the table, with the increase of the pruning rate, the number of model parameters gradually decreases and the inference speed gradually increases, but at the same time, mAP50 and mAP50:95 also decrease to varying degrees. At a pruning rate of 40%, the number of model parameters decreases to 5.939M, and the inference speed increases to 7.1ms, but mAP50 decreases to 0.953 and mAP50:95 decreases to 0.905. Considering both model accuracy and inference speed, the model with a pruning rate of 20% achieves a good balance between parameter count, accuracy, and inference speed, with 7.913M parameters, mAP50 of 0.977, mAP50:95 of 0.931, and an inference speed of 8.1ms.

[0044] Through the above experiments, this invention has achieved significant results in both model improvement and pruning optimization. Ultimately, the model of this invention possesses high practicality and efficiency while meeting the requirements of real-time detection tasks.

[0045] It should be understood that the scope of protection sought by this invention is not limited to the non-limiting embodiments, which are merely illustrative examples. The substantive scope of protection claimed in this application is further embodied in the scope provided by the independent claims and their dependent claims.

Claims

1. A method for detecting opening and closing of a lifting clamp, which is run in a GPU, characterized in that: The application relates to a method for detecting the opening and closing states of a hanger clamp. Step one: saving the shooting pictures of a monitoring camera, cleaning and labeling data, and dividing the data into a training set, a verification set and a test set; Step two: modifying a YOLO11 target detection model framework; Step three: pruning a training model by using a Network Slimming method; Step four: loading a detection model to realize the detection of the opening and closing states of a hanger clamp.

2. The method of claim 1, wherein: In the step one, an explosion-proof monitoring camera SDK is used to read a video picture, and the format of a saved picture is png; in the step two, the attention mechanism is a coordinate attention CA, and the YOLO11 model is composed of a backbone network, a feature fusion network and a prediction decoding head network, the backbone network extracts multi-scale features of an image, the feature fusion network effectively fuses P3, P4 and P5 scale features extracted by the backbone network to enhance the expression ability and robustness of the features, and the prediction decoding head network is responsible for decoding a target detection frame and a target category to realize accurate target detection.

3. The method of claim 2, wherein: P3, P4 and P5 extracted by the backbone network correspond to 1 / 8, 1 / 16 and 1 / 32 down-sampling rates respectively, and P2 adopts a 1 / 4 down-sampling rate fusion network.

4. The method of claim 1, wherein: In the step one, the left upper corner coordinate value (600, 0) of an original image is extracted, and a ROI region with a width and a height of 640 is used for image labeling; labels labeled in the image include open, pattern1, pattern2, close, during and pipe.

5. The method of claim 1, wherein: In the step one, the data set is randomly divided into the training set, the verification set and the test set according to a ratio of 8:1:

1.

6. The method of claim 1, wherein: In the step two, the following steps are specifically included: S201: modifying a yaml file of a configuration model, adding a P2 scale in a network module part of the yaml feature fusion, fusing feature maps of different P3, P4 and P5 scales, and then decoding a coordinate position, a category and a category confidence of a detection target through a decoding head part; S202: registering an attention mechanism module, adding the attention mechanism to the yaml file of the configuration model, and adding a CA attention mechanism to the network in a feature fusion part.

7. The method of claim 1, wherein: In the step three, the following steps are included: S301: Modify the loss function of the YOLO11 model, add an L1 regularization function as follows: , wherein, denote the input and target of training, denote the parameters of training, denote the loss function of normal training of the model, denote the sparsity parameters, denote the regularization strength coefficients; S302: adding a C3K2_pruned module, a SPPF_pruned module and a Detect_pruned module in a model file of a YOLO11s source code; S303: training a normal detection model, setting a parameter path of a training image, training by using a YOLO11s model, and modifying a width value and a depth value of the s model to be half of the original values; S304: sparse training, loading the detection model trained in the step S303; S305: model pruning training, loading the model weight trained in the step S304; S307: loading a trained pruning model by using a torch.export interface function to export an onnx model, and then converting the onnx model into an engine model by using a tensorRT tool trtexec.

8. The method of claim 7, wherein: After the S305 step, further comprising S306: fine-tuning the pruned model, loading the pruned model file trained in the S305 step, and setting the training round to 200.

9. The method of claim 8, wherein: In the S303 step, the training round of the model is set to 1000, the L1 regularization term parameter is set to 0, the SGD optimizer function is selected, and the training patience value of the model is 50 rounds; in the S304 step, the L1 regularization penalty coefficient is set to 0.01, the training round is set to 200, and the patience value is set to 201; in the S305 step, the pruning rate is set to 0.1, 0.2, 0.3, and 0.4, respectively.

10. The method of claim 1, wherein: In the fourth step, comprising: S401: loading the engine model file to obtain the detection category of the model; S402: judging whether the target information of the drilling tool is detected in the detection category; S403: judging whether the center point coordinate of the drilling tool is within the detection frame of the elevator; S404: taking the width of the detected drilling tool as a reference to calculate the distance between the pattern1 and the pattern2, and taking the width of the detected drilling tool as a reference to normalize the calculated distance to the distance of the drilling tool; S405: judging whether the relative distance is lower than a threshold value 1, if the relative distance exceeds the threshold value 1, it is determined that the elevator is in an open state; S406: judging whether the relative distance is lower than a threshold value 2, if the relative distance is lower than the threshold value 2, it is determined that the elevator is in a closed state, otherwise it is determined that the elevator is in a process state of opening and closing.

Images