Rig State Estimation Using Computer Vision and Sensor Fusion

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Modern drilling rigs face challenges in aggregating data from numerous sensors to determine the large-scale rig state, which is crucial for safe and efficient operations, due to the complexity of interacting human behaviors and the inability of individual sensors to provide a complete picture, leading to inefficiencies in time & motion studies and manual video analysis.

Innovation Solution

An automated computer vision system that uses machine learning algorithms to estimate rig state from video data, incorporating human detection and tracking, and integrates with sensor data to provide real-time, annotated video feeds and improve decision-making and automation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If automated computer vision systems are implemented to estimate rig state, then productivity and measurement precision are improved, but device complexity increases

Engineering Contradiction:
Improverig state estimation speedVSAvoidsystem complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent replaces manual human observation and mechanical sensing systems with automated computer vision technology. The system uses video cameras and machine learning algorithms to automatically detect and track rig state, eliminating the need for manual video analysis and reducing reliance on traditional sensor arrays. This substitution directly improves productivity by enabling continuous automated monitoring while managing system complexity through software-based solutions.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system creates visual copies of the rig environment through video cameras and processes these optical copies to estimate rig state. Instead of directly measuring physical parameters with sensors, the system captures visual information and uses computer vision algorithms to infer operational state. This approach improves measurement precision for complex rig behaviors while avoiding the complexity of installing and integrating numerous specialized sensors.

Inventive Principle:
Principle #26Copying

2Measurement precision

If multiple sensors are deployed to measure various rig parameters, then measurement precision for individual parameters is improved, but the ability to determine overall rig state deteriorates due to data aggregation complexity

Engineering Contradiction:
Improveindividual parameter measurement accuracyVSAvoidbig-picture rig state information
Core Design Contradiction:
Measurement precisionVSLoss of information

Solution Approach 1:

The computer vision system performs multiple functions simultaneously: it monitors safety conditions, tracks equipment position, observes human activities, and determines overall rig state from a single integrated system. This multi-functional approach replaces the need for multiple specialized sensors and their complex aggregation, as the vision system naturally captures holistic information about rig operations that individual sensors cannot provide.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent merges multiple data sources including video feeds from various cameras, sensor data, and operational logs into a unified rig state estimation system. By combining these diverse information streams through machine learning algorithms, the system recovers the big-picture rig state information that was lost when relying on individual sensor readings in isolation, effectively reversing the information loss problem.

Inventive Principle:
Principle #5Merging (Combining)

3Measurement precision

If manual video analysis is used for time and motion studies, then detailed observation is possible, but loss of time increases due to the manual effort required

Engineering Contradiction:
Improvetime and motion analysis accuracyVSAvoidvideo analysis time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The system enables self-service automated analysis where the computer vision system independently performs time and motion studies without requiring manual intervention. The machine learning algorithms automatically process video data, identify operations of interest, and generate analytics, allowing the system to serve its own analysis needs without human effort. This eliminates the time loss associated with manual video review while maintaining or improving analysis accuracy through consistent automated evaluation.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS12529288B2Systems and methods for estimating rig state using computer vision
Publication Date: 2026.01.20 HELMERICH & PAYNE TECH LLC
  • US12529288B2 patent drawing
  • US12529288B2 patent drawing
  • US12529288B2 patent drawing

AI summary

The invention relates to a system and method for estimating global rig state. The system comprises a model incorporating multiple variables related to rig state, at least one camera operably connected to at least one processor wherein said camera is capable of gathering visual data regarding at least one variable of rig state and said processor is capable of compiling rig state data, estimating global rig state, or both. The system further comprises multiple sensors for measuring variables related to global rig state wherein said sensors are operably connected to said processor. The method comprises sensing various aspects of the rig state, collecting visual data corresponding with said sensor data, compiling multiple sources of rig data, and estimating the overall rig state.