Mobile Image Monitoring for Earthmoving Action Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Efficiently monitoring and managing activities across large earthmoving sites is challenging due to the size of the sites, the variety of heavy equipment, and the dynamic nature of the environment, with existing stationary image-based systems requiring multiple devices and repositioning to cover broader areas.
Innovation Solution
Implementing an image capturing system with mobile devices fixed to earthmoving machines that perform computer vision and machine learning to identify objects and actions, allowing selective image transmission to a remote server for further processing, thereby reducing the need for multiple stationary devices and enhancing adaptability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If multiple stationary image capturing devices are deployed to cover large earthmoving sites, then the monitoring coverage area is improved, but the device complexity and cost increase
Solution Approach 1:
The patent applies the dynamics principle by transitioning from stationary image capturing devices to mobile devices mounted on earthmoving equipment. The mobile image capturing device moves with the equipment throughout the work site, dynamically expanding the monitoring coverage area without requiring multiple fixed devices. This resolves the contradiction by achieving broad coverage with a single device that adapts its position based on equipment movement.
Solution Approach 2:
The patent applies universality by making the image capturing device multi-functional through integration with earthmoving equipment. The same device serves multiple purposes: capturing images for monitoring, tracking equipment location, and adapting to different work zones as the equipment moves. This eliminates the need for dedicated stationary monitoring devices at each location.
2Adaptability or versatility
If stationary image capturing devices are repositioned to cover different areas, then the monitoring flexibility is improved, but the time loss and operational disruption increase
Solution Approach 1:
The patent resolves this contradiction by making the monitoring system dynamic through integration with mobile earthmoving equipment. Instead of repositioning stationary devices, the image capturing device moves automatically with the equipment, providing continuous adaptability to different work areas without interruption or time loss associated with device relocation.
Solution Approach 2:
The patent applies preliminary action by pre-mounting the image capturing device on the earthmoving equipment before it enters the work site. This preliminary positioning ensures the device is already in the correct location and orientation to capture images from the start of operation, eliminating any time loss that would occur if devices needed to be repositioned during operations.
3Measurement precision
If image streams are transmitted to remote servers for processing, then the measurement precision of actions is improved, but the loss of information and data management complexity increase
Solution Approach 1:
The patent applies the extraction principle by extracting only the essential information from the complete image stream for transmission to the remote server. Instead of transmitting raw image data, the system extracts key frames or processed data representing equipment actions and location, reducing data volume while maintaining measurement precision for action recognition.
Solution Approach 2:
The patent applies preliminary action by performing initial image processing and action identification locally at the mobile device before transmission to the remote server. This preliminary processing filters and prepares data for transmission, reducing the information loss that would occur if only raw streams were transmitted, while still maintaining the precision needed for accurate action recognition.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques for orchestrating activities at an earthmoving site are described. In an example, a stream of images are captured by an image capturing device removably fixed to a mobile earthmoving machine with a field of view overlapping with a working range of an attachment coupled to the mobile earthmoving machine. Using a selected subset of images from the stream captured during a time period, a first trained algorithm generates a probability that the subset of images depicts an action performable by the machine. Based on the probability, it is determined that the machine was performing the action during the time period, and in response, an image from the subset of images is selected. The image and an indication that the machine was performing the action during the time period are transmitted to a remote server system through a network.