Construction Equipment Camera Fusion for Obstructed Front View
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Construction machinery, such as wheel loaders and excavators, face visibility limitations due to work apparatuses like booms and buckets, which can obstruct the operator's view and lead to safety accidents.
Innovation Solution
A control system with dual cameras (upper and lower) captures images from different angles, synthesizes them, and applies transparency processing to enhance visibility by highlighting potential collision hazards with colored bounding boxes or blinking images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a work apparatus (boom, bucket) is used for excavation and loading work, then productivity is improved, but the operator's front view is obstructed leading to safety hazards
Solution Approach 1:
A camera is installed on the work apparatus to capture images of the area obscured by the boom and bucket. The camera acts as an intermediary device that transfers visual information from the blocked front view to the display unit, allowing the operator to see obstacles without direct line of sight. This resolves the contradiction by maintaining productivity (work apparatus in position) while eliminating the harmful effect (view obstruction) through the intermediary camera system.
Solution Approach 2:
The system transitions from direct optical viewing (2D plane) to a multi-dimensional information space by capturing images through a camera and processing them through multiple image processing units. The obscured area is visualized in a different dimensional space (display screen) that can be viewed from the operator's position without physical line of sight, effectively adding a spatial dimension to the viewing capability.
2Object-affected harmful factors
If the work apparatus is positioned to improve visibility, then safety is improved, but the ability to perform excavation and loading work is limited
Solution Approach 1:
The camera mounted on the work apparatus serves as an intermediary that allows the work apparatus to maintain its optimal working position (which may partially obstruct view) while simultaneously providing visual information about the obscured area. The camera captures what the operator cannot see directly, enabling the apparatus to be positioned for maximum productivity without compromising safety monitoring capability.
Solution Approach 2:
Instead of moving the work apparatus to improve visibility, the system creates a visual copy (image) of the obscured area through the camera. This copy is then displayed to the operator, allowing the original work apparatus position to be maintained for productivity while the safety information is replicated and presented in an accessible format on the display unit.
3Measurement precision
If multiple cameras are installed to capture comprehensive views, then measurement precision and object detection are improved, but device complexity increases
Solution Approach 1:
The field of view is segmented into multiple zones, with different cameras assigned to capture specific areas (e.g., one camera for the area obscured by the boom, another for the area obscured by the bucket). Each camera focuses on a particular segment, improving detection precision for that zone. The segmentation allows comprehensive coverage without requiring a single complex camera system, as each camera can be optimized for its specific viewing segment.
Solution Approach 2:
Multiple images from different cameras are merged and synthesized by the image processing units to create a comprehensive view. The synthesis process combines the segmented views into a unified representation that maintains high detection precision while managing system complexity through integrated processing. The merging of data from multiple sources achieves superior object recognition without requiring each individual camera to be overly complex.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A control system for construction machinery includes an upper camera installed in a driver cabin of a rear body to capture a front view of the driver cabin, a lower camera installed in a front body rotatably connected to the rear body to capture a front view of the front body, an object recognizer configured to recognize an object in first and second images captured by the upper camera and the lower camera and to determine warning information according to a position of the recognized object, an image processor configured to synthesize the first and second images into one image, and perform transparency processing on at least one of the first and second images within a transparency processing area of the synthesized image and labeling processing on the warning information of the object, and a display device configured to display the synthesized image labeled with the warning information.