Computer Vision Occlusion Robustness Testing for Bin Picking Systems

Overview of Technical Issues:

Physical occlusion in the bin environment blocks the vision detection module from capturing complete object features, while the object recognition algorithm insufficiently identifies partially hidden targets, causing picking failures and reduced system reliability; the goal is to achieve robust detection and successful picking even when objects are 30-70% occluded by neighboring items.

Solution directions generated for this problem

Problem Direction 1 :

ImproveVisible feature completeness
VS
ConstraintDetection system complexity

Inspiration 1 : Cross-domain reference

Application Principle: #26 Copying
Cross-domain applicability Assess applicability
Building data platform with graph based permissions
Innovative Solution Refine solution

Virtual multi-angle reconstruction from single-camera depth data

Single depth camera generates synthetic views
How to solve :
  • Equip existing camera with structured light module (e.g. Intel RealSense D435, Basler ToF) to capture 3D point cloud (depth accuracy ±2mm at 0.5-1.5m range) in single shot (30-60 fps)
  • Apply point cloud rotation algorithm to computationally generate 2-3 virtual viewpoints (±15-30° angular offset from original) by transforming XYZ coordinates, creating synthetic RGB-D images that reveal occluded surfaces without physical camera movement
  • Fuse original and virtual views using weighted feature stitching—assign confidence scores (0.7-1.0) based on surface normal angle relative to each viewpoint, merge feature descriptors (SIFT/ORB keypoints) where overlap >60%, achieving >85% equivalent feature coverage within 0.8-1.2s total processing time
Expected Effect : Feature coverage >85%; no added cameras; processing <1.2s; success rate >92%
Risk Control :
  • depth sensor noise in reflective/dark objects
  • virtual view geometric distortion at sharp occlusion boundaries
  • point cloud density insufficient for small feature details

Problem Direction 2 :

ImproveVisible feature completeness
VS
ConstraintProcessing time per cycle

Inspiration 1 : Cross-domain reference

Application Principle: #10 Preliminary action
Cross-domain applicability Assess applicability
Method for acquiring resource region information for PHICH and method of receiving PDCCH
Innovative Solution Refine solution

Pipelined parallel vision processing with pre-capture during robot motion

Execute vision processing during robot motion
How to solve :
  • Trigger image capture and processing while robot executes previous pick-place motion (1.5-2s travel time) — computation completes before next cycle starts
  • Implement dual-buffer architecture: GPU processes current target features while CPU queues next target, achieving parallel pipeline with <0.3s handoff latency
  • Deploy progressive refinement algorithm — initial 0.5s coarse detection identifies grasp candidates, remaining 1.5s refines pose during robot transit, final confidence threshold ≥92% before grasp execution
Expected Effect : Cycle time maintained at 0.5-0.8s; feature coverage >85%; picking success rate >95%
Risk Control :
  • motion-induced vibration affecting image quality
  • synchronization failure between vision and motion controller
  • buffer overflow under irregular cycle timing

Problem Direction 3 :

ImproveSystem detection reliability
VS
ConstraintDetection system complexity

Inspiration 1 : Cross-domain reference

Application Principle: #11 Beforehand cushioning
Cross-domain applicability Assess applicability
Clothing processing device
Innovative Solution Refine solution

Conditional standby camera activation for occlusion resilience

Deploy standby camera activated only when needed
How to solve :
  • Install a low-cost secondary USB camera (640×480 resolution, <$50) at 45° offset angle to primary camera, normally powered off to avoid permanent complexity
  • Implement confidence-threshold trigger logic: when primary detection confidence score drops below 0.65 (indicating >40% occlusion), automatically activate secondary camera for 2-frame capture (0.3s), fuse point clouds via ICP alignment (tolerance ±2mm), then power off
  • Use lightweight feature fusion algorithm: combine primary edge features with secondary depth map only in flagged occlusion zones (ROI <30% of frame), avoiding full-frame processing—total added cycle time <0.4s only when triggered
Expected Effect : Detection reliability >95%; standby camera active <20% of cycles; system complexity increase minimal
Risk Control :
  • USB camera calibration drift over time
  • confidence threshold requires tuning per object type
  • ICP alignment fails if overlap <40%

Problem Direction 4 :

ImproveSystem detection reliability
VS
ConstraintProcessing time per cycle

Inspiration 1 : Cross-domain reference

Application Principle: #10 Preliminary action
Cross-domain applicability Assess applicability
Radio link failure handling for dual connectivity
Innovative Solution Refine solution

Pipelined parallel detection architecture with pre-computation during robot motion

Overlap detection with robot motion cycles
How to solve :
  • Implement dual-buffer image processing: while robot executes pick-and-place (typical 1.5-2s), GPU pre-processes next target using advanced occlusion-robust algorithm in parallel, eliminating wait time
  • Deploy asynchronous computation pipeline with three stages—image capture (50ms), feature extraction (800ms during robot transit), and final pose refinement (200ms)—total cycle time remains ≤2s matching robot mechanical cycle
  • Install predictive target queuing: vision system analyzes bin overview during previous pick, ranks next 3-5 targets by occlusion severity and accessibility, pre-loads corresponding detection models to RAM for instant switching
Expected Effect : Cycle time maintained at 2s; picking success rate 60-75%→96%; throughput unchanged
Risk Control :
  • synchronization failure between vision and motion controller
  • GPU thermal throttling under continuous load
  • queue prediction accuracy drops in chaotic bins

Problem Direction 5 :

ImproveAlgorithm recognition robustness
VS
ConstraintMust not deteriorate

Inspiration 1 : Cross-domain reference

Application Principle: #15 Dynamics
Cross-domain applicability Assess applicability
Respiratory support system and blower for respiratory support system
Innovative Solution Refine solution

Adaptive dual-mode algorithm with real-time occlusion-triggered switching

Deploy two parallel algorithms that switch based on real-time occlusion assessment
How to solve :
  • Implement a fast occlusion estimator (edge density analysis, 50ms runtime) that evaluates visible feature ratio in each frame
  • if occlusion <40%, invoke lightweight template matching (0.5s cycle)
  • if occlusion ≥40%, activate deep learning partial-object model (2.0s cycle)
  • Use a confidence threshold gate (0.75 score) where low-confidence fast results automatically trigger the robust algorithm, ensuring reliability without always incurring time penalty
  • Pre-load both algorithm models in GPU memory with hot-swap execution—occlusion estimator runs continuously at 20Hz, decision latency <100ms, enabling seamless mode transition within single pick cycle
Expected Effect : Average cycle time 0.8s (60% cases fast mode); picking success rate >95% at 30-70% occlusion; throughput increase +40% vs always-on deep learning
Risk Control :
  • occlusion estimator false negatives causing mode selection errors
  • GPU memory contention between dual models reducing switching speed
  • confidence threshold calibration drift across object variety
Patsnap Eureka Solution