Remote Assist Traffic Light Recognition for Automated Drive Vehicles

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Automated driving systems face challenges in accurately recognizing provisional traffic lights, leading to potential communication delays and unsafe vehicle operations due to low recognition accuracy, especially in construction zones or private properties.

Innovation Solution

A remote assist system that includes a vehicle controller and a remote facility, where the vehicle temporarily stops and transmits image information to the remote facility for operator-assisted recognition of traffic lights, providing positional information of the traffic light pixels to improve recognition accuracy and ensure safe passage based on real-time traffic light instructions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the vehicle requests recognition assist to a remote facility when traffic light recognition accuracy is low, then the recognition accuracy of traffic light instructions is improved, but communication delay occurs between the vehicle and remote facility

Engineering Contradiction:
Improverecognition accuracy of traffic light instructionsVSAvoidcommunication delay
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The vehicle controller executes a determination process to judge whether passage is permitted based on traffic light information received before the actual passage timing. This preliminary determination allows the vehicle to prepare for passage in advance, compensating for the communication delay by ensuring the traffic light state is assessed prior to the critical passage moment.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system continuously monitors traffic light information and compares it with the determined passage permission status. The controller receives updated traffic light information and re-evaluates passage permission based on current conditions, creating a feedback loop that adapts to changing traffic light states and ensures safe passage timing.

Inventive Principle:
Principle #23Feedback

2Measurement precision

If the vehicle waits for passage permitting signal from remote facility before proceeding, then recognition accuracy of traffic light instructions is improved, but the vehicle may miss the optimal passage timing due to communication delay

Engineering Contradiction:
Improverecognition accuracy of traffic light instructionsVSAvoidvehicle passage timing
Core Design Contradiction:
Measurement precisionVSSpeed

Solution Approach 1:

The vehicle controller performs preliminary determination of passage permission using traffic light information received in advance. This allows the vehicle to identify suitable passage timing opportunities before the actual passage occurs, eliminating the need to wait for remote facility confirmation at the last moment and preventing missed passage opportunities.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts the passage determination process by continuously receiving traffic light information and re-evaluating passage permission. The controller can adapt its passage timing decisions based on real-time traffic light state changes, allowing flexible response to dynamic traffic conditions while maintaining accurate recognition.

Inventive Principle:
Principle #15Dynamics

3Adaptability or versatility

If the vehicle uses camera images to recognize traffic lights in construction zones or private properties, then the vehicle can navigate these areas, but recognition accuracy is low due to difficulty in grasping traffic light location on map

Engineering Contradiction:
Improveability to navigate construction zones and private propertiesVSAvoidrecognition accuracy of traffic light instructions
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The vehicle controller acts as an intermediary that receives traffic light information through communication means and processes it locally. This intermediary processing allows the vehicle to accurately interpret traffic light instructions even when the physical location of traffic lights in construction zones or private properties is not clearly marked on maps, bridging the gap between ambiguous visual information and reliable navigation decisions.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The controller executes preliminary determination of passage permission using received traffic light information before the vehicle reaches the traffic light. This advance processing compensates for the lack of precise location data by ensuring the traffic light state is evaluated and passage permission is determined in advance, improving recognition accuracy in areas with unclear traffic light positioning.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS11891089B2Remote assist system and remote assist method of automated drive vehicles
Publication Date: 2024.02.06 TOYOTA JIDOSHA KK
  • US11891089B2 patent drawing
  • US11891089B2 patent drawing
  • US11891089B2 patent drawing

AI summary

A vehicle determines whether or not a passage permit condition of the vehicle according to an instruction content of a traffic light in front of the vehicle is satisfied. If it is determined that the passage permit condition is not satisfied, the vehicle temporally stops in front of the traffic light and transmits an assist requiring signal to a remote facility. A display device of the remote facility displays reference image information indicating shot image information of the vehicle that temporally stops in front of the traffic light. An operator of the remote facility inputs positional information of pixels that make up the traffic light contained in the reference image information. The remote facility transmits assist information including the positional information to the vehicle. Based on the positional information, the vehicle determines again whether or not the passage permit condition is satisfied.