Crosswalk Detection Using Multi-Camera Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for creating autonomous vehicle (AV) maps manually require extensive time and resources, as they rely on human operators to accurately identify and draw crosswalks, which can be incomplete, inaccurate, and inefficient, posing safety risks and consuming significant network and processing resources.
Innovation Solution
A computer-implemented method using a computing system with processors to receive image data of roadways, generate characteristics such as classification, segmentation, and angles of elements, determine crosswalk positions, and provide map data, employing a convolutional neural network trained to optimize segmentation, alignment, and boundary losses for accurate crosswalk detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual methods are used to identify and draw crosswalks in AV maps, then human operators can accurately identify crosswalk locations, but the process consumes extensive time and resources
Solution Approach 1:
The patent replaces the manual mechanical process of human operators identifying and drawing crosswalks with an automated computer vision system using convolutional neural networks. The system processes images from multiple cameras to automatically detect crosswalk boundaries, positions, and orientations, eliminating the need for manual map creation while maintaining high accuracy through multi-image correlation and structured prediction algorithms.
2Productivity
If manual operators create AV maps, then crosswalk positions can be identified, but the process is incomplete and inaccurate
Solution Approach 1:
The patent merges data from multiple image sources (front-facing, rear-facing, and side-facing cameras) to create a comprehensive view of the environment. By combining images captured at different angles and positions, the system correlates features across multiple images to accurately identify crosswalk boundaries and positions, improving both completeness and accuracy compared to single-image or manual methods.
Solution Approach 2:
The system employs structured prediction with loss functions that provide feedback during the crosswalk detection process. The neural network is trained to minimize prediction errors by comparing detected crosswalk features against ground truth data, continuously improving accuracy through iterative optimization of segmentation, alignment, and boundary detection.
3Quantity of substance
If extensive manual resources are allocated to map creation, then more crosswalks can be identified, but network and processing resources are significantly consumed
Solution Approach 1:
The patent performs preliminary processing of images by capturing and storing multiple images from different cameras and angles before crosswalk detection is needed. This pre-captured image data is then correlated and processed using neural networks to identify crosswalks, reducing the computational burden during real-time operation and minimizing network resource consumption during critical detection phases.
Data Source
AI summary
A method includes receiving image data associated with an image of a roadway including a crosswalk, generating a plurality of different characteristics of the image based on the image data, determining a position of the crosswalk on the roadway based on the plurality of different characteristics, the position including a first boundary and a second boundary of the crosswalk in the roadway, and providing map data associated with a map of the roadway, the map data including the position of the crosswalk on the roadway in the map. The plurality of different characteristics include a classification of one or more elements of the image, a segmentation of the one or more elements of the image, and one or more angles of the one or more elements of the image with respect to a line in the roadway.