Convolutional Neural Network Temporal Sequence Processing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Convolutional neural networks (CNNs) are not configured to process time-dependent data sequences effectively, failing to explicitly recognize temporal dependencies in data, which is crucial for robust semantic analysis, especially in complex scenes like traffic environments where objects move and occlude each other.

Innovation Solution

A CNN device and method that processes input sequences by generating an output sequence through recursive processing, using a sampling unit to sample from past output data and a grid-generation sequence based on input and intermediate sequences, allowing for weighted combination to account for dynamic changes and temporal dependencies, thereby enhancing the recognition of time-dependent information.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If ordinary CNNs process input sequences sequentially by processing one data item after the other, then the processing structure remains simple and spatial patterns are preserved, but time-dependencies in the data are not explicitly recognized

Engineering Contradiction:
Improverecognition of time-dependenciesVSAvoidprocessing structure complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent introduces a temporal dimension to the traditional CNN architecture by adding a time axis to the data processing. The input data is organized as a sequence of data items over time, and the CNN processes this extended dimensional structure, allowing temporal dependencies to be captured through convolution operations across the time dimension while maintaining the spatial processing capabilities of traditional CNNs.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The patent segments the processing into distinct functional components: a data generation unit that creates synthetic training data with known time-dependencies, a training unit that trains the CNN on this segmented data, and an evaluation unit that tests the model. This segmentation allows the complex problem of temporal dependency recognition to be solved through structured, modular processing steps.

Inventive Principle:
Principle #1Segmentation

2Measurement precision

If more temporal information is used to carry out robust semantic analysis of complex scenes, then object detection accuracy improves, but processing time and computational resources increase

Engineering Contradiction:
Improveobject detection accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent generates synthetic training data in advance with known ground truth labels and temporal relationships. This preliminary preparation of training data with embedded time-dependencies allows the CNN to learn temporal patterns efficiently during training, reducing the computational burden during actual object detection while maintaining high accuracy.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent creates synthetic copies of training scenes with controlled temporal variations and known object trajectories. These copied and modified training examples allow the model to learn robust temporal patterns without requiring extensive real-world data collection and processing, thereby reducing processing time while improving detection accuracy.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS11804026B2Device and a method for processing data sequences using a convolutional neural network
Publication Date: 2023.10.31 APTIV TECHNOLOGIES AG
  • US11804026B2 patent drawing
  • US11804026B2 patent drawing
  • US11804026B2 patent drawing

AI summary

A device for processing data sequences by means of a convolutional neural network is configured to carry out the following steps: receiving an input sequence comprising a plurality of data items captured over time using a sensor, each of said data items comprising a multi-dimensional representation of a scene, generating an output sequence representing the input sequence processed item-wise by the convolutional neural network, wherein generating the output sequence comprises: generating a grid-generation sequence based on a combination of the input sequence and an intermediate grid-generation sequence representing a past portion of the output sequence or the grid-generation sequence, generating a sampling grid on the basis of the grid-generation sequence, generating an intermediate output sequence by sampling from the past portion of the output sequence according to the sampling grid, and generating the output sequence based on a weighted combination of the intermediate output sequence and the input sequence.