Spatio-Temporal Image Data Generation via Hilbert Scan
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for recognizing moving images using CNNs face challenges such as high calculation costs due to three-dimensional convolution filters and reduced recognition accuracy due to lack of relevance between adjacent pixels in two-dimensional representations.
Innovation Solution
An image data generation device that converts moving-image data into spatio-temporal image data using a Hilbert scan, arranging one-dimensional spatial data in a time series to maintain spatial and temporal information, allowing for two-dimensional filtering and improved recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a three-dimensional convolution filter is used for moving-image data, then recognition accuracy is improved, but calculation cost increases and large-scale calculating machine is required
Solution Approach 1:
The patent transforms three-dimensional moving-image data into two-dimensional image data by arranging pixel values along a space-filling curve (such as Hilbert curve) in one dimension and time frames in another dimension. This dimensional transformation allows the use of computationally efficient two-dimensional convolution filters while preserving spatial and temporal information, thereby reducing calculation cost without significantly compromising recognition accuracy.
2Power
If a two-dimensional convolution filter is used, then calculation cost is reduced, but there is no relevance of information between pixels of images adjacent in a tile shape, and recognition accuracy is reduced
Solution Approach 1:
The patent reorganizes three-dimensional spatio-temporal data into a two-dimensional format using space-filling curves, which preserve the spatial adjacency relationships. This allows two-dimensional convolution filters to capture both spatial and temporal information effectively, maintaining recognition accuracy while reducing computational complexity.
Solution Approach 2:
The patent employs curved space-filling paths (such as Hilbert curves) to scan and rearrange pixel data, rather than simple linear scanning. This curved arrangement maintains the locality and adjacency relationships of pixels in the original three-dimensional space, enabling two-dimensional filters to effectively capture spatial correlations and temporal changes.
3Ease of manufacture
If frame images are arranged in a tile shape, then two-dimensional image processing can be applied, but there is no relevance of information between adjacent pixels, and recognition accuracy is reduced
Solution Approach 1:
The patent uses space-filling curves to arrange pixel values from three-dimensional moving-image data into two-dimensional arrays. This curved scanning approach preserves the spatial adjacency and temporal continuity of pixel information, allowing standard two-dimensional convolution operations to effectively capture both spatial features and temporal dynamics without information loss.
Data Source
AI summary
An image recognition device executes a Hilbert scan of frame image data constituting moving-image data to generate one-dimensional spatial image data, and further arrays the one-dimensional spatial image data in a time direction to generate two-dimensional spatio-temporal image data that holds spatial information and temporal information. The image recognition device converts the moving-image data into the two-dimensional spatio-temporal image data while holding the spatial and temporal information. By means of a CNN unit, the image recognition device executes a convolution process wherein a two-dimensional filter is used on the spatio-temporal image data to image-recognize a behavior of a pedestrian who is a recognition object. The image recognition device executes behavioral recognition of the pedestrian that requires a three-dimensional analysis consisting of two dimensions for space and one dimension for time by the image recognition process of the two-dimensional image by the CNN, and estimates a state of the pedestrian.