Sound Image Generation Using Harmonics for ML Fault Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for determining device sound abnormalities do not effectively image sound data, limiting the generation of accurate machine learning models.

Innovation Solution

An image processing apparatus and method that converts sound data into visual representations, such as heat maps, using fundamental frequencies and harmonics, enabling the generation of machine learning models for normality and abnormality determination.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If sound data is quantified by physical quantities (sound pressure, frequency), then machine learning models can be generated, but the sound data cannot be effectively imaged

Engineering Contradiction:

Improvesound data quantification accuracyVSAvoidsound data visualization capability

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent transforms one-dimensional sound data (time series) into two-dimensional images by mapping frequency components to spatial positions. The vertical axis represents frequency (fundamental frequency and harmonics) while the horizontal axis represents time, creating a spectrogram-like visualization that preserves both temporal and spectral information simultaneously.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The patent introduces an intermediary representation layer between raw sound data and machine learning models. This intermediate form is a visualized spectrogram that converts acoustic signals into an image domain, serving as a bridge that enables both human interpretation and machine processing of sound characteristics.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If sound data is converted into images, then visualization and machine learning model generation are enabled, but the processing complexity increases

Engineering Contradiction:

Improvemachine learning model compatibilityVSAvoidimage processing apparatus complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments the sound data into distinct frequency components (fundamental frequency and harmonic components) before visualization. This segmentation allows each frequency band to be processed and displayed independently, simplifying the overall processing algorithm while enabling comprehensive spectral analysis.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the parameter representation from acoustic parameters (pressure, frequency in Hz) to spatial parameters (pixel position, intensity). By transforming sound data into an image format with standardized pixel dimensions and intensity values, the system becomes compatible with existing machine learning frameworks while maintaining the essential acoustic information.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentEP4109058B1Image processing apparatus and image processing method

Publication Date: 2026.04.08 NISSAN MOTOR CO LTD

Figure 1
Figure 2A
Figure 2B

AI summary

An image processing apparatus (1) includes a controller (20). The controller (20) calculates a fundamental frequency component included in sound data and a harmonic component corresponding to the fundamental frequency component, converts the fundamental frequency component and the harmonic component into image data, and generates a sound image where the fundamental frequency component and the harmonic component converted into the image data are arranged adjacent each other.