Colored audio and video guide method and colored audio and video guide device

Abstract

Provide a color audio-visual guide method and a color audio-visual guide device, which can calculate the main frequency of the color light wave, color saturation and brightness of each color pixel according to the collected real-time color image data, and then convert it into the corresponding The main frequency of the sound wave, the signal-to-noise ratio of the sound wave, and the loudness of the sound element point, combined with the three-dimensional position information of the color pixel point, generate a pulsed binaural stereo sound wave signal, and output the sound image information through the shellless binaural earphone to guide the blind. The invention can enable blind people to recognize the color, shape and size of three-dimensional objects without training, and the recognition resolution is high.

Description

Technical field: [0001] The invention relates to a method and a device for converting image information into sound information applied in the field of blind guidance. Background technique: [0002] At present, two schemes have been proposed in the field to provide information about surrounding objects for the blind, ie guide sticks and visual substitutes. [0003] Guide stick: Its equipment mainly relies on infrared or ultrasonic reflection to warn the blind of nearby objects, and the blind can judge the distance of obstacles through the sound and tactile information from the guide stick. This solution relies on instruments to identify obstacles and then express them to the blind. Due to the limited information conveyed, the blind can hardly directly recognize information such as the shape, size, and color of objects. [0004] Visual substitution: There are mainly two types of tactile-visual substitution and audio-visual substitution. [0005] Tactile-visual substitution i...

AI Technical Summary

Problems solved by technology

[0006] 1. Due to the stimulation of the electrode array, this method will cause discomfort to the fingers or tongue, which will affect health

[0007] 2. Due to the limited array area accommodated by the tongue and fingertips, and the resolution limitation of sensory cells, the spatial resolution is low

In fact, after long-term training, the recognition rate of blind people using this device for simple-shaped objects is still lower than 30%, so it has not been widely used.

[0011] 2. Low spatial resolution:

[0012] a. The frequency resolution of hearing limits the resolution of the sound image in the y direction

[0013] b. Do not do a lot of segmentation in the x direction, otherwise the scanning time will be too long and the update rate of dynamic pictures will be too low

[0014] c. It is difficult for blind people to distinguish "the y direction of the frequency code and the x direction of the time block scanning code", so generally the resolution of the sound and image of this method can only be 64x64

[0015] 3. Low time resolution: Time block scanning is required in the x direction. In order to ensure that the 64-block segmentation and the encoded sound signal are not distorted, the horizontal scanning time is required to be greater than 1 second, that is, the minimum scanning time is 1 second / frame

[0016] 4. Due to the limitation of the principle of the method, the color of the object cannot be recognized

[0017] 5. This method can only be used for 2D image-to-sound conversion, not for 3D object recognition

[0018] 6. It is very difficult for the human brain to do inverse transformation of sound and image, so this method is rarely used

[0020] 1. Block division is required, resulting in low spatial resolution

Generally only 10x10 resolution can be achieved

[0021] 2. Blocks need to be scanned point by point, resulting in low time resolution

[0022] 3. Limited to the expression of musical instruments, that is to say, it can only be expressed by the limited number of keys on the piano keyboard

[0023] 4. Color resolution is extremely low: only 12 colors can be distinguished

Due to the table method and piano keyboard expression, this method cannot achieve continuous color expression

[0024] 5. The resolution of color saturation is extremely low: only a few pitches can be expressed

[0025] 6. Blind people have difficulty adapting to unnatural sound transitions

[0026] 7. This method cannot recognize 3D objects

Images