A speech cognition evaluation method based on eye movement

An ocular and speech technology used in the field of cognitive and language ability assessment, which can solve the problems that individuals with developmental defects cannot complete

Active Publication Date: 2019-01-15
ZHEJIANG UNIV
View PDF8 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Before using this method, it is necessary to instruct the subject to fix the eye tracker by gazing at a specific position on the screen, and some individuals with developmental defects cannot complete this task
Although it is assumed that the subject's fixation on the picture is spontaneous, it may still be affected by the content of the picture and external factors

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A speech cognition evaluation method based on eye movement
  • A speech cognition evaluation method based on eye movement
  • A speech cognition evaluation method based on eye movement

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0080] In this embodiment, the EOG electrodes are used to capture electrooculogram data generated by eye movements when the subject is listening to the voice. The EOG measurement experiment results are used to illustrate that the synchronous tracking of language structures can be observed using the eye movement data measured by EOG. It should be noted that the existing studies have shown that the oculoelectricity measured by EOG is mainly caused by eye movement, that is, the electric potential change obtained by EOG reflects the movement of eyeballs.

[0081] Step S1:

[0082] The experiment uses 3 kinds of corpus, the experimental corpus is as follows figure 2 As shown, each corpus consists of 48 words. Corpus 1 contains two-character words, Corpus 2 contains two-character words and four-character sentences, and Corpus 3 consists of random words without words and sentences. In the experiment, 30 paragraphs of each of the above three corpora were used.

[0083] Step S2: ...

Embodiment 2

[0108] In this embodiment, an eye tracker is used to capture eye movement data when the subject is listening to the voice. The experimental results of the eye tracker measurement in this embodiment are used to illustrate that the synchronous tracking of the language structure can be observed by using the eye movement data measured by the eye tracker.

[0109] The idea and data processing method of the experimental measurement are similar to those of Example 1, the only difference is that the EOG data used in the EOG measurement experiment of Example 1 is used as eye movement data, while in this example, the eye tracker is used to measure Eye movement data and blink data are substituted as eye movement data. The specific experimental process is as follows:

[0110] Step S1:

[0111] The experimental corpus used figure 2 The corpus 2 and corpus 3 in the two corpora, the experimental corpus is as follows figure 2 As shown, each corpus consists of 48 words. Corpus 2 contain...

Embodiment 3

[0140] Each segment of speech in Embodiment 3 is composed of a plurality of corpus with the same language structure, similar difficulty and the same number of words, and the presentation time of each word of the corpus is 250ms. The method of eye tracker is used to record eye movements, and the data of blinking for analysis. Each segment of speech in Example 4 is composed of multiple corpus with arbitrary language structure, variable difficulty, and arbitrary number of words. Each word in the corpus has an arbitrary presentation time. The EOG method is used to record eye movements and analyze the oculoelectric data. Relatively speaking, the voice of Example 4 is more inclined to the voice heard in real life. Since the corpus of each segment of speech in Embodiment 3 and Embodiment 4 has different characteristics, different data processing methods are adopted in these two cases.

[0141] Example 3

[0142] In this embodiment, Fourier transform is used to judge whether there i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a speech cognition evaluation method based on eye movement. The method comprises the following steps: S1, each segment of speech used for speech cognition evaluation is presented to the subject through the auditory channel of the tester in turn, and eye movement data of the subject is synchronously recorded during the time of presenting speech stimulation, with the eye movement data being eye movement data and blink data when the subject receives speech stimulation; S2, processing the eye movement data and the speech corresponding to the synchronization of the data, andidentifying whether there is an index characteristic synchronized with the language structure in the speech in the eye movement data. If it exists, it is considered that the subject can understand the language structure of the speech; if it does not exist, it is considered that the subject cannot understand the language structure of the speech. The method adopts the auditory sense as the input ofthe perceptual channel, relates to less advanced cognitive system in the information processing process, and has a wider user population.

Description

technical field [0001] The invention belongs to the field of cognition and language ability evaluation, in particular to a speech cognition evaluation method based on eye movement. Background technique [0002] Cognitive and language ability assessment is an important evaluation standard for individual cognitive level and ability development, and is widely used in daily life. With the continuous expansion of applications, language cognitive assessment methods are also constantly developing. Teachers or evaluators can get a preliminary evaluation result by talking with the testee, but this result is easily affected by subjective factors. At present, the most commonly used language tests are mainly carried out in the form of "test questions", that is, to set corresponding questions according to the content of the test, and the testees complete the questions, judge the answers of the testees, and determine the score of the answers. language test scores. The traditional langu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): A61B5/16A61B3/113
CPCA61B3/113A61B5/16
Inventor 丁鼐金培清
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap