A listening and matching ability evaluation system and a storage medium

Abstract

The present invention discloses an auditory matching ability evaluation system and a storage medium, relating to the technical field of cognitive evaluation. The system includes: an information storage and audio output system for storing and playing statement audios in different signal-to-noise ratio environments of different test tasks; a behavioral test module for, during the playback of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system, statistically collecting in real time the reaction time and correct rate of the subject in discriminating each test task; a pupillometer for, during the playback of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system, collecting in real time the pupil information of the subject during the execution of each test task; an electroencephalogram system for, during the playback of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system, collecting in real time the electroencephalogram information of the subject during the execution of each test task; and an auditory matching ability evaluation module for evaluating the auditory matching ability of the subject by using the pupil information, electroencephalogram information of each test task, and the reaction time and correct rate of discriminating the test task.

Description

Technical Field

[0001] The present invention relates to the technical field of cognitive assessment, and particularly relates to a listening effort assessment system and a storage medium. Background Art

[0002] Listening effort refers to the degree of effort exerted by people in mobilizing various cognitive resources, such as attention and working memory, subjectively during the process of performing auditory tasks. The main measurement methods of listening effort can be divided into three categories: behavioral tests, physiological tests, and questionnaire evaluations and scale tests.

[0003] Behavioral tests mainly focus on three dimensions: working memory, attention, and speed of processing, which are interrelated. First, the capacity of working memory is limited, but it can allocate capacity for processing and storing information when performing complex tasks (such as language understanding or listening during multitasking). Second, almost all cognitive activities involve attention during the process of allocating capacity, such as selective attention when performing a single task or divided attention when performing multitasks. Assuming that the memory capacity requirement or attention required for a task continuously increases, then the task processing speed will also decrease, and the error rate of task execution will increase. In short, within this framework, researchers can evolve existing experimental paradigms or models according to the dimensions to be evaluated when designing listening effort behavioral tests.

[0004] Physiological indicators must meet three conditions: (1) they must be sensitive to changes in processing load within a task; (2) they must be sensitive to changes in processing load between tasks; (3) they must be sensitive to differences in processing load among individuals. The physiological test methods of listening effort are mainly based on two aspects, namely brain activity and autonomic nervous system excitation when performing tasks. Tests based on brain activity include event-related potentials (ERPs), electroencephalograms (EEGs), and functional magnetic resonance imaging (fMRI). ERPs and EEGs have excellent temporal resolution, while fMRI can provide more accurate spatial information. Tests based on autonomic nervous system excitation mainly focus on pupil diameter, cardiac response, skin conductance, and hormone levels.

[0005] The degree of effort in listening is a subjective feeling, and many hearing-impaired people who come to the outpatient clinic will also actively report their self-feelings during the listening process. Therefore, researchers can judge the effort level, fatigue level, etc. of patients during listening through questionnaire evaluations and scale tests. Summary of the Invention

[0006] The present invention provides an auditory allocation ability evaluation system and a storage medium, so as to solve the technical problem of how to improve the evaluation accuracy of the auditory allocation ability of a subject.

[0007] An embodiment of the present invention provides an auditory allocation ability evaluation system, including:

[0008] An information storage and audio output system, configured to store and play statement audios in different signal-to-noise ratio environments of different test tasks;

[0009] A behavior test module, configured to, during the playing of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system, statistically record in real time the reaction time and the correct rate of the subject's discrimination of each test task;

[0010] A pupilometer, configured to, during the playing of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system, collect in real time the pupil information of the subject during the execution of each test task;

[0011] An electroencephalogram system, configured to, during the playing of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system, collect in real time the electroencephalogram information of the subject during the execution of each test task;

[0012] An auditory allocation ability evaluation module, configured to evaluate the auditory allocation ability of the subject by using the pupil information, electroencephalogram information of each test task, and the reaction time and correct rate of discriminating the test task.

[0013] Preferably, the test tasks include: a selective attention test task, a processing speed test task, and a working memory test task; the signal-to-noise ratios include 10 dB, 5 dB, 0 dB, -5 dB, -10 dB; wherein, the information storage and audio output system includes:

[0014] A selective attention test task database, which stores first structured short sentence audios for analyzing the allocation of auditory cognitive resources in the state of the selective attention test task and including object types, object colors, and object quantities;

[0015] A processing speed test task database, which stores second structured short sentence audios for analyzing the allocation of auditory cognitive resources in the state of the processing speed test task and including digital operation information;

[0016] A working memory test task database, which stores third structured short sentence audios for analyzing the allocation of auditory cognitive resources in the state of the working memory test task and including multiple groups of five theme information;

[0017] A noise database, configured to store noise audios and form different signal-to-noise ratio environments by adjusting the intensity of the noise audios; and

[0018] An audio output device for simultaneously outputting and playing the first structured short sentence audio or the second structured short sentence audio or the third structured short sentence audio and the adjusted noise audio.

[0019] Preferably, the pupillometer is specifically configured to, during the playback of the sentence audio in each signal-to-noise ratio environment of the selective attention test task by the information storage and audio output system, collect in real time the first pupil information of the subject during the execution of the selective attention test task; during the playback of the sentence audio in each signal-to-noise ratio environment of the processing speed test task by the information storage and audio output system, collect in real time the second pupil information of the subject during the execution of the processing speed test task; and during the playback of the sentence audio in each signal-to-noise ratio environment of the working memory test task by the information storage and audio output system, collect in real time the third pupil information of the subject during the execution of the working memory test task.

[0020] Preferably, the electroencephalogram system is specifically configured to, during the playback of the sentence audio in each signal-to-noise ratio environment of the selective attention test task by the information storage and audio output system, collect in real time the first electroencephalogram information of the subject during the execution of the selective attention test task; during the playback of the sentence audio in each signal-to-noise ratio environment of the processing speed test task by the information storage and audio output system, collect in real time the second electroencephalogram information of the subject during the execution of the processing speed test task; and during the playback of the sentence audio in each signal-to-noise ratio environment of the working memory test task by the information storage and audio output system, collect in real time the third electroencephalogram information of the subject during the execution of the working memory test task.

[0021] Preferably, the listening ability evaluation module includes:

[0022] A first evaluation sub-module for obtaining the first reaction time and the first correct rate of the selective attention test task after the subject hears the sentence audio in each signal-to-noise ratio environment of the selective attention test task, and using the first pupil information, the first electroencephalogram information, and the first reaction time and the first correct rate of the selective attention test task to evaluate the selective attention listening ability of the subject;

[0023] A second evaluation sub-module for obtaining the second reaction time and the second correct rate of the processing speed test task after the subject hears the sentence audio in each signal-to-noise ratio environment of the processing speed test task, and using the second pupil information, the second electroencephalogram information, and the second reaction time and the second correct rate of the processing speed test task to evaluate the processing speed listening ability of the subject;

[0024] The third evaluation sub-module is used to obtain the third response time and the third correct rate of the working memory test task after the subject hears the statement audio in each signal-to-noise ratio environment of the working memory test task, and use the third pupil information, the third electroencephalogram information, the third response time and the third correct rate of the working memory test task to evaluate the auditory matching ability of the subject;

[0025] The fourth evaluation sub-module is used to evaluate the change rate of the auditory matching ability of the subject according to the change rate of the selective attention auditory matching ability, the change rate of the processing speed auditory matching ability and the change rate of the working memory auditory matching ability of the subject;

[0026] The fifth evaluation sub-module is used to evaluate the signal-to-noise ratio that can reflect the auditory matching ability level of the subject according to the correct rate corresponding to the analysis of the allocation of auditory cognitive resources in the task state under different signal-to-noise ratio environments;

[0027] The evaluation sub-module is used to evaluate the auditory matching ability of the subject by using the selective attention auditory matching ability, the processing speed auditory matching ability, the working memory auditory matching ability, the change rate of the auditory matching ability and the signal-to-noise ratio of the subject.

[0028] Preferably, the evaluation sub-module specifically includes:

[0029]

[0030] w 1 +w 2 +w 3 +w 4 +w 5 =1

[0031] where LES i refers to the auditory matching ability of subject i; the refers to the selective attention auditory matching ability of subject i in the j-th signal-to-noise ratio environment; the refers to the processing speed auditory matching ability of subject i in the j-th signal-to-noise ratio environment; the refers to the working memory auditory matching ability of subject i in the j-th signal-to-noise ratio environment; the refers to the change rate of the auditory matching ability of subject i; the refers to the signal-to-noise ratio that can reflect the auditory matching ability level of subject i; w 1 、w 2 、w 3 、

[0032] w 4 、w 5 are weights.

[0033] Preferably, the first evaluation sub-module specifically includes:

[0034] Selective attention listening ability = First reaction time * First correct rate + First pupil information + First EEG information;

[0035] Among them, the first pupil information = First pupil diameter peak value * First average degree of pupil diameter expansion;

[0036] The first EEG information = First EEG wave amplitude.

[0037] Preferably, the second evaluation sub-module specifically includes:

[0038] Processing speed listening ability = Second reaction time * Second correct rate + Second pupil information + Second EEG information;

[0039] Among them, the second pupil information = Second pupil diameter peak value * Second average degree of pupil diameter expansion;

[0040] The second EEG information = Second EEG wave amplitude.

[0041] Preferably, the third evaluation sub-module specifically includes:

[0042] Working memory listening ability = Third reaction time * Third correct rate + Third pupil information + Third EEG information;

[0043] Among them, the third pupil information = Third pupil diameter peak value * Third average degree of pupil diameter expansion;

[0044] The third EEG information = Third EEG wave amplitude.

[0045] An embodiment of the present invention provides a storage medium, on which a program is stored, and when the program is executed by a processor, the following steps are implemented:

[0046] Use the information storage and audio output system to store and play statement audios in different signal-to-noise ratio environments of different test tasks;

[0047] During the playback of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system, the reaction time and correct rate of the subject's discrimination of each test task are statistically analyzed in real time, and at the same time, the pupil information of the subject during the execution of each test task is collected in real time by a pupil measuring instrument, and the EEG information of the subject during the execution of each test task is collected in real time by an EEG system;

[0048] Use the pupil information, EEG information of each test task, and the reaction time and correct rate of the discrimination test task to evaluate the listening ability of the subject.

[0049] The beneficial effects of the present invention are as follows: By jointly evaluating the listening adaptation ability of a subject through behavioral tests and physiological tests, the electroencephalogram and pupil of the subject are continuously monitored during different types of behavioral tests, making the test evaluation content more comprehensive and the results more accurate, and greatly improving the clinical efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a schematic diagram of a listening adaptation ability evaluation system provided by the present invention;

[0051] Figure 2 is provided by the present invention Figure 1 specific schematic diagram of the listening adaptation ability evaluation module in;

[0052] Figure 3 is a schematic diagram of pupil data preprocessing provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] It should be understood that the specific embodiments described herein are merely used to explain the present invention and are not used to limit the present invention. In the following description, suffixes such as "module", "component", or "unit" used to represent elements are only for the convenience of explaining the present invention and have no specific meaning of their own. Therefore, "module", "component", or "unit" can be used interchangeably.

[0054] Figure 1 is a schematic diagram of a listening adaptation ability evaluation system provided by the present invention, as Figure 1 shown, including: an information storage and audio output system for storing and playing statement audios in different signal-to-noise ratio environments of different test tasks; a behavioral test module for statistically analyzing the reaction time and correct rate of the subject's discrimination of each test task in real time during the playback of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system; a pupil measuring instrument for collecting the pupil information of the subject during the execution of each test task in real time during the playback of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system; an electroencephalogram system for collecting the electroencephalogram information of the subject during the execution of each test task in real time during the playback of the statement audios in each signal-to-noise ratio environment of each test task by the information storage and audio output system; a listening adaptation ability evaluation module for evaluating the listening adaptation ability of the subject by using the pupil information, electroencephalogram information, reaction time, and correct rate of discriminating the test task of each test task.

[0055] Among them, the test tasks include: selective attention test tasks, processing speed test tasks, and working memory test tasks; the signal-to-noise ratios include 10 dB, 5 dB, 0 dB, -5 dB, -10 dB; among them, the information storage and audio output system includes: a selective attention test task database, which stores the first structured short sentence audio for analyzing the allocation of auditory cognitive resources in the state of selective attention test tasks and including object types, object colors, and object quantities; a processing speed test task database, which stores the second structured short sentence audio for analyzing the allocation of auditory cognitive resources in the state of processing speed test tasks and including digital operation information; a working memory test task database, which stores the third structured short sentence audio for analyzing the allocation of auditory cognitive resources in the state of working memory test tasks and including multiple groups of five theme information; a noise database, which is used to store noise audio and forms different signal-to-noise ratio environments by adjusting the intensity of the noise audio; and an audio output device, which is used to simultaneously output and play the first structured short sentence audio or the second structured short sentence audio or the third structured short sentence audio and the adjusted noise audio.

[0056] Specifically, the pupillometer is specifically used to collect the first pupil information of the subject during the execution of the selective attention test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the selective attention test task by the information storage and audio output system; collect the second pupil information of the subject during the execution of the processing speed test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the processing speed test task by the information storage and audio output system; and collect the third pupil information of the subject during the execution of the working memory test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the working memory test task by the information storage and audio output system.

[0057] Specifically, the electroencephalogram system is specifically used to collect the first electroencephalogram information of the subject during the execution of the selective attention test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the selective attention test task by the information storage and audio output system; collect the second electroencephalogram information of the subject during the execution of the processing speed test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the processing speed test task by the information storage and audio output system; and collect the third electroencephalogram information of the subject during the execution of the working memory test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the working memory test task by the information storage and audio output system.

[0058] Specifically, the listening and matching ability evaluation module includes: a first evaluation sub-module, configured to obtain the statement audio of the selective attention test task for the subject in each signal-to-noise ratio environment, determine the first response time and the first correct rate of the selective attention test task, and evaluate the selective attention listening and matching ability of the subject by using the first pupil information, the first electroencephalogram information, the first response time and the first correct rate of the selective attention test task; a second evaluation sub-module, configured to obtain the statement audio of the processing speed test task for the subject in each signal-to-noise ratio environment, determine the second response time and the second correct rate of the processing speed test task, and evaluate the processing speed listening and matching ability of the subject by using the second pupil information, the second electroencephalogram information, the second response time and the second correct rate of the processing speed test task; a third evaluation sub-module, configured to obtain the statement audio of the working memory test task for the subject in each signal-to-noise ratio environment, determine the third response time and the third correct rate of the working memory test task, and evaluate the working memory listening and matching ability of the subject by using the third pupil information, the third electroencephalogram information, the third response time and the third correct rate of the working memory test task; a fourth evaluation sub-module, configured to evaluate the change rate of the listening and matching ability of the subject according to the change rate of the selective attention listening and matching ability, the change rate of the processing speed listening and matching ability, and the change rate of the working memory listening and matching ability of the subject; a fifth evaluation sub-module, configured to evaluate the signal-to-noise ratio that can reflect the listening and matching ability level of the subject according to the correct rate corresponding to the analysis of the allocation of auditory cognitive resources in the task state under different signal-to-noise ratio environments; an evaluation sub-module, configured to evaluate the listening and matching ability of the subject by using the selective attention listening and matching ability, the processing speed listening and matching ability, the working memory listening and matching ability, the change rate of the listening and matching ability, and the signal-to-noise ratio of the subject.

[0059] Furthermore, the evaluation sub-module specifically includes:

[0060]

[0061] w 1 +w 2 +w 3 +w 4 +w 5 =1

[0062] wherein, LES i refers to the listening and matching ability of subject i; the refers to the selective attention listening and matching ability of subject i in the jth signal-to-noise ratio environment; the refers to the processing speed listening and matching ability of subject i in the jth signal-to-noise ratio environment; the refers to the working memory listening and matching ability of subject i in the jth signal-to-noise ratio environment; the refers to the change rate of the listening and matching ability of subject i; the refers to the signal-to-noise ratio that can reflect the listening and matching ability level of the subject i; w 1 、w 2 、w 3 、w 4 、w 5 are weights.

[0063] More specifically, the first evaluation sub-module specifically includes: Selective attention listening and matching ability = First reaction time * First correct rate + First pupil information + First electroencephalogram information; wherein, the first pupil information = First pupil diameter peak * First average degree of pupil diameter expansion; the first electroencephalogram information = First electroencephalogram amplitude.

[0064] More specifically, the second evaluation sub-module specifically includes: Processing speed listening and matching ability = Second reaction time * Second correct rate + Second pupil information + Second electroencephalogram information; wherein, the second pupil information = Second pupil diameter peak * Second average degree of pupil diameter expansion; the second electroencephalogram information = Second electroencephalogram amplitude.

[0065] More specifically, the third evaluation sub-module specifically includes: Working memory listening and matching ability = Third reaction time * Third correct rate + Third pupil information + Third electroencephalogram information; wherein, the third pupil information = Third pupil diameter peak * Third average degree of pupil diameter expansion; the third electroencephalogram information = Third electroencephalogram amplitude.

[0066] An embodiment of the present invention provides a storage medium, on which a program is stored, and when the program is executed by a processor, the following steps are implemented: using an information storage and audio output system to store and play statement audios in different signal-to-noise ratio environments of different test tasks; during the information storage and audio output system playing the statement audios in each signal-to-noise ratio environment of each test task, statistically collecting in real time the reaction time and correct rate of the subject to judge each test task, and at the same time using a pupil measuring instrument to collect in real time the pupil information of the subject when performing each test task, and using an electroencephalogram system to collect in real time the electroencephalogram information of the subject when performing each test task; using the pupil information, electroencephalogram information of each test task, and the reaction time and correct rate of judging the test task to evaluate the listening and matching ability of the subject.

[0067] The present invention combines traditional behavioral tests and physiological tests to jointly evaluate the listening adaptation ability of patients. In the physiological test, a pupillometer is mainly used to measure the changes in the pupils of the subjects wearing the pupillometer during the execution of auditory tasks. The main indicators for recording pupil changes include the baseline level of pupil diameter, peak pupil dilation (PPD), and mean dilation (MD). In addition, an electroencephalogram (EEG) system is also worn by the subjects to record real-time EEG responses and observe the EEG amplitudes and latencies at different times.

[0068] The listening adaptation ability evaluation system in the present invention includes four devices: a laptop computer, a speaker, a pupil recording instrument, and an EEG device. The corpus is automatically played by computer software programmed with Matlab, which randomly plays the corpus for different testers to avoid the repetition effect. The corpus is jointly recorded by professional male and female voice actors of Boeing in a standard recording studio. The speaker amplifies the sound source played by the computer, the pupil recorder is worn on the eyes of the subjects for recording, and the EEG device is worn on the heads of the subjects through the EEG system. The subjects sit on a chair one meter away from the speaker, facing the speaker. The subjects answer according to the content of the speaker. That is to say, an in-house compiled corpus for auditory tests is adopted. The corpus is automatically played by the computer, and the sound is amplified by the speaker and played to the subjects. After hearing the voice, the subjects answer according to different test tasks. When answering, the pupil changes and EEG changes of the subjects are recorded, so as to realize the simultaneous scoring of behavioral tests and physiological tests and comprehensive scoring.

[0069] The listening adaptation ability test includes three tests: selective attention test, processing speed test, and working memory test. For these three tests, in-house compiled corpora are used. To avoid the learning effect, the corpora for attention and working memory are recorded by professional male and female voice actors respectively, and the corpus for processing speed has the content recorded by male voice actors and the questions recorded by female voice actors. Each test is carried out at five different signal-to-noise ratios. For the 100 sentences of materials in the selective attention test, at each of the five different signal-to-noise ratios, 20 sentences are tested at each signal-to-noise ratio; for the processing speed test, there are 10 sentences in each test type, and 2 sentences are tested at each of the five different signal-to-noise ratios; for the working memory test, every 5 sentences form a group, and 5 groups are carried out at 5 different signal-to-noise ratios respectively. The sound intensity of the target sentence is 65 dB SPL, and the background noise is Babble noise. The signal-to-noise ratios (SNR) are (10, 5, 0, -5, -10)

[0070] The selective attention test requires the subjects to say the type of object according to the just heard voice. One hundred sentences are randomly played each time, and the reaction time and accuracy rate are recorded.

[0071] Examples of the sentence materials for the selective attention test (balls) are as follows:

[0072] Prepare 1 blue basketball. Prepare 2 blue basketballs.

[0073] Prepare 1 blue volleyball. Prepare 2 blue volleyballs.

[0074] Prepare 1 blue football. Prepare 2 blue footballs.

[0075] Prepare 1 blue tennis ball. Prepare 2 blue tennis balls.

[0076] Prepare 1 red basketball. Prepare 2 red basketballs.

[0077] Prepare 1 red volleyball. Prepare 2 red volleyballs.

[0078] Prepare 1 red football. Prepare 2 red footballs.

[0079] Prepare 1 red tennis ball. Prepare 2 red tennis balls.

[0080] Prepare 1 green basketball. Prepare 2 green basketballs.

[0081] Prepare 1 green volleyball. Prepare 2 green volleyballs.

[0082] Prepare 1 green football. Prepare 2 green footballs.

[0083] Prepare 1 green tennis ball. Prepare 2 green tennis balls.

[0084] ……

[0085] Prepare 9 green basketballs. Prepare 1 green basketball.

[0086] Prepare 9 green volleyballs. Prepare 1 green volleyball.

[0087] Prepare 9 green footballs. Prepare 1 green football.

[0088] Prepare 9 green tennis balls. Prepare 1 green tennis ball.

[0089] Prepare 9 yellow basketballs. Prepare 1 yellow basketball.

[0090] Prepare 9 yellow volleyballs. Prepare 1 yellow volleyball.

[0091] Prepare 9 yellow footballs. Prepare 1 yellow football.

[0092] Prepare 9 yellow tennis balls. Prepare 1 yellow tennis ball.

[0093] Prepare 9 black basketballs. Prepare 1 black basketball.

[0094] Prepare 9 black volleyballs. Prepare 1 black volleyball.

[0095] Prepare 9 black footballs. Prepare 1 black football.

[0096] Prepare 9 black tennis balls. Prepare 1 black tennis ball.

[0097] The corpus of the selective attention test mainly changes the sentence types by varying the types of objects, the quantities of objects, and the colors of objects. Taking balls as an example, the types of balls include: basketballs, volleyballs, footballs, and tennis balls, a total of four types;

[0098] The colors of the balls include: blue, red, green, yellow, and black, a total of five colors;

[0099] The quantities of the balls include: 1, 2, 3, 4, 5, 6, 7, 8, 9, a total of 9 types (to keep the length of the sentences consistent, the quantities are controlled within single digits).

[0100] The test rules for processing speed mainly require the subjects to perform addition and subtraction within 50 according to the voice content, and record the reaction time and accuracy rate.

[0101] The corpus for the processing speed test includes three types of conversations: life-related, study-related, and interesting. Perform addition and subtraction within 50 according to the conversation content. By changing the language scenarios or the quantities of items, the structure of the same sentence can be changed. By changing the question-and-answer method of the sentence, each sentence can be transformed into two forms: addition and subtraction.

[0102] Examples of the sentence materials for processing speed are as follows:

[0103] a) Life-related

[0104] 1. Man: Xiaoli has 25 stamps, which is 15 more than Xiaohong.

[0105] Woman: How many stamps does Xiaohong have?

[0106] 2. Man: Xiaohua's family has 7 apples, and the number of bananas is 3 less than that of apples.

[0107] Woman: How many bananas does Xiaohua's family have?

[0108] 3. Man: Xiaoming has 38 yuan of lucky money, and Xiaohong has 36 yuan of lucky money.

[0109] Woman: How much less money does Xiaohong have than Xiaoming?

[0110] 4. Man: Xiaoming is 11 years old this year, and his father is 27 years older than Xiaoming.

[0111] Woman: How old is the father this year?

[0112] 5. Man: There are 45 red and yellow flowers in the flower shop, and there are 26 red flowers.

[0113] Female: How many yellow flowers are there?

[0114] 6. Male: There are 36 apples in Xiaoqiang's family, and 17 have been eaten.

[0115] Female: How many are left?

[0116] 7. Male: Xiaoming bought 17 red flowers and then bought 15 yellow flowers.

[0117] Female: How many flowers are there in total? 32

[0118] 8. Male: Mom bought 15 apples, and the number of oranges she bought is 6 less than the number of apples.

[0119] Female: How many oranges did Mom buy? 9

[0120] 9. Male: There are 17 ducks in the river and 15 ducks on the shore.

[0121] Female: How many ducks are there in total?

[0122] 10. Male: There are 16 little birds in the tree, and 11 little birds flew away.

[0123] Female: How many little birds are left in the tree?

[0124] ……

[0125] b) Learning category

[0126] 1. Male: Xiaoming read 21 pages of a comic book in the morning and 15 pages in the afternoon. Female: How many pages of the comic book did Xiaoming read in total?

[0127] 2. Male: The school has 16 footballs and 18 basketballs.

[0128] Female: How many fewer footballs are there than basketballs?

[0129] 3. Male: Xiaodong did 10 math problems in the morning, and did 13 more problems in the afternoon than in the morning. Female: How many problems did Xiaodong do in total?

[0130] 4. Male: The learning-from-Lei Feng group repaired 18 chairs in the morning and 19 chairs in the afternoon. Female: How many chairs were repaired in a day?

[0131] 5. Male: The storybook has 74 pages. Xiaoli read 20 pages on the first day and 23 pages on the second day.

[0132] Female: How many pages are left unread?

[0133] 6. Male: The school originally had 25 boxes of chalk and used 11 boxes.

[0134] Female: How many boxes of chalk are left now?

[0135] 7. Man: Xiaoming did math calculation problems. He got 15 right and 3 wrong.

[0136] Woman: How many problems did Xiaoming do in total?

[0137] 8. Man: Xiaoli bought a box of 36 - color watercolor pens and gave 18 to Xiaohua.

[0138] Woman: How many watercolor pens does Xiaoli have left?

[0139] 9. Man: Xiaoli and Xiaohua had a shuttlecock - kicking competition. Xiaohua kicked 16 and Xiaoli kicked 21.

[0140] Woman: How many fewer did Xiaohua kick than Xiaoli?

[0141] 10. Man: Xiaoming and Xiaodong got 32 points and 27 points respectively in the math exam.

[0142] Woman: How many more points did Xiaoming get than Xiaodong?

[0143] ……

[0144] c) Fun - type

[0145] 1. Man: The little rabbit ate 26 carrots and the little pig ate 17.

[0146] Woman: How many more carrots did the little pig eat than the little rabbit?

[0147] 2. Man: The little rabbit ate 26 carrots and the little pig ate 11 fewer.

[0148] Woman: How many carrots did the little pig eat in total?

[0149] 3. Man: Xiaoming bought 7 red flowers and bought the same number of yellow flowers.

[0150] Woman: How many pots of flowers are there in total?

[0151] 4. Man: There were originally 16 little dogs in the pet store. 6 left first and then 7 left.

[0152] Woman: How many are there now?

[0153] 5. Man: The little rabbit and the little pig ate 40 carrots in total and the little pig ate 15.

[0154] Woman: How many carrots did the little rabbit eat?

[0155] 6. Man: Xiaoming ate 12 candies and Xiaojun ate 15.

[0156] Woman: How many more candies did Xiaojun eat than Xiaoming?

[0157] 7. Man: Xiaoming and Xiaoli made paper - cuttings together. Xiaoming made 17 and Xiaoli made 12.

[0158] Female: How many more did Xiaoming do than Xiaoli?

[0159] 8. Male: Xiaoming and Xiaojun ate a total of 38 candies. Xiaoming ate 23 candies.

[0160] Female: How many candies did Xiaojun eat?

[0161] 9. Male: When Xiaoming was doing paper-cutting, he used 18 pieces of red paper and another 12 pieces of green paper.

[0162] Female: How many pieces of paper did he use in total?

[0163] 10. Male: There are 33 cars in total at the bus station. 13 cars drove away.

[0164] Female: How many cars are left at the bus station?

[0165] ……

[0166] There are 40 sentences for each type of corpus. Each time during the test, 10 sentences of each type are randomly played, and the reaction time and accuracy rate are calculated.

[0167] For working memory, the subject needs to continuously listen to 5 sentences. Each sentence consists of 11 characters and is about common life scenarios. The last two characters form a meaningful word, including a verb and a noun. The corpus is changed by replacing the last word of the sentence with a synonym. After the playback ends, the subject is required to say the last word after each sentence according to the voice content.

[0168] Examples of working memory sentence materials are as follows:

[0169] 1. The scorching heat in summer makes people feel irritable.

[0170] 2. I saw Professor Zhang running on the playground.

[0171] 3. There is a goldfish under the lotus leaves in the pool.

[0172] 4. Grandpa always goes for a walk in the park after meals.

[0173] 5. He can skip dinner in order to lose weight.

[0174] 6. Mom said we would go to have hot pot tonight.

[0175] 7. Remember to bring the keys and an umbrella before going out.

[0176] 8. Li Hua will go to take part in a math competition tomorrow.

[0177] 9. His father is the logistics director of No. 4 Middle School.

[0178] 10. Oranges are suitable for growing in southern cities.

[0179] ……

[0180] There are 30 sentences in the corpus of working memory, and six groups of tests need to be carried out. During each test, the sentences in each group will be randomly combined, and the reaction time and accuracy rate will be counted.

[0181] When conducting selective attention tests, processing speed tests, and working memory tests on the subjects respectively, the behavioral test results under different signal-to-noise ratios are statistically analyzed, mainly including reaction time and accuracy rate.

[0182] When conducting selective attention tests, processing speed tests, and working memory tests on the subjects respectively, the pupil changes under different signal-to-noise ratios are statistically analyzed. The indicators for recording pupil changes mainly include the baseline level of pupil diameter, peak pupil dilation (PPD), and mean dilation (MD). Calculating the baseline level too small will relatively increase the degree of pupil dilation, and vice versa will underestimate the degree of pupil dilation. Therefore, the calculation of the baseline level is particularly important. The baseline levels before all stimuli are added together and the average value is taken as the pupil baseline. This value is used as the benchmark when calculating PPD or MP thereafter.

[0183] Since the initially obtained pupil data in the experiment is relatively messy, it must be preprocessed before analyzing the visualized data specifically. Open the Matlab software and clean the pupil data. The brief steps are as Figure 3 shown. Pupil data preprocessing steps: 1 - Find abnormal data and separate the data; 2 - Eliminate abnormal data such as blinks and artifacts; 3 - Perform linear interpolation; 4 - Perform filtering and curve smoothing.

[0184] When conducting selective attention tests, processing speed tests, and working memory tests on the subjects respectively, the electroencephalogram changes under different signal-to-noise ratios are statistically analyzed. The amplitude and latency of P300 are recorded. Event-related potentials can reflect the time-phase information of neural activities and stimulus information. In auditory event-related potentials, the amplitude of the P300 component is closely related to the listener's attention and effort. P3a is a positive wave recorded at scalp electrodes, with a latency of 250 - 280 ms and the largest amplitude when the electrode is located in the middle of the forehead, reflecting the unconscious attention shift and the process of the brain's processing of novel stimuli. During speech audiometry in noise, as the signal-to-noise ratio decreases, both young and old people will show an increase in the amplitude of P3a.

[0185] Listening allocation ability (practical example): In a loud or noisy environment, the behavioral test results may be the same. The reason may be that in a noisy environment, attention is more concentrated, more cognitive resources need to be allocated for listening, more effort is required than in a loud environment, and at this time, eye movements are more frequent and electroencephalograms are more active.

[0186] Therefore, this system jointly evaluates the listening ability of the subject through behavioral tests and physiological tests. During different types of behavioral tests, the electroencephalogram and pupil of the subject are monitored at all times, making the test evaluation content more comprehensive and the results more accurate, greatly improving the clinical efficiency.

[0187] The following specifically introduces the specific calculation processes of the behavioral test and the physiological test:

[0188] 1. Evaluation function LES of the behavioral test a (Listening Effort Score, listening ability score) establishment

[0189] This evaluation includes the average time and accuracy rate of answering questions.

[0190] The level of listening ability reflects an individual's performance and effort in an auditory task. Therefore, the longer the average time for the subject to answer questions, the higher the listening ability. The product of the accuracy rate and the average time for answering questions is more suitable as an indicator of listening ability.

[0191]

[0192]

[0193]

[0194]

[0195] 2. Evaluation function LES of pupil information in the physiological test b establishment

[0196] This evaluation includes pupil change indicators.

[0197] The peak value of pupil diameter and the average degree of dilation reflect the influence of factors such as the subject's attention, cognitive load, and emotional state on the auditory task. A larger pupil peak is usually related to a higher cognitive load and a higher attention level, and may show a higher listening ability; the average degree of dilation refers to the average value of the pupil change degree during the entire task, which reflects the subject's overall response to the stimuli in the task. A larger average degree of dilation may mean that the subject is more focused when performing the task, and thus may show a higher listening ability.

[0198]

[0199]

[0200] Symbol Meaning <![CDATA[d ij > Pupil diameter of the i-th subject at the j-th moment <![CDATA[D i > Baseline level of pupil diameter of the i-th subject T Total time <![CDATA[PPD i > Peak pupil diameter of the i-th subject <![CDATA[MD i > Average degree of pupil dilation of the i-th subject

[0201] 3. Evaluation function LES of electroencephalogram information in the physiological testc Establishment

[0202] This evaluation includes electroencephalogram amplitude.

[0203] The amplitude of the electroencephalogram waveform can provide information about the specific cognitive processes involved when the subject is performing an auditory task. Generally, people with higher auditory matching ability will show a greater electroencephalogram amplitude when performing an auditory task.

[0204]

[0205] Symbol Meaning <![CDATA[amplitude i > Brain wave amplitude of the i-th subject

[0206] 4. Evaluation function Establishment

[0207] This evaluation includes the change rate at low signal-to-noise ratio compared to high signal-to-noise ratio of.

[0208]

[0209]

[0210]

[0211]

[0212]

[0213] 5. Evaluation function Establishment

[0214] This evaluation includes the signal-to-noise ratio dropping to the 50% discriminable level (SNR50). The signal-to-noise ratio corresponding to the correct rate dropping to a certain extent (such as 50%) can, to a certain extent, reflect the auditory matching ability level.

[0215]

[0216] accuracy i,high -accuracy i,α ≥50%

[0217] SNR50 = max(α)

[0218] Symbol Meaning <![CDATA[accuracy i,high > Accuracy rate of the i-th subject at the highest signal-to-noise ratio <![CDATA[accuracy i,α > Accuracy rate of the i-th subject at the α signal-to-noise ratio

[0219] Simulation calculation:

[0220] Assumed data:

[0221]

[0222]

[0223] Then the accuracy i,high = accuracy i,3 = 98%, when the signal-to-noise ratio α is 0.5 or 0.0, it satisfies accuracy i,high - accuracy i,α ≥ 50% So the value range of the signal-to-noise ratio α is the set {0.5, 0.0}. Finally, take the maximum value 0.5, then SNR50 is 0.5.

[0224] Ultimately, the listening and matching ability level can be regarded as

[0225]

[0226] w 1 + w 2 + w 3 + w 4 + w 5 = 1

[0227] The weights of each index need to be determined according to the actual situation and can be obtained through data analysis and other means.

[0228]

[0229] In summary, the present invention has the following advantages: combining behavioral tests and electrophysiological tests to more comprehensively evaluate the listening and matching ability. And the behavioral tests are divided into three types, all of which are self-compiled corpora, making the evaluation content richer, the evaluation results more accurate, and the evaluation efficiency better.

[0230] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings. The scope of the present invention is not limited thereby. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the present invention shall fall within the scope of the present invention.

Claims

1. An auditory allocation ability evaluation system, characterized in that, it includes: An information storage and audio output system for storing and playing statement audios in different signal-to-noise ratio environments of different test tasks; The test tasks include: selective attention test tasks, processing speed test tasks, and working memory test tasks; A behavioral test module for statistically recording the reaction time and accuracy rate of the subject's discrimination of each test task in real time during the playback of the statement audio of each signal-to-noise ratio environment of each test task by the information storage and audio output system; A pupillometer for collecting the pupil information of the subject during the execution of each test task in real time during the playback of the statement audio of each signal-to-noise ratio environment of each test task by the information storage and audio output system; An electroencephalogram system for collecting the electroencephalogram information of the subject during the execution of each test task in real time during the playback of the statement audio of each signal-to-noise ratio environment of each test task by the information storage and audio output system; An auditory allocation ability evaluation module for evaluating the auditory allocation ability of the subject by using the pupil information, electroencephalogram information, reaction time, and accuracy rate of discriminating the test task of each test task; Among them, the auditory allocation ability evaluation module includes: A first evaluation sub-module for obtaining the first reaction time and the first accuracy rate of the subject's discrimination of the selective attention test task after hearing the statement audio of each signal-to-noise ratio environment of the selective attention test task, and using the first pupil information, the first electroencephalogram information, the first reaction time, and the first accuracy rate of the selective attention test task to evaluate the selective attention auditory allocation ability of the subject; A second evaluation sub-module for obtaining the second reaction time and the second accuracy rate of the subject's discrimination of the processing speed test task after hearing the statement audio of each signal-to-noise ratio environment of the processing speed test task, and using the second pupil information, the second electroencephalogram information, the second reaction time, and the second accuracy rate of the processing speed test task to evaluate the processing speed auditory allocation ability of the subject; A third evaluation sub-module for obtaining the third reaction time and the third accuracy rate of the subject's discrimination of the working memory test task after hearing the statement audio of each signal-to-noise ratio environment of the working memory test task, and using the third pupil information, the third electroencephalogram information, the third reaction time, and the third accuracy rate of the working memory test task to evaluate the working memory auditory allocation ability of the subject; A fourth evaluation sub-module for evaluating the change rate of the subject's auditory allocation ability according to the change rates of the subject's selective attention auditory allocation ability, processing speed auditory allocation ability, and working memory auditory allocation ability; A fifth evaluation sub-module for evaluating the signal-to-noise ratio that can reflect the auditory allocation ability level of the subject according to the accuracy rate corresponding to the analysis of the allocation of auditory cognitive resources in the task state under different signal-to-noise ratio environments; An evaluation sub-module for evaluating the auditory allocation ability of the subject by using the selective attention auditory allocation ability, processing speed auditory allocation ability, working memory auditory allocation ability, change rate of auditory allocation ability, and signal-to-noise ratio of the subject.

2. The system according to claim 1, characterized in that, The signal-to-noise ratios include 10 dB, 5 dB, 0 dB, -5 dB, -10 dB; wherein, the information storage and audio output system includes: A selective attention test task database, which stores first structured short sentence audios for analyzing the allocation of auditory cognitive resources in the state of selective attention test tasks and including object types, object colors, and object quantities; A processing speed test task database, which stores second structured short sentence audios for analyzing the allocation of auditory cognitive resources in the state of processing speed test tasks and including digital operation information; A working memory test task database, which stores third structured short sentence audios for analyzing the allocation of auditory cognitive resources in the state of working memory test tasks and including multiple groups of five theme information; A noise database, which is used to store noise audios and forms different signal-to-noise ratio environments by adjusting the intensity of the noise audios; and An audio output device, which is used to simultaneously output and play the first structured short sentence audio or the second structured short sentence audio or the third structured short sentence audio and the adjusted noise audio.

3. The system according to claim 2, wherein, the pupilometer is specifically used to collect the first pupil information of the subject during the execution of the selective attention test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the selective attention test task by the information storage and audio output system; collect the second pupil information of the subject during the execution of the processing speed test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the processing speed test task by the information storage and audio output system; and collect the third pupil information of the subject during the execution of the working memory test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the working memory test task by the information storage and audio output system.

4. The system according to claim 3, wherein, the electroencephalogram system is specifically used to collect the first electroencephalogram information of the subject during the execution of the selective attention test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the selective attention test task by the information storage and audio output system; collect the second electroencephalogram information of the subject during the execution of the processing speed test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the processing speed test task by the information storage and audio output system; and collect the third electroencephalogram information of the subject during the execution of the working memory test task in real time during the playback of the sentence audio in each signal-to-noise ratio environment of the working memory test task by the information storage and audio output system.

5. The system according to claim 4, wherein, the evaluation sub-module specifically includes: Among them, refers to the listening ability of subject i; the refers to the selective attention listening ability of subject i in the jth signal-to-noise ratio environment; the refers to the processing speed listening ability of subject i in the jth signal-to-noise ratio environment; the refers to the working memory listening ability of subject i in the jth signal-to-noise ratio environment; the refers to the change rate of the listening ability of subject i; the refers to the signal-to-noise ratio that can reflect the listening ability level of the subject i; is the weight.

6. The system according to claim 5, wherein, the first evaluation sub-module specifically includes: Selective attention listening ability = First reaction time * First correct rate + First pupil information + First electroencephalogram information; wherein, the first pupil information = First pupil diameter peak value * First average pupil diameter expansion degree; The first electroencephalogram information = First electroencephalogram wave amplitude.

7. The system according to claim 5, wherein, the second evaluation sub-module specifically includes: Processing speed listening and matching ability = Second reaction time * Second correct rate + Second pupil information + Second EEG information; Among them, the second pupil information = Second pupil diameter peak value * Second average pupil diameter expansion degree; The second EEG information = Second EEG wave amplitude.

8. The system according to claim 5, characterized in that, The third evaluation sub-module specifically includes: Working memory listening and matching ability = Third reaction time * Third correct rate + Third pupil information + Third EEG information; Among them, the third pupil information = Third pupil diameter peak value * Third average pupil diameter expansion degree; The third EEG information = Third EEG wave amplitude.

9. A storage medium, characterized in that, A program is stored on the storage medium, and when the program is executed by a processor, the following steps are implemented: Use the information storage and audio output system to store and play the statement audio in different signal-to-noise ratio environments of different test tasks; The test tasks include: selective attention test task, processing speed test task, and working memory test task; During the playback of the statement audio in each signal-to-noise ratio environment of each test task by the information storage and audio output system, the reaction time and correct rate of the subject's discrimination of each test task are statistically analyzed in real time. At the same time, the pupil information of the subject during the execution of each test task is collected in real time by a pupillometer, and the EEG information of the subject during the execution of each test task is collected in real time by an EEG system; Use the pupil information, EEG information of each test task, and the reaction time and correct rate of the discrimination test task to evaluate the listening and matching ability of the subject, including: Obtain the first reaction time and the first correct rate of the subject's discrimination of the selective attention test task after hearing the statement audio in each signal-to-noise ratio environment of the selective attention test task, and use the first pupil information, the first EEG information, and the first reaction time and the first correct rate of the selective attention test task to evaluate the selective attention listening and matching ability of the subject; Obtain the second reaction time and the second correct rate of the subject's discrimination of the processing speed test task after hearing the statement audio in each signal-to-noise ratio environment of the processing speed test task, and use the second pupil information, the second EEG information, and the second reaction time and the second correct rate of the processing speed test task to evaluate the processing speed listening and matching ability of the subject; Obtain the third reaction time and the third correct rate of the subject's discrimination of the working memory test task after hearing the statement audio in each signal-to-noise ratio environment of the working memory test task, and use the third pupil information, the third EEG information, and the third reaction time and the third correct rate of the working memory test task to evaluate the working memory listening and matching ability of the subject; Evaluate the change rate of the subject's listening and matching ability according to the change rate of the subject's selective attention listening and matching ability, the change rate of the processing speed listening and matching ability, and the change rate of the working memory listening and matching ability; Evaluate the signal-to-noise ratio that can reflect the listening and matching ability level of the subject according to the correct rate corresponding to the analysis of the allocation of auditory cognitive resources in the task state under different signal-to-noise ratio environments; Evaluate the listening aptitude of the subject by using the subject's selective attention listening aptitude, processing speed listening aptitude, working memory listening aptitude, listening aptitude change rate, and signal-to-noise ratio.

Images