Attention-based social cognition assessment method, system and terminal

Abstract

The invention relates to the technical field of data processing, and discloses an attention-based social cognition evaluation method and system, a terminal and a readable storage medium, and the method comprises the steps: guiding a user to make a plurality of social cognition selections, and collecting a video data stream and selection data in each social cognition selection process; calculating a systematic index and an attention index according to the selection data and the video data stream; and according to the systematic index and the attention index, generating an evaluation result of social cognition. According to the method, the video of the user in each cognitive selection process is collected, analysis is carried out according to the video, and whether the attention of the user is concentrated in a place where the attention should be concentrated or not when the user makes a selection is judged, so that the attention condition of the user in the cognitive selection process is analyzed, and the social cognitive ability of the user is judged with the assistance of selection data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an attention-based social cognition assessment method, system, terminal and readable storage medium. Background Art

[0002] Autism Spectrum Disorder (ASD) is one of the most common neurodevelopmental disorders in children. Its clinical symptoms mainly include social communication disorders, stereotyped behaviors and narrow interests.

[0003] "Early detection, early diagnosis, and early intervention" is the consensus of autism rehabilitation strategies. Early education and behavioral intervention have been found to effectively improve many behavioral, functional, social, and cognitive defects of ASD children. Authoritative research and years of clinical experience of professional doctors have shown that through early behavioral intervention, the learning and living abilities of children with autism can be greatly improved, and the earlier the intervention, the better the effect. Among them, intervention in social skills is crucial for children with autism, because social cognitive defects exacerbate the challenges faced by autistic patients in society. Therefore, it is very important to evaluate and intervene in the social skills of children with autism.

[0004] Traditional social ability assessment is usually conducted manually, which has a lot of ability deficiencies and subjective interference from human judgment. For this reason, the existing technology proposes some methods of assessment and judgment based on automation, but most of these methods rely on whether the subject responds correctly or not, and ignore the subject's concentration during the test, resulting in inaccurate evaluation results. Summary of the invention

[0005] The purpose of the present invention is to provide an attention-based social cognition assessment method, system, terminal and readable storage medium, aiming to solve the problem that the prior art ignores the subject's concentration during the test when conducting social ability assessment, resulting in inaccurate assessment results.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] The present invention provides a social cognition assessment method based on attention, and the social cognition assessment method based on attention includes:

[0008] Guide the user to make multiple social cognitive choices, and collect video data streams and selection data during each social cognitive choice process;

[0009] Calculating a systematic index and an attention index based on the selection data and the video data stream;

[0010] An evaluation result of social cognition is generated based on the systematic indicators and the attention indicators.

[0011] Furthermore, guiding the user to make multiple social cognition choices specifically includes:

[0012] In the stage of understanding social scenarios, guide users to make social scenario cognitive choices;

[0013] In the stage of clarifying social behavior, guide users to make cognitive choices about social behavior;

[0014] In the stage of paying attention to social cues, guide users to make social cue recognition choices.

[0015] Further, the calculating of the systematic index and the attention index according to the selected data and the video data stream specifically includes:

[0016] Calculate the accuracy rate index according to the selected data;

[0017] Calculating a response time indicator based on the selected data;

[0018] Using the accuracy index and the response time index as the systematic index;

[0019] The attention index is calculated based on the video data stream.

[0020] Furthermore, the accuracy index is calculated as follows:

[0021]

[0022] Among them, Acc s represents the accuracy index of the sth stage, represents the i-th choice in the s-th stage, n s represents the number of choices in the sth stage. If the i-th choice in the sth stage is the correct choice, then If the i-th choice in the s-th stage is a wrong choice, then

[0023] Furthermore, the response time indicator is calculated as:

[0024]

[0025] Among them, Rt s represents the response time indicator of the sth stage, It shows the duration of the i-th choice in the s-th stage.

[0026] Further, the calculating the attention index according to the video data stream specifically includes:

[0027] According to the video data stream, obtaining viewpoint data by using a viewpoint estimation algorithm;

[0028] The attention index is calculated based on the viewpoint data.

[0029] Furthermore, the attention index is calculated as:

[0030]

[0031] Among them, Att s represents the attention index of the sth stage, represents the coordinates of the attention point of the jth frame in the sth stage, m s represents the frame number of the sth stage, and f represents the expected attention region function, Indicates that the coordinates of this attention point are within the expected attention area. I is an indicator function. When the value inside the function is true, it takes the value 1, and when the value inside the function is false, it takes the value 0.

[0032] In addition, to achieve the above-mentioned purpose, the present invention also provides an attention-based social cognition evaluation system, the attention-based social cognition evaluation system comprising:

[0033] A guidance collection module, used to guide the user to make multiple social cognition selections, and collect video data streams and selection data during each social cognition selection process;

[0034] An indicator calculation module, used for calculating a systematic indicator and an attention indicator according to the selection data and the video data stream;

[0035] The result generation module is used to generate the evaluation result of social cognition according to the systematic index and the attention index.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, which includes: a memory, a processor, and an attention-based social cognition assessment program stored in the memory and executable on the processor, wherein the attention-based social cognition assessment program, when executed by the processor, controls the terminal to implement the steps of the attention-based social cognition assessment method as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also provides a readable storage medium, which stores an attention-based social cognition assessment program, and when the attention-based social cognition assessment program is executed by a processor, it implements the steps of the attention-based social cognition assessment method as described above.

[0038] The present invention adopts the above technical solution to achieve the following effects:

[0039] The present invention collects videos of users during each cognitive selection process, analyzes the videos, and determines whether the user's attention is focused on where it should be when making a choice, thereby analyzing the user's attention in the cognitive selection process and using the selection data to make a judgment on the user's social cognitive ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of the steps of a method for social cognition assessment based on attention in a preferred embodiment of the present invention;

[0041] Figure 2 is a task flow chart of an attention-based social cognition assessment method in a preferred embodiment of the present invention;

[0042] Figure 3 It is a structural schematic diagram of a social cognition assessment system based on attention in a preferred embodiment of the present invention;

[0043] Figure 4 A schematic diagram of an operating environment of a preferred embodiment of a terminal of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0045] Embodiment 1

[0046] See also Figure 1 and Figure 2 Embodiment 1 of the present application is an attention-based social cognition assessment method, which mainly targets the social cognition and comprehension ability of autistic children, combines functional games for training social cognition and comprehension with a collection, recording and assessment system, and realizes a full-process automated progressive assessment of the social cognition and comprehension ability of autistic children, so as to make certain contributions to the formulation of rehabilitation plans for autistic children in social cognition and autism research. Specifically, the method comprises the following steps:

[0047] S1. Guide the user to make multiple social cognitive choices, and collect video data streams and selection data during each social cognitive choice process.

[0048] The present invention plays a guiding animation at the front end to help the subject understand the use of the overall system, and is used to guide children to make choices about social scenes, social behaviors and social clues during the evaluation stage, which can train children's social cognitive abilities to a certain extent, while providing selection and video data for the evaluation system.

[0049] At the same time, the video data stream is collected by the camera at the back end, as well as the social cognitive choices made by the subjects, so as to make judgments based on the social cognitive choices made by the subjects and give corresponding feedback, including praise feedback and encouragement feedback, so as to ensure the normal progress of the test on the basis of ensuring the cognitive guidance goals of the subjects.

[0050] In this embodiment, the user is guided to make multiple social cognition choices, including a guidance phase and an evaluation phase.

[0051] The guidance stage is the social cognition and interaction guidance stage, which is used to guide children to initially understand social scenes, social behaviors and social clues, and at the same time guide children to understand that this game interacts through touch and click.

[0052] Specifically, in the social cognition and interaction guidance stage, a social scene (e.g., restaurant, playground, classroom) is provided in the functional game interface. In addition, this social scene is introduced through voice at the same time to facilitate the subjects to understand the social scene.

[0053] Afterwards, the social behaviors performed by different people in the social scene (for example: chatting, running, skipping rope, kicking a ball in the playground) are circled and magnified one by one, and the social behaviors are explained through voice, and some explanations are given to the social behaviors. At the same time, in the process of explaining the social behaviors, social clues are circled (for example: what kind of ball is kicked, the expressions of the chatting people, etc.) and the subjects are reminded to pay attention to the observation of social clues.

[0054] During the entire social cognition and interaction guidance stage, after the introduction of each social scene, social behavior and social clue, the subject will be guided by voice to click on the corresponding behavior and clue on the screen, so as to advance to the next introduction. In the process of learning social scenes, social behaviors and social clues, the subject will clearly understand that the way to interact with the game is touch interaction, and understand the relationship between social scenes, social behaviors and social clues.

[0055] After completing the guidance stage, the assessment stage begins. The assessment stage is used to assess the social cognitive ability of the subject, which specifically includes three stages: recognizing social scenes, clarifying social behaviors, and paying attention to social clues.

[0056] Among them, the stage of recognizing social scenes is used to enable children to select designated social scenes from multiple social scenes; the stage of clarifying social behaviors is used to enable children to choose people who perform designated social behaviors in designated social scenes; and the stage of paying attention to social clues is used to enable children to choose the correct social clues in designated social behaviors.

[0057] In the evaluation stage, the three stages of recognizing social scenes, clarifying social behaviors, and noticing social clues are cyclically coupled and progressive. That is, in the stage of recognizing social scenes, after the subject selects the corresponding social scene, he / she enters the stage of clarifying social behaviors, that is, selecting the corresponding social behavior in this social scene. After the subject selects the corresponding social behavior, he / she will enter the stage of noticing social clues, that is, selecting the designated social clues in this social behavior. After finding the social clues, he / she will return to the previous stage, that is, the stage of clarifying social behaviors, and continue to select designated social behaviors in the scene. If there are no unselected social behaviors, he / she will return to the earliest stage, that is, the stage of recognizing social scenes, and designate the child to choose other social scenes. In this way, the subject's cognitive ability of social scenes, social behaviors, and social clues is progressively evaluated.

[0058] In this embodiment, in both the guidance stage and the evaluation stage, corresponding feedback will be given according to whether the subject's choice is correct or not, so as to reinforce the subject's information in case of success and encourage the subject in case of failure to ensure the correct conduct of subsequent tests.

[0059] In the stage of understanding social scenes, the functional game interface will provide large social scenes such as shopping malls, playgrounds, classrooms, restaurants, parks, stations, supermarkets, etc. At this time, the voice will ask questions, so that the test subject can click on the designated large social scene, and give different feedback according to whether the test subject's choice is correct or not. In this process, clicking on the correct scene will give praise feedback; wrong selection will give encouraging feedback, circle the correct scene and guide the test subject to click on the correct scene. After clicking on the correct scene, the selected social scene becomes larger and covers the entire screen, entering the next stage of the corresponding social scene, that is, the stage of clarifying social behavior.

[0060] In the stage of clarifying social behaviors, the functional game interface will arrange social behaviors in several scenes according to the current scene (for example, in a shopping mall: chatting, checking out, playing doll machines, shopping, etc.). At this time, a social behavior will be specified, and the voice will ask questions based on the specified social behavior, so that the subject can click to select the specified social behavior, and different feedback will be given according to whether the subject's selection is correct or not. In this process, clicking on the correct social behavior will give praise feedback; choosing the wrong one will give encouraging feedback, and the correct social behavior will be circled and guided to click. After the subject clicks on the correct social behavior, the character in the correct social behavior will be enlarged, and then enter the next stage of this behavior - paying attention to social clues.

[0061] In the stage of paying attention to social clues, the functional game interface will ask the subjects to find the key social clues in the current social behavior through questions based on the current social behavior (for example: social clues of children playing with toys: toys; social clues of giving gifts: gifts, social clues of adults comforting children: children's sad expressions and ice cream given by adults), so that the subjects can click on the location of the social clues in the questions, and give different feedback based on whether the subjects' choices are correct or not. In this process, clicking on the correct social clues will give praise feedback; incorrect choices will give encouraging feedback, and the correct social clue location will be circled and guided to click. After the subject clicks on the correct social clue, at the end of the current stage, return to the previous stage to select other social behaviors and their social clues. If there are no other social behaviors in the previous stage, return to the initial stage to select other social scenes.

[0062] During the evaluation phase, the system records the subject's clicks before the prompt appears in each stage, and records the video data stream of the entire evaluation phase for subsequent evaluation.

[0063] In this embodiment, during the evaluation phase, the correctness of each social option selection and the length of time for the selection are recorded and saved as selection data, and during the evaluation phase, a video stream of the subject playing the social cognition game is collected as a video data stream.

[0064] S2. Calculate systematic indicators and attention indicators based on the selection data and the video data stream.

[0065] Specifically, in this embodiment, the selection data records are passed through the indicator calculation formula to calculate the selection accuracy and response time of each stage as a systematic indicator. The viewpoint data of the subject in each frame of the video data stream is calculated through the viewpoint estimation algorithm, that is, the attention coordinate position of each frame in the functional game interface. The proportion of attention time for social scenes, social behaviors and social clues in the evaluation stage is then calculated based on the viewpoint data as the attention indicator.

[0066] Among them, the systematic indicators include the accuracy index and the response time index. The calculation formula of the accuracy index of the sth stage is as follows:

[0067]

[0068] Among them, Acc s represents the accuracy index of the sth stage, represents the i-th choice in the s-th stage. In the s-th stage, there are n s If the i-th choice in the s-th stage is the correct choice, then If the i-th choice in the s-th stage is a wrong choice, then Therefore, the accuracy of selection at each stage is the proportion of the correct number in the total number.

[0069] The calculation formula for the response time indicator in each stage is as follows:

[0070]

[0071] Among them, Rt s represents the response time indicator of the sth stage, It shows the time taken for the i-th choice in the s-th stage. Therefore, the response time indicator of each stage is the average time of the choices in this stage.

[0072] The selection accuracy and response time of each stage can constitute the system index Sm s =[Acc s ,Rt s ],Sm s It represents the systematic index of the sth stage. The systematic index of the three stages can be expressed as Sm s =[Sm1,Sm2,Sm3].

[0073] During the attention index evaluation phase, the attention data obtained from the viewpoint data of the subject's gaming video stream is used to calculate the attention percentage of key positions in each stage.

[0074] First, the video stream of the test subject's choice at each evaluation stage is collected and passed to the viewpoint estimation algorithm. The viewpoint estimation algorithm calculates the test subject's viewpoint data in each frame of video in real time, that is, the test subject's attention coordinates for the functional game interface in each frame, which indicates where the test subject focuses on the functional game interface at every moment during the game stage. The above attention data is classified and spliced ​​according to each stage to obtain the attention data of the three stages, which are passed to the attention index calculation stage. By calculating the proportion of attention to the correct option area in each stage, the test subject's attention to social scenes, behaviors and clues can be obtained, which helps to evaluate the test subject's social perception ability.

[0075] Specifically, the calculation formula of the attention index at each stage is as follows:

[0076]

[0077] Among them, Att s represents the attention index of the sth stage, represents the coordinates of the attention point of the jth frame in the sth stage. There are m sframe. And f represents the expected attention region function, which is the correct scene region in the social scene stage, the correct social behavior region in the social behavior stage, and the correct social cue region in the social cue stage. Indicates that the coordinates of this attention point are within the expected attention area. I is an indicator function. When the value inside the function is true, it takes a value of 1; when the value inside the function is false, it takes a value of 0. Therefore, this formula represents the proportion of time that children pay attention to the correct area in the sth stage.

[0078] The attention ratio of the sth stage is the attention index Fm of the sth stage s =Att s , the set of attention indicators of the three stages can be expressed as Fm s =[Fm1,Fm2,Fm3].

[0079] S3. Generate social cognition evaluation results based on the systematic indicators and attention indicators.

[0080] After the calculation of the indicators of the above stages is completed, the two indicators of each stage will be printed out together with the data set of the test subject and the evaluation time as the data of the report, which will serve as the basis for the test subject's score for this evaluation. At the same time, such data will also be stored in the database for later data processing, analysis and access.

[0081] The above will realize the fully automated assessment of the social cognition and comprehension abilities of autistic subjects.

[0082] Embodiment 2

[0083] See also Figure 3 Based on the above method, the present invention also provides an attention-based social cognition evaluation system, the attention-based social cognition evaluation system comprising:

[0084] A guidance collection module 51 is used to guide the user to make multiple social cognition selections and collect video data streams and selection data during each social cognition selection process;

[0085] An index calculation module 52, for calculating a systematic index and an attention index according to the selection data and the video data stream;

[0086] The result generation module 53 is used to generate the evaluation result of social cognition according to the systematic index and the attention index.

[0087] Embodiment 3

[0088] See also Figure 4Based on the above method, the present invention further provides a terminal, which includes a processor 10, a memory 20 and a display 30. However, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0089] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores an attention-based social cognition assessment program 40, which can be executed by the processor 10 to realize the terminal in the present application.

[0090] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing relevant programs of the attention-based social cognition assessment method.

[0091] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface.

[0092] In one embodiment, when the processor 10 executes the attention-based social cognition assessment program 40 in the memory 20 , the steps of the attention-based social cognition assessment method described above are implemented.

[0093] Embodiment 4

[0094] This embodiment provides a storage medium, wherein the readable storage medium stores an attention-based social cognition assessment program, and when the attention-based social cognition assessment program is executed by a processor, the steps of the attention-based social cognition assessment method described above are implemented.

[0095] In summary, the present invention reduces the demand for rehabilitation professionals by formulating a systematic evaluation process, avoids the problems of insufficient evaluation professionals and waste of human resources caused by manual evaluation, and the results of manual evaluation are often based on the subjective feelings of professionals and lack specific and objective evaluation indicators.

[0096] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0097] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, an optical disk, etc.

[0098] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A social cognition assessment method based on attention, characterized in that: The attention-based social cognition assessment method comprises: Guide the user to make multiple social cognitive choices, and collect video data streams and selection data during each social cognitive choice process; Calculating a systematic index and an attention index based on the selection data and the video data stream; An evaluation result of social cognition is generated based on the systematic indicators and the attention indicators.

2. The attention-based social cognition assessment method according to claim 1, characterized in that: The step of guiding the user to make multiple social cognition choices specifically includes: In the stage of understanding social scenarios, guide users to make social scenario cognitive choices; In the stage of clarifying social behavior, guide users to make cognitive choices about social behavior; In the stage of paying attention to social cues, guide users to make social cue recognition choices.

3. The attention-based social cognition assessment method according to claim 2, characterized in that: The calculating of the systematic index and the attention index according to the selected data and the video data stream specifically includes: Calculating the accuracy index and the response time index according to the selected data; Using the accuracy index and the response time index as the systematic index; The attention index is calculated based on the video data stream.

4. The attention-based social cognition assessment method according to claim 3, characterized in that: The calculation of the accuracy index is: Among them, Acc s represents the accuracy index of the sth stage, represents the i-th choice in the s-th stage, n s represents the number of choices in the sth stage. If the i-th choice in the sth stage is the correct choice, then If the i-th choice in the s-th stage is a wrong choice, then 5. The attention-based social cognition assessment method according to claim 4, characterized in that: The response time indicator is calculated as: Among them, Rt s represents the response time indicator of the sth stage, It shows the duration of the i-th choice in the s-th stage.

6. The attention-based social cognition assessment method according to claim 2, characterized in that: The calculating the attention index according to the video data stream specifically includes: According to the video data stream, obtaining viewpoint data by using a viewpoint estimation algorithm; The attention index is calculated based on the viewpoint data.

7. The attention-based social cognition assessment method according to claim 6, characterized in that: The attention index is calculated as: Among them, Att s represents the attention index of the sth stage, represents the coordinates of the attention point of the jth frame in the sth stage, m s represents the frame number of the sth stage, and f represents the expected attention region function, Indicates that the coordinates of this attention point are within the expected attention area. I is an indicator function. When the value inside the function is true, it takes the value 1, and when the value inside the function is false, it takes the value 0.

8. A social cognition assessment system based on attention, characterized in that: The attention-based social cognition assessment system includes: A guidance collection module, used to guide the user to make multiple social cognition selections, and collect video data streams and selection data during each social cognition selection process; An indicator calculation module, used to generate an evaluation result of social cognition based on the systematic indicator and the attention indicator; A result generation module is used to generate an evaluation result based on the systematic indicators and attention indicators.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and an attention-based social cognition assessment program stored in the memory and executable on the processor. When the attention-based social cognition assessment program is executed by the processor, the terminal is controlled to implement the steps of the attention-based social cognition assessment method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores an attention-based social cognition assessment program, and when the attention-based social cognition assessment program is executed by a processor, the steps of the attention-based social cognition assessment method according to any one of claims 1 to 7 are implemented.

Images