A social cognitive assessment method, system and terminal based on attention

Abstract

The application relates to the technical field of data processing, and discloses a social cognitive evaluation method and system based on attention, a terminal and a readable storage medium. The method comprises the following steps: guiding a user to make a plurality of social cognitive selections, collecting video data streams and selection data in each social cognitive selection process; calculating a systematic index and an attention index according to the selection data and the video data streams; and generating an evaluation result of social cognitive according to the systematic index and the attention index. Through the collection of the video of the user in each cognitive selection process, analysis is carried out according to the video, whether the attention of the user is concentrated in the place where the attention should be concentrated when the selection is made is judged, the attention condition of the user in the cognitive selection process is analyzed, and the social cognitive ability of the user is judged by using the selection data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an attention-based social cognition assessment method, system, terminal, and readable storage medium. Background Technology

[0002] Autism Spectrum Disorder (ASD) is one of the most common neurodevelopmental disorders in childhood. Its clinical symptoms are mainly manifested as social communication impairment, stereotyped behaviors, and narrow interests.

[0003] Early detection, early diagnosis, and early intervention are the consensus in autism rehabilitation strategies. Early education and behavioral intervention have been found to effectively improve many behavioral, functional, social, and cognitive deficits in children with ASD. Authoritative research and years of clinical experience from professional physicians have shown that early behavioral intervention can significantly improve the learning and daily living abilities of children with autism, and the earlier the intervention, the better the results. Among these, intervention in social skills is crucial for children with autism because social cognitive deficits exacerbate the challenges they face in society. Therefore, assessing and intervening in the social skills of children with autism is extremely important.

[0004] Traditional social skills assessments are usually conducted manually, which suffers from significant skill deficiencies and subjective interference from human judgment. To address this, existing technologies have proposed some automated assessment methods, but these methods largely rely on whether the test subject responds correctly, neglecting the subject's level of concentration during the test, resulting in inaccurate assessments. Summary of the Invention

[0005] The purpose of this invention is to provide an attention-based social cognition assessment method, system, terminal, and readable storage medium, aiming to solve the problem that existing technologies neglect the subject's attention concentration during the test, resulting in inaccurate assessment results.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] This invention provides an attention-based social cognition assessment method, the attention-based social cognition assessment method comprising:

[0008] Guide users to make multiple social cognition choices, and collect video data streams and selection data during each social cognition choice process;

[0009] Based on the selected data and the video data stream, calculate the systematic indicators and attention indicators;

[0010] Based on the aforementioned systematic and attentional indicators, an assessment of social cognition is generated.

[0011] Furthermore, guiding users to make multiple social cognitive choices specifically includes:

[0012] During the stage of understanding social scenarios, guide users to make choices regarding their understanding of social scenarios;

[0013] During the stage of defining social behaviors, guide users to make cognitive choices about social behaviors;

[0014] During the stage of paying attention to social cues, guide users to make choices regarding their understanding of social cues.

[0015] Furthermore, the calculation of systematic indicators and attention indicators based on the selected data and the video data stream specifically includes:

[0016] Calculate the accuracy metric based on the selected data;

[0017] Calculate the response time metric based on the selected data;

[0018] The accuracy rate and response time metrics are used as the systematic metrics;

[0019] The attention metric is calculated based on the video data stream.

[0020] Furthermore, the accuracy rate metric is calculated as follows:

[0021]

[0022] Among them, Acc s This represents the accuracy metric for the s-th stage. This represents the i-th choice within the s-th stage, where n s Let represent the number of choices in the s-th stage. If the ith choice in the s-th stage is the correct choice, then... If the i-th choice in the s-th stage is an incorrect choice, then

[0023] Furthermore, the response time metric is calculated as follows:

[0024]

[0025] Among them, Rt s This represents the response time metric for the s-th stage. This indicates the time taken for the i-th choice within the s-th stage.

[0026] Furthermore, the step of calculating the attention metric based on the video data stream specifically includes:

[0027] Based on the video data stream, viewpoint data is obtained through a viewpoint estimation algorithm;

[0028] The attention index is calculated based on the viewpoint data.

[0029] Furthermore, the attention index is calculated as follows:

[0030]

[0031] Among them, Att s This represents the attention metric for the s-th stage. Let m represent the coordinates of the attention point in the j-th frame of the s-th stage. s Let f represent the frame number in the s-th stage, and let f represent the expected attention region function. This indicates that the coordinates of the attention point are within the expected attention region. I is an indicator function; it takes the value 1 when the function is true and 0 when the function is false.

[0032] Furthermore, to achieve the above objectives, the present invention also provides an attention-based social cognition assessment system, the attention-based social cognition assessment system comprising:

[0033] The guided data acquisition module is used to guide users to make multiple social cognition choices and to collect video data streams and selection data during each social cognition choice process;

[0034] The metric calculation module is used to calculate systematic metrics and attention metrics based on the selected data and the video data stream;

[0035] The results generation module is used to generate assessment results of social cognition based on the systematic indicators and attention indicators.

[0036] In addition, to achieve the above objectives, the present invention also provides a terminal, the terminal comprising: a memory, a processor, and an attention-based social cognitive assessment program stored in the memory and executable on the processor, wherein when the attention-based social cognitive assessment program is executed by the processor, the terminal controls the terminal to implement the steps of the attention-based social cognitive assessment method as described above.

[0037] Furthermore, to achieve the above objectives, the present invention also provides a readable storage medium storing an attention-based social cognition assessment program, which, when executed by a processor, implements the steps of the attention-based social cognition assessment method as described above.

[0038] The present invention, by employing the above technical solution, has the following effects:

[0039] This invention collects videos of users during each cognitive choice process, analyzes the videos to determine whether the user's attention is focused on the appropriate area when making a choice, and thus analyzes the user's attention during the cognitive choice process. This analysis, combined with the choice data, is used to assess the user's social cognitive ability. Attached Figure Description

[0040] Figure 1 This is a flowchart of the steps of a preferred embodiment of the present invention for an attention-based social cognition assessment method;

[0041] Figure 2 This is a task flowchart of an attention-based social cognition assessment method according to a preferred embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the structure of an attention-based social cognition assessment system in a preferred embodiment of the present invention;

[0043] Figure 4 A schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0045] Example 1

[0046] Please see Figure 1 and Figure 2 Embodiment 1 of this application is an attention-based social cognition assessment method. It primarily targets the social cognition and comprehension abilities of children with autism, combining functional games that train social cognition and comprehension with a data collection, recording, and assessment system. This method achieves a fully automated, progressive assessment of the social cognition and comprehension abilities of children with autism, contributing to the development of rehabilitation plans for children with autism and to autism research. Specifically, it includes the following steps:

[0047] S1. Guide users to make multiple social cognition choices and collect video data streams and selection data during each social cognition choice process.

[0048] This invention plays guiding animations at the front end to help test subjects understand the use of the overall system and guide children to make choices about social scenarios, social behaviors, and social cues during the assessment phase. This can train children's social cognitive abilities to a certain extent, while providing the assessment system with selection and video data.

[0049] Simultaneously, the backend collects video data streams via camera and collects the social cognitive choices made by the test subjects. It then judges the social cognitive choices made by the test subjects and provides corresponding feedback, including praise and encouragement, to ensure the normal conduct of the test while maintaining the cognitive guidance goals for the test subjects.

[0050] In this embodiment, users are guided to make multiple social cognitive choices, including a guidance phase and an evaluation phase.

[0051] The guidance stage, also known as the social cognition and interaction guidance stage, is used to guide children to understand social scenarios, social behaviors, and social cues, while also helping them understand that the game is played through touch-based interaction.

[0052] Specifically, in the social cognition and interaction guidance stage, a social scene (such as a restaurant, playground, classroom, etc.) is provided in the functional game interface. In addition, the social scene is introduced through voice so that the test subjects can understand the social scene.

[0053] Then, different social behaviors performed by different people in social scenarios (such as chatting, running, skipping rope, and playing football on the playground) are circled and magnified one by one. The social behaviors are explained through voice and some interpretation is given. At the same time, social cues (such as what kind of ball is being kicked, the facial expressions of the people chatting, etc.) are circled during the explanation of the social behaviors, and the test subjects are reminded to pay attention to the observation of social cues.

[0054] Throughout the entire social cognition and interaction guidance phase, after each social scenario, social behavior, and social cue is introduced, the test subject will be guided by voice to click on the corresponding behavior or cue on the screen to proceed to the next step of the introduction. This allows the test subject to understand that the interaction method with the game is touch interaction and to comprehend the relationship between social scenarios, social behaviors, and social cue while learning about social scenarios, social behaviors, and social cue.

[0055] After the guidance phase, the assessment phase begins. The assessment phase evaluates the test subject's social cognitive abilities and includes three stages: recognizing the social scene, defining social behavior, and noticing social cues.

[0056] The three stages are: the Recognizing Social Scenarios stage, which helps children choose a specific social scenario from multiple options; the Clarifying Social Behaviors stage, which helps children choose the person who performs a specific social behavior within a given social scenario; and the Noticing Social Clues stage, which helps children select the correct social cues within a given social behavior.

[0057] The assessment phase comprises three sub-phases: the social scene recognition phase, the social behavior identification phase, and the social cue observation phase. These phases are cyclically coupled and progressive. In the social scene recognition phase, after selecting a corresponding social scene, the test-taker enters the social behavior identification phase, where they choose a corresponding social behavior within that scene. After selecting a social behavior, they enter the social cue observation phase, where they choose a specific social cue within that behavior. Once all social cues have been identified, the test-taker returns to the previous phase, the social behavior identification phase, to continue selecting a specific social behavior within the scene. If no further social behaviors are selected, the test-taker returns to the initial social scene recognition phase, where they are then instructed to choose another social scene. This cyclical, progressive assessment evaluates the test-taker's cognitive abilities regarding social scenes, social behaviors, and social cues.

[0058] In this embodiment, during both the guidance and evaluation phases, corresponding feedback is provided based on whether the test subject's selection is correct or not. This reinforces the test subject's information upon success and encourages the test subject upon failure, ensuring the correct conduct of subsequent tests.

[0059] In the social scene recognition stage, the functional game interface presents large social scenes such as shopping malls, playgrounds, classrooms, restaurants, parks, train stations, and supermarkets. Voice prompts questions, guiding participants to click on specific social scenes. Different feedback is given based on the participant's correct selection. Clicking the correct scene receives praise; incorrect selections receive encouragement, and the correct scene is circled and guided to click it. After clicking the correct scene, it enlarges to cover the entire screen, leading to the next stage: defining social behavior.

[0060] In the stage of defining social behaviors, the functional game interface will arrange several social behaviors within the current scenario (e.g., in a shopping mall: chatting, checking out, playing claw machines, shopping, etc.). A specific social behavior will be designated, and a voice prompt will ask a question based on that behavior, prompting the test-taker to select it. Different feedback will be given based on the correctness of the selection. Selecting the correct behavior will receive praise; selecting the wrong behavior will receive encouragement, and the correct behavior will be circled and guided to be selected. After the test-taker selects the correct behavior, the character in that behavior will be enlarged, and the test-taker will then proceed to the next stage – the social cues observation stage.

[0061] During the social cues observation phase, the functional game interface prompts participants to identify key social cues within the current social interaction through questions (e.g., children playing with toys: toys; giving gifts: gifts; adults comforting children: children's sad expressions and ice cream). Participants then click on the location of these cues, receiving different feedback based on the correctness of their selection. Clicking the correct cues elicits praise, while incorrect selections receive encouragement, with the correct cues highlighted and guided to the participant. After clicking the correct cues, the current phase ends, and participants return to the previous phase to select other social behaviors and their corresponding cues. If no other social behaviors are available in the previous phase, the game returns to the initial phase to select other social scenarios.

[0062] During the evaluation phase, the system records the clicks made by the test subject before the prompt appears at each stage, and records the video data stream of the entire evaluation phase for subsequent evaluation.

[0063] In this embodiment, during the evaluation phase, the correctness of each social option selection and the length of time the selection takes are recorded and saved as selection data. During the evaluation phase, video streams of the test subjects playing the social cognitive game are collected as video data streams.

[0064] S2. Calculate the systematic index and attention index based on the selected data and the video data stream.

[0065] Specifically, in this embodiment, the selected data records are calculated using an index calculation formula to determine the selection accuracy and response time for each stage, which are used as systematic indicators. The viewpoint data of the test subject in each frame of the video data stream is calculated using a viewpoint estimation algorithm, which is the attention coordinate position of each frame in the functional game interface. Then, based on the viewpoint data, the proportion of attention time for social scenes, social behaviors, and social cues during the evaluation stage is calculated as attention indicators.

[0066] Among them, the systematic indicators include accuracy and response time. The formula for calculating the accuracy in the s-th stage is as follows:

[0067]

[0068] Among them, Acc s This represents the accuracy metric for the s-th stage. This represents the i-th choice within the s-th stage. Within the s-th stage, there are a total of n choices. s The selection process is as follows: if the i-th selection in the s-th stage is the correct selection, then... If the i-th choice in the s-th stage is an incorrect choice, then Therefore, the accuracy rate selected at each stage is the percentage of correct answers in the total number of correct answers.

[0069] The formulas for calculating the response time metrics for each stage are as follows:

[0070]

[0071] Among them, Rt s This represents the response time metric for the s-th stage. Let represent the time taken for the i-th choice within the s-th stage. Therefore, the response time metric for each stage is the average time taken for the choice within that stage.

[0072] The selection accuracy and response time at each stage can form a systematic indicator Sm s =[Acc s ,Rt s ], Sm s This represents the systematic indicator for the s-th stage. The systematic indicator for all three stages can be represented as Sm. s =[Sm1,Sm2,Sm3].

[0073] The attention index assessment phase calculates the attention percentage at key locations in each phase by using viewpoint data obtained from the test subjects' gameplay video stream.

[0074] First, video streams of the test subjects' choices at each evaluation stage are collected and fed into a viewpoint estimation algorithm. This algorithm calculates the subject's viewpoint data in real-time for each frame of the video, representing the subject's attention coordinates relative to the functional game interface at each moment during gameplay. This attention data is then categorized and concatenated according to each stage, resulting in three stages of attention data. This data is then fed into the attention metric calculation stage. By calculating the percentage of attention paid to the correct option area at each stage, the test subject's level of attention to social scenarios, behaviors, and cues can be determined, helping to assess their social perception abilities.

[0075] Specifically, the formula for calculating the attention index at each stage is as follows:

[0076]

[0077] Among them, Att s This represents the attention metric for the s-th stage. This represents the coordinates of the attention point in the j-th frame of the s-th stage. The s-th stage has a total of m... sFrame. And f represents the expected attention region function, which is the correct scene region in the social scene stage, the correct social behavior region in the social behavior stage, and the correct social cue region in the social cue stage. This indicates that the coordinates of the attention point are within the expected attention area. I is an indicator function; it takes the value 1 when the function is true and 0 when it is false. Therefore, this formula represents the percentage of time a child spends paying attention to the correct area during the s-th stage.

[0078] The attention percentage in the s-th stage is the attention metric Fm for the s-th stage. s =Att s The set of attention metrics for the three stages can be represented as Fm s =[Fm1,Fm2,Fm3].

[0079] S3. Based on the aforementioned systematic indicators and attention indicators, generate an assessment result for social cognition.

[0080] After the indicators for the above stages are calculated, the two indicators for each stage, along with the dataset of the test subjects and the evaluation time, will be printed out as data in the report. This data will serve as the basis for the test subjects' scores in this evaluation. At the same time, this data will also be stored in the database for later data processing, analysis, and retrieval.

[0081] The above enables a fully automated assessment of the social cognition and comprehension abilities of individuals with autism.

[0082] Example 2

[0083] Please see Figure 3 Based on the above method, the present invention also provides an attention-based social cognition assessment system, the attention-based social cognition assessment system comprising:

[0084] The guided acquisition module 51 is used to guide the user to make multiple social cognition choices and to collect video data streams and selection data during each social cognition choice process.

[0085] The indicator calculation module 52 is used to calculate systematic indicators and attention indicators based on the selected data and the video data stream;

[0086] The result generation module 53 is used to generate an assessment result of social cognition based on the systematic indicators and attention indicators.

[0087] Example 3

[0088] Please see Figure 4Based on the above method, the present invention also provides a terminal, which includes a processor 10, a memory 20, and a display 30. However, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0089] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores an attention-based social cognition assessment program 40, which can be executed by the processor 10 to realize the terminal of this application.

[0090] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing related programs of the attention-based social cognition assessment method.

[0091] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light Emitting Diode) touchscreen, etc. The display 30 is used to display information on the terminal and to display a visual user interface.

[0092] In one embodiment, when the processor 10 executes the attention-based social cognitive assessment program 40 in the memory 20, it implements the steps of the attention-based social cognitive assessment method as described above.

[0093] Example 4

[0094] This embodiment provides a storage medium that stores an attention-based social cognition assessment program. When executed by a processor, the attention-based social cognition assessment program implements the steps of the attention-based social cognition assessment method as described above.

[0095] In summary, this invention reduces the need for rehabilitation professionals by establishing a systematic assessment process, avoiding the problems of insufficient assessment professionals and human resource consumption caused by manual assessment, as well as the fact that the results of manual assessment are often based on the subjective feelings of professionals and lack specific and objective assessment indicators.

[0096] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0097] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The storage medium can be a memory, magnetic disk, optical disk, etc.

[0098] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An attention-based social cognition assessment method, characterized in that, The attention-based social cognitive assessment method includes: Guide users to make multiple social cognition choices, and collect video data streams and selection data during each social cognition choice process; Based on the selected data and the video data stream, calculate the systematic indicators and attention indicators; Based on the aforementioned systematic indicators and attention indicators, an assessment result of social cognition is generated; The guidance provided to users to make multiple social cognition choices specifically includes: During the stage of understanding social scenarios, guide users to make choices regarding their understanding of social scenarios; During the stage of defining social behaviors, guide users to make cognitive choices about social behaviors; During the social cues observation phase, guide users to make choices regarding their understanding of social cues; The step of calculating systematic indicators and attention indicators based on the selected data and the video data stream specifically includes: Based on the selected data, calculate the accuracy metric and the response time metric; The accuracy rate and response time metrics are used as the systematic metrics; The attention metric is calculated based on the video data stream; The accuracy rate metric is calculated as follows: ； in, Indicates the first Accuracy metrics for each stage Indicates the first The first phase within the first stage One option, Indicates the first The number of choices in the first stage, if the first stage The first phase within the first stage If the choice is the correct choice, then If the first The first phase within the first stage If the choice is incorrect, then ; The response time metric is calculated as follows: ； in, Indicates the first Response time metrics for each stage Indicates the first The first phase within the first stage The duration of each option; The attention index is calculated as follows: ； in, Indicates the first Attention metrics at each stage Indicates the first The first stage The coordinates of the attention point in the frame. Indicates the first The number of frames in each stage, and This represents the expected attention region function. This indicates that the coordinates of this attention point are within the expected attention area. It is an indicator function. When the value inside the function is true, it takes the value 1, and when the value inside the function is false, it takes the value 0.

2. The attention-based social cognition assessment method according to claim 1, characterized in that, The calculation of the attention metric based on the video data stream specifically includes: Based on the video data stream, viewpoint data is obtained through a viewpoint estimation algorithm; The attention metric is calculated based on the viewpoint data.

3. An attention-based social cognitive assessment system, said attention-based social cognitive assessment system being used to implement the attention-based social cognitive assessment method according to any one of claims 1-2, characterized in that, The attention-based social cognitive assessment system includes: The guided data acquisition module is used to guide users to make multiple social cognition choices and to collect video data streams and selection data during each social cognition choice process; The metric calculation module is used to calculate systematic metrics and attention metrics based on the selected data and the video data stream; The results generation module is used to generate assessment results of social cognition based on the systematic indicators and attention indicators.

4. A terminal, characterized in that, The terminal includes: a memory, a processor, and an attention-based social cognitive assessment program stored in the memory and executable on the processor. When the attention-based social cognitive assessment program is executed by the processor, it controls the terminal to implement the steps of the attention-based social cognitive assessment method as described in any one of claims 1-2.

5. A readable storage medium, characterized in that, The readable storage medium stores an attention-based social cognition assessment program, which, when executed by a processor, implements the steps of the attention-based social cognition assessment method as described in any one of claims 1-2.

Images