An autism emotional ability evaluation method based on electroencephalogram and visual emotional features

Abstract

The application provides an autism emotion ability evaluation method based on electroencephalogram and visual emotion features, acquires electroencephalogram signals and facial video data of an autism subject, respectively extracts electroencephalogram emotion features and visual emotion features, the electroencephalogram features include frequency band energy calculation and brain region function connection analysis, the visual features include facial key point expression features and emotion arousal degree sequences. Through an emotion arousal degree screening mechanism, effective emotion segments are screened, and multi-modal features are input into a structured Prompt reasoning model for semantic analysis to generate an emotion ability analysis report of the subject. According to the analysis result, scores of the subject in emotion recognition ability, emotion understanding ability and emotion regulation ability and other dimensions are output, and objective and quantitative evaluation of the emotion ability of an autism individual is realized. Through the cooperative processing of electroencephalogram and visual information, the evaluation accuracy and reliability can be improved, and scientific basis and application value are provided for early screening, intervention assistance and individualized rehabilitation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence medical assessment technology, and in particular to a method for assessing the emotional abilities of autistic patients based on EEG and visual emotional characteristics. Background Technology

[0002] In recent years, with the rapid development of neuroscience research and artificial intelligence technology, the assessment of emotional abilities in children with autism spectrum disorder (ASD) has gradually received widespread attention from the academic and clinical fields. Emotional abilities encompass multiple aspects, including emotion recognition, emotion understanding, and emotion regulation, and are a crucial foundation for the social interaction abilities of children with autism. Currently, clinical assessment of emotional abilities primarily relies on behavioral observation and standardized psychological scales, such as emotion recognition tasks or behavioral rating scales. However, these methods often depend on expert experience, resulting in high subjectivity, low assessment efficiency, and difficulty in conducting continuous quantitative analysis.

[0003] With the development of electroencephalography (EEG) analysis technology, researchers have begun to use EEG signals to analyze the neural activity patterns of autistic children during emotional processing, analyzing the activity of brain regions related to emotional processing through indicators such as frequency band energy and functional connectivity. Meanwhile, the development of computer vision technology has made facial expression recognition and emotional behavior analysis possible. Through video data, visual features such as facial key points, facial expression units, and emotional intensity can be extracted, thereby reflecting an individual's emotional expression ability.

[0004] However, unimodal emotion assessment methods have significant limitations. For example, when using only EEG signal analysis, it is difficult to directly observe overt emotional behavior; while analyzing only visual expressions cannot reflect the individual's internal neural activity. To address these issues, some studies have begun to explore multimodal emotion assessment methods, which improve the accuracy and reliability of assessments by integrating EEG signals with visual behavioral information. However, existing multimodal methods still have several problems, such as difficulties in synchronizing multimodal data over time, insufficient mechanisms for selecting emotional segments, poor model interpretability, and the difficulty in generating structured analysis reports from the assessment results.

[0005] Furthermore, with the development of large language model technology, semantic reasoning-based intelligent analysis methods are gradually being applied in the fields of medical and psychological assessment. However, how to effectively transform multimodal emotional features into interpretable emotional competence analysis reports and achieve structured assessment of the emotional competence of children with autism remains a pressing issue in current research. Summary of the Invention

[0006] This invention provides a method for assessing the emotional abilities of individuals with autism based on EEG and visual emotional characteristics. This method addresses the shortcomings of existing methods for assessing the motor abilities of individuals with autism, such as strong subjectivity, insufficient single-modal information, difficulty in effectively fusing multimodal data, and insufficient interpretability of assessment results. By simultaneously collecting EEG signals and motion video data, and combining EEG signal analysis technology, computer vision posture recognition technology, and a multimodal deep learning fusion model, this invention achieves an objective, quantitative, and intelligent assessment of the motor abilities of individuals with autism.

[0007] In a first aspect, the present invention provides a method for assessing the emotional abilities of autistic individuals based on electroencephalogram (EEG) and visual emotional characteristics, comprising: Collect electroencephalogram (EEG) signal data and facial video data from autistic subjects; The EEG signal data is preprocessed and emotional features are extracted to obtain an EEG emotional feature vector; Facial expression recognition and emotional arousal calculation are performed on the facial video data to obtain a visual emotion feature vector; Multimodal time segments are filtered based on emotional arousal, retaining high-arousal emotional segments and constructing an effective set of emotional features; The effective set of emotional features is input into the structured Prompt inference model for emotional semantic analysis, generating a report on the subject's emotional capacity analysis. Based on the subject's emotional capacity analysis report, the subject's emotional capacity assessment results are output.

[0008] According to the present invention, a method for assessing the emotional abilities of autistic individuals based on electroencephalogram (EEG) and visual emotional characteristics is provided, which collects EEG signal data and facial video data of autistic subjects, including: A multi-channel EEG device was used to acquire electroencephalogram (EEG) signal data of autistic subjects, and the sampling frequency and number of channels were determined. The EEG signal data included the sampling potential value of any channel at any time point. Facial video of autistic subjects is captured using camera equipment to obtain facial video data, and the video frame rate, total number of frames, and width and height of the video resolution are determined. The facial video data includes the pixel matrix of any frame image. The EEG signal data and the facial video data are synchronized, abnormal signals are removed, and dropped frames and artifacts are detected, so that the EEG signal data and the facial video data correspond in the time dimension.

[0009] According to the present invention, a method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional features is provided. The method involves preprocessing the EEG signal data and extracting emotional features to obtain an EEG emotional feature vector, including: The EEG signal data is subjected to bandpass filtering and notch filtering to obtain filtered EEG signals; The filtered EEG signal is subjected to eye movement artifact removal and electromyography artifact removal, and clean EEG signal is obtained by using independent component analysis or other blind source separation algorithms. Time-frequency analysis was performed on the clean EEG signal to calculate the power of different frequency bands and obtain a set of frequency band energy characteristics; Calculate the functional connectivity features of brain regions, including coherence, phase synchronization index, or functional connectivity matrix, to obtain a set of brain region activity pattern features; The frequency band energy feature set is combined with the brain region activity pattern feature set to form the EEG emotion feature vector.

[0010] According to the present invention, a method for assessing the emotional abilities of autistic individuals based on electroencephalogram (EEG) and visual emotional features is provided. This method performs facial expression recognition and emotional arousal calculation on the facial video data to obtain a visual emotional feature vector, including: Face detection and facial key point localization are performed on the facial video data to obtain a set of facial key points for each frame of the image. The set of facial key points includes the spatial coordinates of any key point. Based on the set of facial key points, facial expression action unit features are calculated to obtain a set of facial expression features; Based on the set of facial expression features, the emotional arousal value is calculated to obtain the emotional arousal sequence; The set of facial expression features is combined with the emotional arousal sequence to form a visual emotional feature vector.

[0011] According to the present invention, a method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics is provided. This method filters multimodal time segments based on emotional arousal levels, retains high-arousal emotional segments, and constructs an effective set of emotional features, including: A threshold is determined based on the emotional arousal sequence, and emotional segments are filtered to obtain a set of valid emotional segments; The effective emotional feature set is obtained by extracting multimodal features within the corresponding time window based on the effective fragment set; The effective set of emotional features is segmented into time windows and integrated to form multimodal emotional feature input data.

[0012] According to the present invention, a method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional features is provided. The effective set of emotional features is input into a structured Prompt inference model for emotional semantic analysis, generating a report on the subject's emotional abilities, including: Construct a structured Prompt template, including an instruction layer, an input layer, and an output layer. The instruction layer is used to define the model roles and analysis tasks, the input layer is used to input multimodal emotion feature data, and the output layer is used to define the emotion capability analysis report format. The effective set of emotional features is input into the structured Prompt template to form a complete model input; A large language model is invoked to perform semantic analysis on multimodal emotional features, generating an emotional ability analysis report for the subject.

[0013] According to the present invention, a method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics outputs the assessment results of the subjects' emotional abilities based on the subjects' emotional ability analysis reports, including: The ability indicators of each dimension in the subject's emotional ability analysis report are quantitatively scored to obtain an emotional ability score vector, which includes an emotion recognition ability score, an emotion understanding ability score, and an emotion regulation ability score. The emotional ability scoring vector is standardized to normalize the scoring range to a preset scoring range; Generate an emotional competence assessment report, including scoring results and competence analysis explanations; The emotional competence assessment report is output to the assessment system or user terminal to provide a basis for clinical assessment or intervention decisions.

[0014] Secondly, the present invention also provides an autism emotional ability assessment system based on EEG and visual emotional characteristics, comprising: The acquisition module is used to collect electroencephalogram (EEG) signal data and facial video data of autistic subjects; The EEG processing module is used to preprocess the EEG signal data and extract emotional features to obtain an EEG emotional feature vector. The visual processing module is used to perform facial expression recognition and emotional arousal calculation on the facial video data to obtain a visual emotion feature vector. The filtering module is used to filter multimodal time segments based on emotional arousal, retaining high-arousal emotional segments and constructing an effective set of emotional features; The analysis module is used to input the set of effective emotional features into the structured Prompt inference model for emotional semantic analysis and generate a report on the subject's emotional ability analysis. The output module is used to output the subject's emotional ability assessment results based on the subject's emotional ability analysis report.

[0015] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the autism emotional ability assessment method based on EEG and visual emotional characteristics as described above.

[0016] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the autism emotional ability assessment method based on EEG and visual emotional characteristics as described above.

[0017] This invention provides a method for assessing emotional abilities in autism based on EEG and visual emotional features. By integrating EEG signals and visual emotional features, it achieves joint analysis of neural activity and behavioral performance in the emotional processing of autistic children, thereby improving the comprehensiveness and accuracy of emotional ability assessment. It introduces an emotional arousal screening mechanism to improve the effectiveness of multimodal emotional features and reduce the impact of noisy data on assessment results by screening high-emotional-arousal segments. Through multimodal feature modeling and structured Prompt inference, complex multimodal features are transformed into interpretable emotional ability analysis reports, improving the interpretability of assessment results. The automated analysis process enables quantitative assessment of emotional abilities, reducing the subjectivity of traditional scale assessments and improving assessment efficiency. The generated emotional ability assessment results can provide a scientific basis for personalized intervention and rehabilitation training for autistic children. Combining EEG analysis, computer vision, and large language model technology, a complete multimodal intelligent assessment framework is constructed, possessing high application value. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the autism emotional capacity assessment method based on EEG and visual emotional characteristics provided by the present invention. Figure 2 This is the overall system structure diagram provided by the present invention; Figure 3 This is a schematic diagram of EEG signal acquisition and processing provided by the present invention; Figure 4 This is a schematic diagram of visual motion acquisition and feature extraction provided by the present invention; Figure 5 This is a flowchart of the multimodal fusion process provided by the present invention; Figure 6 This is a flowchart of the large language model emotion analysis based on structured Prompt provided by the present invention; Figure 7 This is a schematic diagram of the structure of the autism emotional ability assessment system based on EEG and visual emotional characteristics provided by the present invention. Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] Figure 1 This is a flowchart illustrating the autism emotional capacity assessment method based on EEG and visual emotional characteristics provided in this embodiment of the invention. Figure 1 As shown, it includes: Step 100: Collect EEG signal data and facial video data of autistic subjects; Step 200: Preprocess the EEG signal data and extract emotional features to obtain an EEG emotional feature vector; Step 300: Perform facial expression recognition and emotional arousal calculation on the facial video data to obtain a visual emotion feature vector; Step 400: Filter multimodal time segments based on emotional arousal, retain high-arousal emotional segments, and construct an effective set of emotional features; Step 500: Input the set of effective emotional features into the structured Prompt inference model for emotional semantic analysis and generate a subject's emotional ability analysis report; Step 600: Based on the subject's emotional ability analysis report, output the subject's emotional ability assessment results.

[0022] Specifically, the specific execution steps of this embodiment of the invention are as follows: Step 1: Collect electroencephalogram (EEG) signal data from autistic subjects. and facial video data .

[0023] Step 2, analyze the electroencephalogram (EEG) signals. Preprocessing and emotion feature extraction are performed, including frequency band energy calculation and brain region functional connectivity analysis, to obtain EEG emotion feature vectors. .

[0024] Step 3, process the facial video data Facial expression recognition and emotional arousal calculation are performed. Facial expression features are extracted and emotional arousal sequences are calculated to obtain visual emotion feature vectors. .

[0025] Step 4: Filter multimodal time segments based on emotional arousal, retain high-arousal emotional segments, and construct an effective set of emotional features. .

[0026] Step 5, set the effective emotional features The structured Prompt inference model is used for emotion semantic analysis to generate a report on the subject's emotional capacity analysis. .

[0027] Step 6, based on the emotional competence analysis report Output the results of the subject's emotional capacity assessment. The scores include scores for emotion recognition ability, emotion understanding ability, and emotion regulation ability, and are fed back to the assessment system or relevant users.

[0028] This invention combines EEG signal analysis with visual emotional behavior feature extraction, using multimodal feature modeling and large language model inference as the core assessment mechanism. After receiving EEG data and facial video data from autistic subjects, it first extracts emotion-related features from the EEG signals and video data respectively. Then, it filters valid emotional segments through an emotional arousal screening mechanism. Next, it inputs the multimodal features into the Prompt inference model to generate an emotion analysis report. Finally, based on the analysis results, it outputs the subject's emotional ability assessment results in dimensions such as emotion recognition ability, emotion understanding ability, and emotion regulation ability, thereby achieving an objective and quantitative assessment of the emotional abilities of autistic individuals.

[0029] like Figure 2 As shown, the system of the present invention mainly includes a data acquisition module, a data processing module, a multimodal feature analysis module, and a result analysis module.

[0030] The data acquisition module is used to acquire multimodal data generated by the subjects during the experimental tasks. Specifically, subjects wear EEG acquisition devices to record EEG signals, and simultaneously, facial expression video data is acquired through camera equipment. The acquired data includes EEG timing signals and facial video sequences, along with timestamp information, for subsequent data processing and analysis.

[0031] The data processing module is used to preprocess and extract features from the acquired EEG and video data. Specifically, the EEG data is first filtered to remove high-frequency noise and power supply interference, and then independent component analysis is used to remove artifact signals. Subsequently, frequency domain and time-frequency analysis is performed on the cleaned EEG signals to extract energy features of different frequency bands and functional connectivity features of brain regions in order to construct an EEG emotion feature vector.

[0032] At the same time, face detection and facial key point localization are performed on the video data, facial expression features and emotional intensity indicators are extracted through expression recognition algorithms, and emotional arousal sequence is calculated to form a visual emotional feature vector.

[0033] The multimodal feature analysis module is used to fuse and analyze EEG and visual features. First, effective emotional segments are selected based on emotional arousal level to reduce interference from low-emotional-intensity data. Then, the selected multimodal features are input into a structured Prompt inference model for semantic analysis. The model generates an emotional ability analysis report through a Prompt structure consisting of an instruction layer, an input layer, and an output layer.

[0034] The results analysis module generates the final assessment results based on the emotional competence analysis report. Specifically, the system quantifies the emotional competence indicators in the analysis report, calculates scores for emotion recognition ability, emotion understanding ability, and emotion regulation ability, and standardizes the scores. The system then generates a structured assessment report and visualizes the subject's emotional competence level.

[0035] Ultimately, the assessment results can be output to clinical assessment or intervention systems to provide decision support for doctors, rehabilitation therapists, or researchers.

[0036] Based on the above embodiments, step 1 includes: Step 1.1: EEG signal acquisition was performed on autistic subjects using a multi-channel EEG device. The sampling frequency is Hz, number of channels EEG signals are represented as

[0037] in, Indicates the first Channel in time The sampled potential value, This indicates the total number of sampling points. This step ensures the integrity and high spatiotemporal resolution of the collected EEG signals, providing foundational data for subsequent emotion feature extraction.

[0038] Step 1.2: Facial video of the subject is captured using a camera device to obtain video data. The video frame rate is fps, total frames The video resolution is Video data is represented as

[0039] in, Indicates the first Frame image pixel matrix. This step is used to obtain information on changes in the subject's facial expressions and emotional behavior.

[0040] Step 1.3: Synchronize EEG signals With video data This ensures that EEG data and video frames establish a temporal correspondence, i.e.

[0041] in, Indicates the time of brain signals The set of sampled values, This indicates the corresponding video frame. This synchronization operation is used for time alignment during multimodal sentiment analysis.

[0042] Step 1.4: During the acquisition process, preliminary quality control is performed on the EEG signals and video data, including removing abnormal signals, detecting dropped frames and artifacts, to ensure data integrity and reliability.

[0043] Based on the above embodiments, such as Figure 3 As shown, step 2 includes: Step 2.1: Process the collected EEG signals Filtering is performed, including bandpass filtering and notch filtering, to remove high-frequency noise, power supply interference, and low-frequency drift, resulting in filtered EEG signals.

[0044] Step 2.2: Filter the EEG signal Artifact removal, including eye movement artifacts and electromyography artifacts, is performed using independent component analysis (ICA) or other blind source separation algorithms to obtain clean EEG signals.

[0045] Step 2.3: Perform time-frequency analysis on the clean EEG signal, calculate the power of different frequency bands, including... , , , and Frequency band, to obtain the set of frequency band energy characteristics

[0046] Step 2.4: Calculate brain region functional connectivity features, including coherence, phase synchronization index, or functional connectivity matrix, to obtain a set of brain region activity pattern features.

[0047] Step 2.5: Analyze the frequency band energy characteristics Features of functional connection Combined, forming an EEG emotion feature vector.

[0048] The EEG emotion feature vector is used for subsequent multimodal emotion ability assessment.

[0049] Based on the above embodiments, such as Figure 4 As shown, step 3 includes: Step 3.1: Process the video data Face detection and facial landmark localization are performed to obtain a set of facial landmarks for each frame of the image.

[0050] in Indicates the first Spatial coordinates of key points.

[0051] Step 3.2: Calculate facial expression action unit features (AUs) based on facial key points to obtain the facial expression feature set.

[0052] Step 3.3: Calculate the emotional arousal value based on facial expression features. The emotional arousal sequence was obtained.

[0053] Step 3.4: Extract facial features With arousal sequence Combined, forming a visual emotion feature vector.

[0054] This visual emotion feature vector is used to reflect the emotional expression state of the subject.

[0055] Based on the above embodiments, such as Figure 5 As shown, step 4 includes: Step 4.1: Based on the emotional arousal sequence Set threshold The emotional fragments are filtered to obtain a set of valid emotional fragments.

[0056] Step 4.2: Extract multimodal features within the corresponding time window based on the set of valid segments.

[0057] Step 4.3: Perform time window segmentation and feature integration on the effective emotional feature set to form multimodal emotional feature input data for subsequent emotional ability assessment models.

[0058] Based on the above embodiments, such as Figure 6 As shown, step 5 includes: Step 5.1: Construct a structured Prompt template, including an instruction layer, an input layer, and an output layer, where... The instruction layer is used to define model roles and analysis tasks; the input layer is used to input multimodal emotion feature data; and the output layer is used to define the emotion capability analysis report format.

[0059] Step 5.2: Set up effective emotional features Input the structured Prompt template to form a complete model input.

[0060] Step 5.3: Call the large language model to perform semantic analysis on the multimodal emotion features and generate an emotion capability analysis report A.

[0061] Based on the above embodiments, step 6 includes: Step 6.1: Quantify and score the ability indicators of each dimension in the emotional ability analysis report to obtain the emotional ability score vector.

[0062] in This indicates a score for emotion recognition ability. This indicates a score for emotional comprehension ability. This indicates a score for emotion regulation ability.

[0063] Step 6.2: Standardize the scoring vector to normalize the scoring range to 0~100 points.

[0064] Step 6.3: Generate an emotional competence assessment report This includes the scoring results and an explanation of the ability analysis.

[0065] Step 6.4: Present the evaluation results Output to the assessment system or user terminal to provide a basis for clinical assessment or intervention decisions.

[0066] The following describes the autism emotional ability assessment system based on EEG and visual emotional characteristics provided by this invention. The autism emotional ability assessment system based on EEG and visual emotional characteristics described below can be referred to in correspondence with the autism emotional ability assessment method based on EEG and visual emotional characteristics described above.

[0067] Figure 7 This is a schematic diagram of the structure of the autism emotional ability assessment system based on EEG and visual emotional characteristics provided in an embodiment of the present invention, as shown below. Figure 7 As shown, it includes: an acquisition module 71, an EEG processing module 72, a visual processing module 73, a screening module 74, an analysis module 75, and an output module 76, wherein: The acquisition module 71 is used to acquire EEG signal data and facial video data of autistic subjects; the EEG processing module 72 is used to preprocess the EEG signal data and extract emotional features to obtain an EEG emotional feature vector; the visual processing module 73 is used to perform facial expression recognition and emotional arousal calculation on the facial video data to obtain a visual emotional feature vector; the filtering module 74 is used to filter multimodal time segments according to emotional arousal, retain high-arousal emotional segments, and construct an effective emotional feature set; the analysis module 75 is used to input the effective emotional feature set into a structured Prompt inference model for emotional semantic analysis and generate a subject's emotional ability analysis report; the output module 76 is used to output the subject's emotional ability assessment results based on the subject's emotional ability analysis report.

[0068] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute an autism emotional ability assessment method based on EEG and visual emotional features. The method includes: collecting EEG signal data and facial video data of autistic subjects; preprocessing and extracting emotional features from the EEG signal data to obtain an EEG emotional feature vector; performing facial expression recognition and calculating emotional arousal from the facial video data to obtain a visual emotional feature vector; filtering multimodal time segments according to emotional arousal, retaining high-arousal emotional segments, and constructing an effective emotional feature set; inputting the effective emotional feature set into a structured Prompt inference model for emotional semantic analysis to generate a subject emotional ability analysis report; and outputting the subject's emotional ability assessment result based on the subject's emotional ability analysis report.

[0069] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0070] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the autism emotional ability assessment method based on EEG and visual emotional features provided by the above methods. The method includes: collecting EEG signal data and facial video data of autistic subjects; preprocessing and extracting emotional features from the EEG signal data to obtain an EEG emotional feature vector; performing facial expression recognition and emotional arousal calculation on the facial video data to obtain a visual emotional feature vector; filtering multimodal time segments according to emotional arousal, retaining high-arousal emotional segments and constructing an effective emotional feature set; inputting the effective emotional feature set into a structured Prompt inference model for emotional semantic analysis to generate a subject emotional ability analysis report; and outputting the subject's emotional ability assessment result based on the subject's emotional ability analysis report.

[0071] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0072] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics, characterized in that, include: Collect electroencephalogram (EEG) signal data and facial video data from autistic subjects; The EEG signal data is preprocessed and emotional features are extracted to obtain an EEG emotional feature vector; Facial expression recognition and emotional arousal calculation are performed on the facial video data to obtain a visual emotion feature vector; Multimodal time segments are filtered based on emotional arousal, retaining high-arousal emotional segments and constructing an effective set of emotional features; The effective set of emotional features is input into the structured Prompt inference model for emotional semantic analysis, generating a report on the subject's emotional capacity analysis. Based on the subject's emotional capacity analysis report, the subject's emotional capacity assessment results are output.

2. The method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics according to claim 1, characterized in that, Electroencephalogram (EEG) data and facial video data were collected from autistic subjects, including: A multi-channel EEG device was used to acquire electroencephalogram (EEG) signal data of autistic subjects, and the sampling frequency and number of channels were determined. The EEG signal data included the sampling potential value of any channel at any time point. Facial video of autistic subjects is captured using camera equipment to obtain facial video data, and the video frame rate, total number of frames, and width and height of the video resolution are determined. The facial video data includes the pixel matrix of any frame image. The EEG signal data and the facial video data are synchronized, abnormal signals are removed, and dropped frames and artifacts are detected, so that the EEG signal data and the facial video data correspond in the time dimension.

3. The method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics according to claim 1, characterized in that, The EEG signal data is preprocessed and emotional features are extracted to obtain an EEG emotional feature vector, including: The EEG signal data is subjected to bandpass filtering and notch filtering to obtain filtered EEG signals; The filtered EEG signal is subjected to eye movement artifact removal and electromyography artifact removal, and clean EEG signal is obtained by using independent component analysis or other blind source separation algorithms. Time-frequency analysis was performed on the clean EEG signal to calculate the power of different frequency bands and obtain a set of frequency band energy characteristics; Calculate the functional connectivity features of brain regions, including coherence, phase synchronization index, or functional connectivity matrix, to obtain a set of brain region activity pattern features; The frequency band energy feature set is combined with the brain region activity pattern feature set to form the EEG emotion feature vector.

4. The method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics according to claim 1, characterized in that, Facial expression recognition and emotional arousal calculation are performed on the facial video data to obtain a visual emotion feature vector, including: Face detection and facial key point localization are performed on the facial video data to obtain a set of facial key points for each frame of the image. The set of facial key points includes the spatial coordinates of any key point. Based on the set of facial key points, facial expression action unit features are calculated to obtain a set of facial expression features; Based on the set of facial expression features, the emotional arousal value is calculated to obtain the emotional arousal sequence; The set of facial expression features is combined with the emotional arousal sequence to form a visual emotional feature vector.

5. The method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics according to claim 4, characterized in that, Multimodal time segments were filtered based on emotional arousal levels, retaining high-arousal emotional segments and constructing an effective set of emotional features, including: A threshold is determined based on the emotional arousal sequence, and emotional segments are filtered to obtain a set of valid emotional segments; The effective emotional feature set is obtained by extracting multimodal features within the corresponding time window based on the effective fragment set; The effective set of emotional features is segmented into time windows and integrated to form multimodal emotional feature input data.

6. The method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics according to claim 1, characterized in that, The set of effective emotional features is input into a structured Prompt inference model for emotional semantic analysis, generating a report on the subject's emotional capacity analysis, including: Construct a structured Prompt template, including an instruction layer, an input layer, and an output layer. The instruction layer is used to define the model roles and analysis tasks, the input layer is used to input multimodal emotion feature data, and the output layer is used to define the emotion capability analysis report format. The effective set of emotional features is input into the structured Prompt template to form a complete model input; A large language model is invoked to perform semantic analysis on multimodal emotional features, generating an emotional ability analysis report for the subject.

7. The method for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics according to claim 1, characterized in that, Based on the subject's emotional capacity analysis report, the subject's emotional capacity assessment results are output, including: The ability indicators of each dimension in the subject's emotional ability analysis report are quantitatively scored to obtain an emotional ability score vector, which includes an emotion recognition ability score, an emotion understanding ability score, and an emotion regulation ability score. The emotional ability scoring vector is standardized to normalize the scoring range to a preset scoring range; Generate an emotional competence assessment report, including scoring results and competence analysis explanations; The emotional competence assessment report is output to the assessment system or user terminal to provide a basis for clinical assessment or intervention decisions.

8. A system for assessing the emotional abilities of autistic individuals based on EEG and visual emotional characteristics, characterized in that, include: The acquisition module is used to collect electroencephalogram (EEG) signal data and facial video data of autistic subjects; The EEG processing module is used to preprocess the EEG signal data and extract emotional features to obtain an EEG emotional feature vector. The visual processing module is used to perform facial expression recognition and emotional arousal calculation on the facial video data to obtain a visual emotion feature vector. The filtering module is used to filter multimodal time segments based on emotional arousal, retaining high-arousal emotional segments and constructing an effective set of emotional features; The analysis module is used to input the set of effective emotional features into the structured Prompt inference model for emotional semantic analysis and generate a report on the subject's emotional ability analysis. The output module is used to output the subject's emotional ability assessment results based on the subject's emotional ability analysis report.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the autism emotional ability assessment method based on EEG and visual emotional characteristics as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the autism emotional ability assessment method based on EEG and visual emotional characteristics as described in any one of claims 1 to 7.

Images