User category determination method based on auditory brainstem reaction normal form and source positioning

Abstract

The invention relates to the technical field of electroencephalogram signal monitoring and processing, in particular to a user category determination method based on an auditory brainstem reaction normal form and source localization, and the method comprises the following steps: obtaining auditory brainstem reaction data and electroencephalogram data when a target user is stimulated by external sound; based on a feature extraction module, feature extraction is carried out on source estimation data corresponding to the auditory brainstem response data and the electroencephalogram data; performing feature fusion on the basis of the features extracted from the auditory brainstem reaction data and the features extracted from the source estimation data, and determining fusion features; and determining the category of the target user based on the fusion features and a user category prediction model. According to the scheme for carrying out early diagnosis and screening on the autism infants based on the auditory brainstem reaction data and the electroencephalogram data, early autism (ASD) screening of the infants can be finally achieved under the support of the electroencephalogram equipment and auditory brainstem reaction testing.

Description

Technical Field

[0001] The present application relates to the technical field of electroencephalogram (EEG) signal monitoring and processing, and in particular to a method for determining user categories based on an auditory brainstem response paradigm and source localization. Background Art

[0002] Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by social communication disorders, narrow interests and repetitive behaviors.

[0003] In recent years, the prevalence of ASD has increased significantly. According to a 2023 report from the US CDC, the incidence of ASD has reached approximately 1 / 36. Early identification and intervention are recognized as key factors in improving the prognosis and quality of life of children.

[0004] However, existing ASD screening methods mainly rely on behavioral observations and questionnaire assessments of parents, which are highly subjective and difficult to effectively screen newborn babies. Summary of the Invention

[0005] One objective of this application is to propose a method for determining user categories based on the auditory brainstem response paradigm and source localization. This application determines fusion features based on the fusion of auditory brainstem response data and EEG data source localization features, and combines them with a user category prediction model to predict the category of the target user. This application provides a method for determining user categories based on the auditory brainstem response paradigm and source localization, characterized in that the method includes:

[0006] Acquire auditory brainstem response data and EEG data of the target user when they are stimulated by external sounds;

[0007] Based on the feature extraction module, feature extraction is performed on the source estimation data corresponding to the auditory brainstem response data and the electroencephalogram data respectively;

[0008] performing feature fusion based on the features extracted from the auditory brainstem response data and the features extracted from the source estimation data to determine a fusion feature;

[0009] The category of the target user is determined based on the fusion features and the user category prediction model.

[0010] In some embodiments, the feature extraction module includes an auditory brainstem response feature extraction unit, a source localization unit, a whole-brain feature extraction unit, and a feature fusion unit;

[0011] The feature extraction module extracts features from the auditory brainstem response data and the source estimation data corresponding to the electroencephalogram data, respectively, and performs feature fusion based on the features extracted from the auditory brainstem response data and the features extracted from the source estimation data to determine a fusion feature, including:

[0012] Determining a first feature based on the auditory brainstem response feature extraction unit and the auditory brainstem response data; the first feature is a feature of the auditory brainstem response data;

[0013] determining the whole-brain source estimation data based on the source localization unit and the EEG data;

[0014] Determining a second feature based on the whole-brain feature extraction unit, the whole-brain source estimation data, and the brain region of interest; wherein the second feature is a feature of the source estimation data of the brain region of interest;

[0015] The first feature and the second feature are fused based on the feature fusion unit.

[0016] In some embodiments, the brain region of interest is a portion of all brain regions related to the screening target;

[0017] The determining of a second feature based on the whole-brain feature extraction unit, the whole-brain source estimation data, and the brain region of interest includes:

[0018] determining features of whole-brain source estimation based on the whole-brain source estimation data and a brain region of interest;

[0019] Based on the whole-brain range feature extraction unit, feature extraction is performed on the feature of the whole-brain range source estimation to obtain a second feature.

[0020] In some embodiments, the brain region of interest includes at least one of Hersch's gyrus, the fusiform gyrus, the superior frontal gyrus, and the inferior frontal gyrus.

[0021] In some embodiments, obtaining auditory brainstem response data and electroencephalogram (EEG) data of the target user when the target user is stimulated by external sound includes:

[0022] Get external sound material;

[0023] Randomly arranging the order in which the external sounds are played based on the external sound materials;

[0024] Based on the collected and randomly arranged external sound materials, they are played to the target user, and the auditory brainstem response data and the electroencephalogram data corresponding to the target user are collected.

[0025] In some embodiments, the external sound material is a sound of different emotions of a user associated with the target user, and the emotion includes at least one of a neutral emotion, a happy emotion, and a sad emotion.

[0026] In some embodiments, the step of constructing the user category prediction model includes:

[0027] Obtaining a user's category label, and auditory brainstem response data and electroencephalogram (EEG) data of the user when the user is stimulated by external sound;

[0028] Determining, based on a feature extraction module, a first feature corresponding to the auditory brainstem response data;

[0029] Determining source estimation data of the brain region of interest based on the feature extraction module, the EEG data, and the brain region of interest, and extracting a second feature corresponding to the source estimation data of the brain region of interest;

[0030] fusing the first feature and the second feature to determine the fused feature;

[0031] A user category prediction model is trained based on the fused features and the category of the user; the user category prediction model is a mapping relationship between the category of the user and the fused features.

[0032] In some embodiments, the screening target includes whether the target user suffers from autism, and the category of the target user includes one of: the target user suffers from autism and the target user does not suffer from autism.

[0033] In some embodiments, the method further comprises:

[0034] Obtain the head structure image corresponding to the target user;

[0035] constructing a head model of the target user and a brain source space to be estimated based on the head structural image of the target user;

[0036] Determining a forward model for source localization based on the electrode layout of the EEG acquisition device, the head model of the target user, and the brain source space to be estimated; wherein the forward model for source localization is used to characterize the physical relationship between the brain source space to be estimated and the EEG data mapped through the head model;

[0037] The determining of whole-brain source estimation data comprises:

[0038] The EEG data of the target user is obtained, the EEG data is solved based on an inverse solution model and a forward model of the source localization, and a source estimation result of a source space corresponding to the EEG data is determined.

[0039] In some embodiments, the method further comprises:

[0040] Acquiring the target user's EEG data includes acquiring the target user's EEG data through a contact EEG acquisition device or monitoring the user's EEG data through a non-contact EEG device;

[0041] Monitor the user's EEG data through non-contact EEG equipment, including:

[0042] Obtain the head image of the target user at the current moment;

[0043] Determining the head posture of the target user corresponding to the current moment based on the head image corresponding to the current moment;

[0044] Based on the head posture at the current moment and the head posture corresponding to the previous moment, the posture of the EEG cap is determined, wherein the EEG cap includes multiple brain electrodes, and the multiple brain electrodes monitor the EEG data of the target user in the posture corresponding to the EEG cap.

[0045] The present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method provided in any one of the above embodiments.

[0046] The present application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of a method for determining a user category based on an auditory brainstem response paradigm and source localization provided in any of the above embodiments.

[0047] The present application also provides a computer program product having a program or instruction stored thereon, which, when executed by a processor, implements the steps of a method for determining a user category based on an auditory brainstem response paradigm and source localization provided in any of the above embodiments.

[0048] This application collects auditory brainstem response data and EEG data of the target user when he is stimulated by external sound, determines the characteristics of the auditory brainstem response data and the characteristics of the EEG data source positioning results, performs feature fusion and determines the fusion features, and determines the category of the target user based on the fusion features and the user category prediction model, thereby enabling more accurate diagnosis, earlier intervention on the target user, and achieving better results.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0051] Figure 1 1 is a flow chart of a method for determining a user category based on an auditory brainstem response paradigm and source localization provided in one embodiment of the present application;

[0052] Figure 2 is a schematic diagram of a method for determining a user category based on an auditory brainstem response paradigm and source localization provided in another embodiment of the present application;

[0053] Figure 3 This is a schematic diagram of device placement provided in one embodiment of the present application;

[0054] Figure 4 Schematic diagram of the emotional auditory brainstem response paradigm provided in one embodiment of the present application;

[0055] Figure 5 is a schematic diagram of an auditory brainstem response signal provided in another embodiment of the present application;

[0056] Figure 6 The UNC-Infant standard brain provided in one embodiment of the present application;

[0057] Figure 7 This is a schematic diagram of a forward model of source localization provided by an embodiment of the present application;

[0058] Figure 8 This is a schematic diagram of abnormal brain regions in children with ASD provided in one embodiment of the present application;

[0059] Figure 9 It is a structural diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] The technical solutions between the various embodiments of the present invention can be combined with each other, but they must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0062] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0063] See also Figure 1 The present application provides a method for determining user categories based on an auditory brainstem response paradigm and source localization, the method comprising:

[0064] S101, obtaining auditory brainstem response data and electroencephalogram (EEG) data of a target user when the target user is stimulated by external sound.

[0065] In some embodiments, the auditory brainstem response (ABR) test device can be used to obtain the auditory brainstem response data of the target user when being stimulated by external sounds. The influence of other noises can be minimized by combining headphones and earplugs. The standardization and accuracy of the device wearing should be strictly checked before the experiment.

[0066] In some embodiments, EEG data can be collected using EEG equipment when the target user is exposed to external sound stimulation. A combination of headphones and earplugs can be used to minimize the impact of other noise. The correct fit and accuracy of the equipment must be strictly checked before the experiment.

[0067] In some embodiments, the target user needs to be exposed to external sound stimulation. External sound stimulation is achieved through an experimental paradigm module. The experimental paradigm module includes three units: a human voice collection unit, a trial randomization unit, and a sound playback unit.

[0068] The voice collection unit is responsible for collecting external sounds, which are sounds expressing different emotions associated with the target user. Emotions include at least one of neutral, happy, and sad. For example, the voice of an infant's mother can be collected. The mother pronounces the sound / da / and switches between three different emotions (neutral, happy, and sad) as unique experimental material for each infant.

[0069] The trial randomization unit is used to randomly arrange the collected sounds of different emotions. This is to simulate external sound stimulation in real situations and eliminate the influence of irrelevant factors on neural responses, thereby obtaining realistic data. In this example, the trial duration is set to 1 second, that is, the average interval time (ISI) between the end of one trial and the beginning of the next trial is 1 second for pseudo-randomization.

[0070] Among them, the sound playing unit is responsible for playing the collected and randomized external sounds to the target user.

[0071] In some embodiments, the target user can be an infant or a test subject of other age groups. For infants, early identification and intervention of autism spectrum disorders are recognized as key factors in improving the prognosis and quality of life of infants. This embodiment can be used for early screening of infants with autism.

[0072] Specifically, the target user first wears the device, combining the traditional ABR test equipment placement scheme with the high-density EEG equipment placement scheme, and combining headphones and earplugs to minimize the influence of other noise. The mother's voice is collected, and the mother's voice is pronounced in the form of "da" and switches between three different emotions (neutral, happy, and sad) as unique experimental material for each infant. The three different emotional sounds of the mother collected are randomly selected and played to the infant, with the interval time (ISI) from the end of each trial to the beginning of the next trial being one second to eliminate the influence of irrelevant factors.

[0073] In some embodiments, a data acquisition module is included to obtain the EEG data of the infant during the experiment.

[0074] S102: Based on a feature extraction module, feature extraction is performed on the auditory brainstem response data and the electroencephalogram data respectively.

[0075] Specifically, the feature extraction module is used to extract two types of EEG signals. This module consists of two units: the auditory brainstem response (ABR) signal extraction unit, which extracts classic auditory brainstem response (ABR) data; and the global signal extraction unit, which extracts EEG data from all electrodes in the entire brain.

[0076] Further based on the feature extraction unit, a first feature corresponding to the auditory brainstem response (ABR) data and a second feature corresponding to the EEG data are provided.

[0077] S103 , performing feature fusion based on the features extracted from the auditory brainstem response data and the features extracted from the electroencephalogram data to determine fusion features.

[0078] Furthermore, the extracted first feature and the second feature are fused by a feature fusion unit to obtain a fused feature. The fused feature contains data information of different dimensions, so that the feature parameters are richer.

[0079] S104: Determine the category of the target user based on the fusion feature and the user category prediction model.

[0080] The fused features are input into a pre-trained user category prediction model to obtain the category of the target user, where the target user's predicted category can be a category of whether the target user has autism. The prediction model is pre-trained.

[0081] In some embodiments, the feature extraction module includes an auditory brainstem response feature extraction unit, a source localization unit, a whole-brain feature extraction unit, and a feature fusion unit;

[0082] The feature extraction module extracts features from the auditory brainstem response data and the electroencephalogram (EEG) data respectively, and performs feature fusion based on the features extracted from the auditory brainstem response data and the features extracted from the EEG data to determine fusion features, including:

[0083] Based on the auditory brainstem response feature extraction unit and the auditory brainstem response data, a first feature is determined; the first feature is a feature of the auditory brainstem response data; based on the source localization unit and the EEG data, the source estimation data of the whole brain is determined; based on the whole-brain feature extraction unit, the source estimation data of the whole brain and the brain region of interest, a second feature is determined; the second feature is a feature of the source estimation data of the brain region of interest; based on the feature fusion unit, the first feature and the second feature are fused.

[0084] Specifically, the feature extraction module is used to extract features from two types of EEG signals (auditory brainstem response data and EEG data). This module consists of five units: an auditory brainstem response feature extraction unit, which is responsible for extracting features from ABR signals based on classical components (waves I to V); a source localization unit, which is responsible for inversely solving the electrode data (EEG data) based on the reference MRI structural image and electrode distribution positions to obtain the activation values ​​of the whole-brain source space to be estimated (whole-brain source estimation data); a whole-brain feature extraction unit, which is responsible for extracting brain regions of interest (ROIs) (such as the fusiform gyrus and Hersch's gyrus) based on the results of source localization in ASD-related fMRI studies; a convolutional neural network unit, which is responsible for further extracting features extracted from the two types of EEG signals; and a feature fusion unit, which is responsible for fusing the features extracted from the two types of EEG signals.

[0085] In some embodiments, the brain region of interest is a portion of all brain regions related to the screening target;

[0086] The determining of a second feature based on the whole-brain feature extraction unit, the whole-brain source estimation data, and the brain region of interest includes:

[0087] Based on the source estimation data of the whole brain and the brain region of interest, the characteristics of the whole-brain source estimation are determined; and based on the whole-brain feature extraction unit, the characteristics of the whole-brain source estimation are subjected to feature extraction to obtain a second feature.

[0088] The whole-brain EEG data were combined with the forward model, and an inverse solution model was constructed using the dSPM method to obtain whole-brain source estimation results. Features of brain regions of interest were extracted, such as activation in the auditory cortex (Herschke's gyrus), the face processing cortex (fusiform gyrus), which is typically underactive in children with ASD, and the cortex (superior and inferior frontal gyri) that processes emotional voices, which are abnormal in children with ASD. These features were used as features (secondary features) for whole-brain source estimation.

[0089] Here, providing source estimation data corresponding to the brain regions related to the screening target and performing feature extraction can make data extraction more targeted, the data volume can be controlled, and the data processing capability can be improved.

[0090] In some embodiments, a forward model and an inverse solution model can be constructed to restore the EEG signal to neural source activity in a given source space to obtain a source estimation result. The EEG data after source estimation (i.e., the source estimation result) has improved spatial resolution and can restore the activity state of specific brain functional areas over time. In this way, the accuracy of subsequent data calculations can be improved based on the source estimation result.

[0091] In the above embodiment, the acquired EEG data is converted into a source estimation result of the whole brain, and then feature extraction can be performed based on the acquired source estimation result to obtain richer information features.

[0092] In some embodiments, a physical model for mapping between EEG data and source estimation data may be pre-constructed, and the source estimation data may be subsequently calculated based on the acquired EEG data using the physical model.

[0093] In some embodiments, determining the whole-brain source estimation data based on the EEG data of the target user includes: obtaining the EEG data of the target user, solving the EEG data based on an inverse solution model and a forward model of the source localization, and determining a source estimation result of the source space corresponding to the EEG data. The forward model determination method includes: obtaining a head structural image corresponding to the target user; constructing a head model of the target user and a brain source space to be estimated based on the head structural image of the target user; determining a forward model of source localization based on the electrode layout of the EEG acquisition device, the head model of the target user, and the brain source space to be estimated; and the forward model of source localization is used to characterize the physical relationship between the brain source space to be estimated and the EEG data through the head model.

[0094] The inverse solution model is shown in the following formula.

[0095] W=eLORETAW(G,λ)

[0096] J=W·X EEG ,J∈R 3m×t

[0097] Among them, eLORETAW is a standard source localization method with good localization accuracy. W represents the inverse model constructed based on the forward model G and the covariance matrix of the EEG data itself, which is used to estimate the activation value of each source point in the source space from the EEG data.

[0098] G represents the forward model composed of a three-layer head model constructed based on the MRI structural image, EEG channel information, and the constructed source space. It is used to describe how each source point in the source space jointly influences and forms the signal observed on each EEG channel.

[0099] J represents the activation map of the source space estimated based on the collected EEG data and the constructed inverse model, where each source point in the source space has an estimated activation value.

[0100] X_EEG represents the collected EEG data.

[0101] In the above embodiment, the EEG data obtained by the EEG acquisition device is used to calculate the source estimation data of the source space, thereby knowing the brain neural source activity of the target user. By combining the EEG data and the source estimation data, rich and anatomically significant features are obtained, thereby achieving more accurate information decoding.

[0102] The brain region of interest includes at least one of Hersch's gyrus, the fusiform gyrus, the superior frontal gyrus, and the inferior frontal gyrus.

[0103] The brain region of interest in the above embodiment is a brain region associated with autism. By analyzing the characteristics of the source estimation data of the above brain region, characteristic parameters related to autism can be obtained, thereby achieving prediction of autism.

[0104] In some embodiments, obtaining the auditory brainstem response data and EEG data of the target user when being stimulated by external sound includes: obtaining external sound material; randomly arranging the playback order of the external sound based on the external sound material; playing the collected and randomly arranged external sound material to the target user, and collecting the auditory brainstem response data and EEG data corresponding to the target user.

[0105] In some embodiments, the external sound material is a sound of different emotions of a user associated with the target user, and the emotion includes at least one of a neutral emotion, a happy emotion, and a sad emotion.

[0106] In some embodiments, the step of constructing the user category prediction model includes:

[0107] Obtaining a user's category label, and auditory brainstem response data and electroencephalogram (EEG) data of the user when the user is stimulated by external sound;

[0108] Determining, based on a feature extraction module, a first feature corresponding to the auditory brainstem response data;

[0109] Determining source estimation data of the brain region of interest based on the feature extraction module, the EEG data, and the brain region of interest, and extracting a second feature corresponding to the source estimation data of the brain region of interest;

[0110] fusing the first feature and the second feature to determine the fused feature;

[0111] A user category prediction model is trained based on the fused features and the category of the user; the user category prediction model is a mapping relationship between the category of the user and the fused features.

[0112] The first feature is the auditory brainstem response (ABR) feature. Data from the auditory pathway structures, including Wave I (auditory nerve), Wave II (cochlear nucleus), Wave III (superior olivary nucleus), Wave IV (extralemmal nucleus), and Wave V (inferior colliculus), are extracted within a fixed time window to serve as the auditory brainstem response (ABR) feature. This is shown in the following formula.

[0113]

[0114] In the above formula, x i Indicates the auditory brainstem response data X extracted within the preset time period ABR characteristics.

[0115] The first and second features are extracted separately using a standard convolutional neural network to obtain deeper features of the two different types of signals. These two features are then fused together to form the overall extracted features of the infant's neural response. See the following formula for details:

[0116] Z ABR =CNN ABR (F ABR )

[0117] Z SRC =CNN SRC (F SRC )

[0118] Z fusion =Concat(Z ABR , Z SRC )

[0119] Among them, Z ABRRepresents the first feature extracted by the CNN module, Z SRC Represents the second feature extracted by the CNN module, Z Fusion Indicates fusion features.

[0120] In some embodiments, the screening target includes whether the target user suffers from autism, and the category of the target user includes one of: the target user suffers from autism and the target user does not suffer from autism.

[0121] For details, see Figure 2 , is a schematic diagram of a method for screening autism patients based on brainstem response and source localization technology provided in one embodiment.

[0122] Step 1: Equipment placement, combining the traditional ABR test equipment placement scheme and the high-density EEG equipment placement scheme, headphones and earplugs are combined to minimize the influence of other noises, as follows Figure 3 shown.

[0123] Step 2: Collect the mother's voice, which is pronounced in the form of / da / and switches between three different emotions (neutral, happy, and sad) as unique experimental materials for each baby.

[0124] Step 3: Randomly play the three different emotional sounds of the mother collected in step 2 to the baby, and randomize the time interval (ISI) from the end of each trial to the beginning of the next trial to eliminate the influence of irrelevant factors. Figure 4 shown.

[0125] Step 4: During the emotional auditory brainstem response paradigm, collect the infant's EEG data and extract two types of signals at different scales: the classic ABR signal X ABR (t) and the whole-brain EEG signal X EEG (t,c).

[0126] Step 5: The ABR data obtained in step 4 are extracted in fixed time windows for the auditory pathway structures such as wave I (auditory nerve), wave II (cochlear nucleus), wave III (superior olivary nucleus), wave IV (extralemmal nucleus) and wave V (inferior colliculus) as ABR features, as follows: Figure 5 shown.

[0127]

[0128] In the above formula, x i Indicates the auditory brainstem response data X extracted within the preset time period ABR characteristics.

[0129] Step 6: Use the International Universal Infant Development Standard Brain (UNC-Infant) and use the standard brain closest to the infant's birth month as the MRI structural image of the infant. Figure 6 As shown. Based on this, the baby's 3-layer head model B is constructed: scalp model S scalp 、External skull model S outer_skull and endoskull model S inner_skull . And construct the infant brain source space S{S1,S2...,S m}.

[0130]

[0131] Step 7: Combine the infant brain structure information obtained in step 6 with the electrode position information marked by the device in step 1 to construct a forward model for source localization, as follows: Figure 7 shown.

[0132] / Step 8: Combine the whole-brain EEG data obtained in step 4 with the forward model obtained in step 7, and use the dSPM method to build an inverse solution model to obtain the whole-brain source estimation results. The activation of the auditory cortex (Herschl's gyrus), the facial processing cortex (fusiform gyrus) that is typically underactive in children with ASD, and the cortex (superior and inferior frontal gyri) that processes emotional voices in children with ASD are extracted as features of the whole-brain source estimation. Figure 8 shown.

[0133] Step 9: The features obtained in step 5 and step 8 are respectively extracted through a standard convolutional neural network to obtain deeper features of the two different types of signals. The features of these two parts are then fused together as the total extracted features of the infant's neural response. ABR Represents the first feature extracted by the CNN module, Z SRC Represents the second feature extracted by the CNN module, Z Fusion Indicates fusion features.

[0134] Z ABR =CNN ABR (F ABR )

[0135] Z SRC =CNN SRC (F SRC )

[0136] Z fusion =Concat(Z ABR ,Z SRC )

[0137] Step 10: Each sample is assigned a label (ASD confirmed or not, or ASD high or low risk) based on the subsequent diagnosis or family ASD diagnosis. The neural response feature vector fused in Step 9 is input into a support vector machine (SVM) classifier to construct a predictive model that can distinguish infants with ASD from typically developing infants. During model training, the structural risk objective function is minimized and a regularization term is introduced to mitigate overfitting, ultimately resulting in an optimal set of support vectors and hyperplane parameters. The trained model outputs a prediction of the infant's ASD status based on the input features.

[0138] The current diagnosis of ASD mostly relies on relatively subjective indicators, and it is difficult to make an objective diagnosis in infants at an early stage. For infants with ASD, if intervention measures can be taken early, the effectiveness of the intervention will be greatly improved. ABR is an auditory evoked process that can be used shortly after the baby is born. The combination of the characteristics carried by its signal itself and the results of the brain area of ​​interest obtained by high-density EEG source localization can help with the early screening of ASD infants. Therefore, the present invention proposes a scheme for early diagnosis and screening of ASD infants based on ABR and EEG source localization technology, in order to achieve a more objective ASD diagnosis shortly after the baby is born.

[0139] This training process, coupled with the patented vocal ABR experimental process, collects infant EEG data. Using the collected infant ABR signals and whole-brain source-localized EEG data, a convolutional neural network and feature fusion mechanism are combined to train a predictive model capable of distinguishing between children with ASD and healthy children. This allows for the prediction of ASD soon after birth.

[0140] Through the scheme of early diagnosis and screening of ASD infants based on ABR and EEG source localization technology proposed in the present invention, early ASD screening of infants can eventually be achieved with the support of EEG equipment and ABR testing.

[0141] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method provided in any one of the above embodiments when executing the computer program.

[0142] A computer program product stores a program or instruction thereon, wherein when the program or instruction is executed by a processor, the steps of the method provided in any one of the above embodiments are implemented.

[0143] In some embodiments, the step of constructing a forward model of source localization between the brain source space to be estimated and the EEG data includes:

[0144] Step 1: Obtain the head structure image corresponding to the target user.

[0145] The head structural image may be an MRI structural image of the target user.

[0146] In some embodiments, the head MRI structural image may be an MRI structural image of the user himself. In other embodiments, the head MRI structural image may also be a head MRI structural image similar to that of the user, for example, an MRI structural image obtained by matching from a standard brain library.

[0147] Step 2: Based on the head structure image of the target user, a head model of the target user and a brain source space to be estimated are constructed.

[0148] An MRI structural image corresponding to the target user is obtained, and a corresponding head model is constructed based on the MRI structural image.

[0149] The head model may include a three-layer head model, a scalp model, an outer skull model and an inner skull model.

[0150] The specific formula is as follows:

[0151]

[0152] Among them, B represents the head model, S scalp Indicates the scalp model (left side), S outer_skull Represents the external skull model (middle), S inner_skull Indicates the internal skull model (right side).

[0153] It can be understood that the accuracy of the constructed head model can be customized.

[0154] An MRI structural image corresponding to the target user is obtained, and a corresponding brain source space to be estimated is determined based on the MRI structural image.

[0155] The specific formula is as follows:

[0156]

[0157] Among them, S represents the brain source space to be estimated, s1, s2...s m Represents the source point in the brain source space.

[0158] Point data in the brain source space represents source points. The cortical source space is constructed based on standard brain or user-defined MRI structural images, with a total of 8196 source points. Note that the spatial resolution and depth of the constructed brain source space can be customized.

[0159] Step three: determining a forward model for source localization based on the electrode layout of the EEG acquisition device, the head model of the target user, and the brain source space to be estimated.

[0160] The forward model of source localization is used to characterize the physical relationship between the brain source space to be estimated and the EEG data through the head model.

[0161] Among them, the dots represent brain electrodes, which are used to collect EEG data. The electrode layout includes but is not limited to the number and position layout of the brain electrodes.

[0162] A forward model for source localization is determined based on the electrode layout of the EEG acquisition device, the head model of the target user, and the brain source space to be estimated.

[0163] The specific electrode layout refers to the following formula:

[0164]

[0165] Among them, e1…e n Represents the spatial position information of several EEG electrodes.

[0166] Through the above steps, the electrode layout of the EEG acquisition device is aligned with the head model space. This establishes how the source space to be estimated is mapped onto the electrodes, and establishes an objective physical model between the electrode layout of the EEG acquisition device and the head model space. Based on this objective physical model, the corresponding source space data can be estimated from the acquired EEG data.

[0167] In some embodiments, the step of acquiring the MRI structural image of the target user includes:

[0168] First, it is queried whether the target user has his or her own MRI structural image. If the target user's own MRI structural image is queried, the queried MRI structural image is used as the target user's MRI structural image; if the target user's own MRI structural image is not queried, the user feature vector of the target user is obtained; the user feature vector of the target user is matched with the standard feature vector in the standard brain library, and the MRI structural image corresponding to the standard brain with the highest matching degree is used as the MRI structural image corresponding to the target user.

[0169] Specifically, if the target user has their own MRI structural image data, their own MRI structural image is used directly. If not, the most similar standard brain in the standard brain database is selected as the target user's standard brain. Based on the obtained standard brain or the user's own MRI structural image, the brain source space to be estimated is constructed.

[0170] In some embodiments, the user feature vector and the standard feature vector include user demographic information and head circumference data.

[0171] Collect the user's demographic information and head circumference data to obtain a feature vector specific to the user. The feature vector includes parameters such as age, gender, race, and head circumference, as shown below:

[0172]

[0173] Establish a standard brain bank, including standard brains of different genders, ages and races (MNI152, fsaverage, UNCinfant, NKI...) (see Figure 5 ), construct the demographic information and head model data of the standard brain for each standard brain record, and obtain the feature vector of each standard brain. The feature vector is as follows:

[0174]

[0175]

[0176] Here, T refers to the standard brain database, which includes feature vectors of different users.

[0177] The user's feature vector is compared with the feature vectors of each standard brain in the constructed standard brain database, and the standard brain with the highest similarity is selected as the standard brain used by the user. If the user has their own MRI structural image data, there is no need to match it in the standard brain database, and their own MRI structural image can be used directly. The formula for matching the standard brain with the highest similarity to the target user in the standard brain database is as follows:

[0178]

[0179] Among them, argmax represents the similarity, f user Represents the feature vector of the target user, f template A feature vector representing a standard brain from the standard library.

[0180] It should be noted that the aforementioned standard brain database includes standard brains of different genders, ages, and ethnicities. For each standard brain, demographic information and head model data are constructed to generate a feature vector for each standard brain. For target users who do not have their own MRI structural images, demographic information and head circumference data are collected to generate a feature vector specific to that user. This feature vector is then compared with the feature vectors of each standard brain in the standard brain database, and the standard brain with the highest similarity is selected as the standard brain for that user.

[0181] In the above embodiment, by establishing a standard brain database and a matching algorithm, the most suitable standard brain template is provided for users who do not have MRI structural images, which has the advantages of reducing costs while maximizing source localization accuracy.

[0182] In some embodiments, based on the source estimation result of the source space and the target user's brain region of interest, EEG data of the target user's brain region of interest is determined.

[0183] In some embodiments, the examination purposes of different target users are different, which can be health examinations or targeted examinations for specific causes of disease, so as to achieve the purpose of cognitive improvement or intervention in pathological conditions. For example, the purpose of neural training can include attention deficit hyperactivity disorder, depression, autism, sleep disorders, etc.

[0184] For different training purposes, the target users have different brain areas of interest. Therefore, according to the training purpose of the target users, the EEG data of the brain area corresponding to the training purpose is extracted and fed back to the users. This can help users know the activity status of the corresponding brain area, and then further intervene, adjust or train the brain area in a targeted manner to achieve the purpose of neurofeedback.

[0185] In some embodiments, determining the target user's brain region of interest includes:

[0186] Step 1: Obtain a brain region segmentation atlas library, wherein the brain region segmentation atlas library includes brain region segmentation atlases that divide the brain into different regions according to one or more principles.

[0187] Step 2: Obtain a whole-brain activation map, which is used to characterize the correlation between the activated brain area and the neural training target.

[0188] Step three: Based on the neural training goal of the target user, the whole-brain activation map is matched with the brain region segmentation map in the brain region segmentation map library, the brain region with the highest activation level is obtained in the brain region segmentation map, and the brain region with the highest activation level is used as the brain region of interest of the target user.

[0189] In some embodiments, the brain can be divided into different brain region segmentation maps based on different segmentation principles. For example, the brain region segmentation map can be divided according to the brain structure, or the brain structure can be divided into different brain region segmentation maps according to the division principle of brain functional areas.

[0190] The cortex is divided into several functional areas based on the brain segmentation map. The following formula is shown:

[0191]

[0192] Among them, the formula represents the brain region segmentation atlas library L atlas Different brain region segmentation maps in R1…R k .

[0193] The whole-brain activation map is used to characterize the activation of the brain source points corresponding to the neural training target. Different neural training targets correspond to different activation of brain source points.

[0194] The whole-brain activation map represents the activated state diagram of the entire brain. In the whole-brain activation map, you can query the correlation between the activated source points and the neural training target. For example, if the neural training target is to improve attention, then the corresponding activated source point in the whole-brain activation map can be found to be the first source point area. If the neural training target is to overcome sleep disorders, then the corresponding activated source point in the whole-brain activation map can be found to be the second source point area.

[0195] Through the whole brain activation map, we can know the source point data activated in the brain under different neural training goals.

[0196] After determining the whole-brain activation map corresponding to the neural training target, it also includes matching the whole-brain activation map with the brain region segmentation map in the brain region segmentation map library, and then obtaining the brain region with the highest activation level in the brain region segmentation map, and using the brain region with the highest activation level as the brain region of interest for the target user. In other words, it is finally achieved to obtain the brain region segmentation map that is most relevant to the training purpose of the target user. Subsequently, the brain region segmentation map can be used as the brain region of interest for the target user's neural training this time. It is only necessary to focus on the neural feedback parameters of this brain region to achieve the neural feedback training goal of this time. And in this process, the brain source point data associated with the neural training target is converted into a specific brain region. The amount of source point data will be very large. By converting the source point data into brain regions, the dimensionality reduction of the data is achieved, and the user's attention is focused on a specific area through the brain region, rather than scattered source points, which can also improve the user's neural feedback intervention effect.

[0197] In some embodiments, the whole-brain activation map includes an area composed of one or more activated source points, and the area composed of the source points is matched with the brain region segmentation map, and the brain region with the largest matching degree is selected in the brain region segmentation map as the brain region of interest of the target user.

[0198] It should be noted that the activated source points in the whole-brain activation map may be scattered in multiple areas, and the sizes and shapes of the multiple areas are different. By matching one or more activated source points in the whole-brain activation map with the brain region segmentation map in the brain region segmentation map library, the brain region with the highest degree of activation (the brain region corresponding to the brain region segmentation map) can be matched in the brain region segmentation map library based on the principle of maximum matching as the user's brain region of interest. Among the selected brain regions of interest, it is the area with the greatest correlation with the neural training target and the area with the highest degree of activation during the training process.

[0199] In some embodiments, the neural training goal based on the target user, matching the whole-brain activation map with the brain region segmentation map in the brain region segmentation map library, obtaining the brain region with the highest activation level in the brain region segmentation map, and using the brain region with the highest activation level as the brain region of interest for the target user, includes: based on the neural training goal of the target user, determining the whole-brain activation map corresponding to the neural training goal based on meta-analysis; matching the whole-brain activation map with the brain region segmentation map in the brain region segmentation map library, obtaining the brain region with the highest activation level in the brain region segmentation map, and using the brain region with the highest activation level as the brain region of interest for the target user.

[0200] In other words, the user's brain region of interest is determined from the brain region segmentation atlas library. The specific determination method is based on the maximum value matching between the activated source points in the whole brain activation map and the brain region segmentation atlas library. The specific formula is as follows:

[0201]

[0202] Among them, a j represents the average activation intensity of the jth brain region; R j represents the jth brain region, which is composed of several source points in the source space; S i represents the i-th source point in a certain brain region; A F represents the meta-analysis activation map obtained from neurosynth; j* represents the index of the brain region with the strongest activation among all brain regions.

[0203] In the above embodiment, a brain segmentation atlas library is established and combined with functional magnetic resonance imaging meta-analysis results to assist in positioning, providing the most appropriate brain functional area selection for a specified cognitive function and user, which has the advantage of providing the best choice for the selection of brain functional areas.

[0204] Real-time source positioning is performed during the EEG data acquisition process to provide users with neurofeedback indicators of specific brain functional areas (brain areas of interest), which has the advantages of improving neurofeedback accuracy and adaptability to different cognitive functions and individuals.

[0205] In some embodiments, based on the EEG data of the target user's brain region of interest, a characteristic value of the brain region of interest is determined as a neurofeedback indicator of the user's attention.

[0206] Real-time source positioning is performed during the EEG data acquisition process to provide users with EEG data of specific brain functional areas (brain areas of interest), and the characteristic values ​​of the brain areas of interest are calculated. The characteristic values ​​are used as the neurofeedback indicators of the users' concern. Users only need to pay attention to the neurofeedback indicators of the local brain areas of interest to achieve neural adjustment and training.

[0207] The characteristic value may be a peak value or a mean value, etc., which is not limited here.

[0208] In the above embodiment, the EEG data collected from the user is converted into the source estimation result of the source space. Considering that the data in the source estimation result of the source space exists in the form of source points, the data volume is large, and it is impossible to locate a clear brain area. Since neural movement is associated with a specific brain area, this application uses meta-analysis to determine the brain area of ​​interest corresponding to the neural training target, and then converts the source estimation result of the user's source space into the EEG data of a specific brain area. The user only needs to pay attention to the data changes in the brain area to achieve neural feedback training.

[0209] In some embodiments, neurofeedback is achieved by providing the user with a specific brain activity waveform and performing neural adjustment training through the waveform. The waveform is relatively abstract and not intuitive, which makes it difficult for the user to obtain the activity state of their own brain more intuitively and to perform neural training. In the above embodiment, the waveform is intuitively expressed as a certain feature of the brain area of ​​interest, that is, the feature of the brain area of ​​interest in this embodiment, and the feature value of interest is calculated as the neurofeedback indicator that the user needs to pay attention to. In this way, the user can intuitively know the activity state of his or her own brain based on the feature value of the area of ​​interest, and then perform targeted neural training to improve the effect of neural training.

[0210] Electroencephalogram (EEG), as a non-invasive brain function detection technology, has been widely used in neuroscience, medical diagnosis, and brain-computer interface due to its high temporal resolution, low cost, and convenient operation. Neurofeedback technology, as a special form of biofeedback training, generally calculates various brain activity indicators based on real-time collected EEG signals as control feedback signals. By observing these feedbacks, users can autonomously adjust their cognition and brain activity state, thereby achieving the purpose of cognitive improvement or pathological intervention. Currently, EEG-based neurofeedback methods have been widely used, such as for intervention in attention deficit hyperactivity disorder, depression, autism, sleep disorders, etc., but there are the following limitations: (1) Feedback positioning accuracy is limited. Most neurofeedback systems are based on electrode channels (such as C3, C4, etc.) for frequency / power feedback, and the spatial resolution is poor; (2) Feedback is not targeted. Neurofeedback usually cannot be clearly mapped to specific brain functional areas, making it difficult to further refine the training goals; (3) The feedback dimension is single, ignoring the spatiotemporal dynamic changes of the neural network of brain activity.

[0211] In order to solve the problem of insufficient spatial resolution of traditional EEG-based neurofeedback methods, in some embodiments, an EEG source localization method can be used to estimate the activity of the brain source space to improve the spatial resolution.

[0212] EEG source localization technology combines structural brain information obtained from magnetic resonance imaging (MRI) with multi-channel EEG data to mathematically model and invert it, estimating neural activity in the cerebral cortex and even subcortical regions. The core principle of EEG source localization technology is that EEG signals measured at a limited number of scalp electrodes are affected by volume conduction effects, resulting in only a mixed representation of neural activity. By constructing forward and inverse solution models, EEG signals are reconstructed into neural activity within a given source space. Source-localized EEG data has improved spatial resolution and can be used to reconstruct the temporal activity of specific brain functional areas (such as the lateral occipital cortex, medial occipital cortex, or ventral occipital cortex). Neurofeedback based on EEG source localization can significantly improve the accuracy of neurofeedback, tailor it to specific cognitive functions and users, and expand the dimensionality of neurofeedback metrics.

[0213] In some embodiments, the method further includes monitoring the user's EEG data using a non-contact EEG device. For example, the target user may be an infant, a toddler, or a vegetative patient who cannot control their own body. Non-contact EEG acquisition devices are not worn directly on the target user's head, making data collection more flexible. When the target user's head posture changes, the corresponding EEG acquisition device's posture can also be adaptively adjusted.

[0214] In this way, for infants and young children who need to undergo motor imagery classification and identification, real-time acquisition of EEG data can be achieved through non-contact EEG acquisition equipment, and then motor imagery classification and identification can be performed on the acquired EEG data.

[0215] A non-contact EEG signal acquisition method is adopted. During the EEG signal acquisition process, the brain electrodes are not directly attached to the scalp. The EEG signals of the monitored brain areas are accurately monitored by adjusting the posture of the brain electrodes.

[0216] In some embodiments, the step of monitoring the user's EEG data through a non-contact EEG device includes: Step 1, obtaining a head image corresponding to the target user at the current moment.

[0217] In some embodiments, target users may include infants or other users who require EEG monitoring.

[0218] In some embodiments, the head image at the current moment may include an image acquired in real time by a depth sensor or other image acquisition device.

[0219] In some embodiments, the image acquisition device can be fixed on a bracket, wherein the bracket is immovable, and the position of the image acquisition device can be adjusted.

[0220] Step 2: determining the head posture of the target user at the current moment based on the head image corresponding to the current moment.

[0221] Among them, the pose includes position and orientation.

[0222] In some embodiments, after obtaining the head image corresponding to the target user at the current moment through the image acquisition device, it also includes performing posture analysis on the head image to determine the head position and head direction of the target user at the current moment.

[0223] Step three: Determine the posture of the EEG cap based on the head posture at the current moment and the head posture corresponding to the previous moment, wherein the EEG cap includes multiple brain electrodes, and the multiple brain electrodes monitor the EEG data of the target user in the posture corresponding to the EEG cap.

[0224] Among them, the EEG cap is similar to a helmet, and brain electrodes can be arranged inside the helmet, and each brain electrode can move horizontally and vertically.

[0225] In some embodiments, the previous moment is earlier than the current moment, and the head posture corresponding to the previous moment is a historical head position and direction calculated based on the head image collected at the previous moment.

[0226] In some embodiments, the EEG cap's position can be determined based on the current head position and the previous N historical head positions. The EEG cap includes multiple EEG electrodes, which are used to monitor the target user's EEG data. N is an integer greater than or equal to 1.

[0227] In some embodiments, the target user may include an infant. When collecting brain data from an infant, the infant's head is prone to uncontrolled movement, which may cause the relative position between the brain electrodes and the infant's scalp to shift, resulting in unstable position of the collected signal. Therefore, during the EEG monitoring process, it is necessary to continuously obtain an image of the infant's head. Based on the head image corresponding to the current moment, the infant's head position and head direction at the current moment are determined. The head posture at the current moment is compared with the historical posture at one or more previous moments to assess the magnitude of the change in the infant's head posture, and then determine whether the position and direction of the brain electrodes need to be adjusted to ensure the accuracy of the infant's brain data monitoring.

[0228] In the above embodiment, when the target user is a newborn or infant, non-contact EEG monitoring can be used to avoid irritation and damage to the skin of the newborn or infant. This non-contact EEG monitoring method prevents the EEG cap from directly contacting the infant's head. However, when the infant's head moves, the electrodes can easily deviate from the EEG monitoring area, resulting in inaccurate EEG data monitoring.

[0229] The EEG monitoring method provided in this application can calculate the similarity between the current head posture and the previous N historical postures based on the current head posture and the previous N historical head postures, so as to determine whether the EEG cap needs to be adjusted and how to adjust the posture of the EEG cap to ensure that the brain electrodes can continuously and accurately monitor the monitored brain area, thereby continuously and stably monitoring the EEG signals of the target user.

[0230] In some embodiments, determining the position of the EEG cap based on the head posture at the current moment and the head posture corresponding to the previous moment includes:

[0231] Step 1: Determine the distance between the head posture at the current moment and the head posture at the previous moment.

[0232] Step 2: In response to the distance being greater than a preset threshold, the current posture of the EEG cap is adjusted based on the head posture at the current moment, and the brain electrodes in the EEG cap continue to monitor the EEG data of the target user in the adjusted posture.

[0233] Step three: in response to the distance being no greater than a preset threshold, the EEG electrodes in the EEG cap continue to monitor the EEG data of the target user in the current posture.

[0234] In some embodiments, the position of the EEG cap can be determined based on the current head posture and the previous N head postures, including: determining the distance between the current head posture and the previous N head postures. If the distance is greater than a preset threshold, it indicates that the target user's head posture has changed significantly and the position of the EEG cap needs to be adjusted synchronously. Therefore, the position of the EEG cap can be adjusted based on the current head posture, and the EEG data of the target user's head can be continuously monitored based on the adjusted EEG cap posture, so that the position of the EEG cap always remains consistent with the target user's head posture.

[0235] In the above embodiment, EEG monitoring is performed on a specific area of ​​the target user's head using the electrodes in the EEG cap. A pre-set EEG electrode deviation threshold is used to calculate the distance between the current head posture and the previous N head postures at the previous moment in real time. If the distance is greater than the preset threshold, the EEG cap position is adjusted based on the previous posture, allowing the EEG cap electrodes to continuously and accurately monitor the specific area of ​​the target user's head. If the distance is less than or equal to the preset threshold, it indicates that the target user's posture is consistent or substantially consistent between the current moment and the previous moment. There is no need to adjust the EEG cap position or the EEG electrodes at the current moment, so EEG monitoring can continue at the current moment based on the EEG cap position at the previous moment. At the next moment, the target user's head posture can continue to be obtained through sensors, and the distance between the current head posture and the previous N head postures can be repeatedly determined. The difference between the current distance and the preset threshold is determined, and the judgment result determines whether to adjust the EEG cap posture.

[0236] It is understandable that the brain electrodes are arranged in the EEG cap, and adjusting the position of the EEG cap also adjusts the position of the brain electrodes.

[0237] In some embodiments, adjusting the current position of the EEG cap based on the head posture at the current moment includes: taking the head posture of the target user at the current moment as the control target of a robotic arm, and controlling the robotic arm to adjust the current position and current direction of the EEG cap so that the EEG cap moves to a position that matches the head position and head direction of the target user at the current moment, and the brain electrodes in the EEG cap collect EEG data of the target user at the adjusted position.

[0238] Specifically, the EEG cap can be placed at the end of a robotic arm, and its position can be controlled by the robotic arm. When it detects that the target user's head posture has changed significantly at the current moment, the robotic arm synchronously adjusts the EEG cap's position to match the target user's head posture at the current moment, enabling continuous and accurate detection of the target user's head EEG data.

[0239] In some embodiments, the step of determining the head pose of the target user includes: determining a set of head edge point coordinates of the target user based on the head image; and determining the head pose of the target user based on the set of head edge point coordinates.

[0240] In some embodiments, the authorized head image can be segmented based on an image segmentation algorithm to remove irrelevant information to obtain a target image, and then edge detection can be performed on the target image using an edge detection algorithm to obtain an edge point coordinate set of the head image, and the head posture of the target user can be determined based on the head edge point coordinate set.

[0241] It is understandable that the edge detection algorithm and the image segmentation algorithm can be existing mature algorithms and are not limited here.

[0242] In some embodiments, the method further includes: determining the head size of the target user based on the head image of the target user; determining the scaling factor of the EEG cap and the layout scaling factor of the brain electrodes in the EEG cap based on the head size of the target user; adjusting the size of the EEG cap based on the scaling factor, and adjusting the coordinate position layout of the brain electrodes in the EEG cap based on the layout scaling factor.

[0243] It is understandable that the head sizes of different target users are different, so different target users have different requirements for the size of the EEG cap. It is very important to set an EEG cap that matches the head size of the corresponding target user. This is the basic condition for obtaining accurate monitoring data.

[0244] In some embodiments, it also includes determining the head size of the target user based on the head image of the target user, and then adjusting the size of the EEG cap based on the head size of the target user so that the size of the EEG cap matches the head size of the target user.

[0245] The brain electrodes are set in the EEG cap, and the position of each brain electrode in the EEG cap can be changed. When the size of the EEG cap changes, the position of each brain electrode in the EEG cap will also change adaptively.

[0246] In some embodiments, the relative position relationship of the brain electrodes in the EEG cap is fixed. The initial layout coordinates of the brain electrodes in the EEG cap can be obtained first, and the target coordinates of the brain electrodes can be obtained by scaling the distance between the brain electrodes based on the scaling coefficient of the EEG cap.

[0247] In this way, the head size of the target user is determined based on the head image of the target user, and then the size of the EEG cap and the position of the brain electrodes in the EEG cap are adaptively adjusted, so that the final adjusted EEG cap and brain electrodes match the target user to monitor and obtain more accurate brain data.

[0248] In some embodiments, the head size of the target user includes a horizontal size of the head and a vertical size of the head;

[0249] The step of determining the head size of the target user includes: determining the ear key points, nose key points, and head key points of the target user based on the head image, wherein the ear key points include left ear key points and right ear key points; determining the horizontal size of the head of the target user based on the ear key points and the head key points; and determining the vertical size of the head of the target user based on the nose key points and the head key points.

[0250] In some embodiments, the head size can be determined based on head key points. For example, after obtaining a head image based on a sensor, a trained key point detection model is used to output the coordinates of the head key points, nose key points, left ear key points, and right ear key points.

[0251] The coordinates of the left ear key point concave and the right ear key point are used with the coordinates of the head edge point set to fit the top of the head curve, and the length of the curve is calculated as the lateral size of the head.

[0252] Calculate the distance between the key point of the nose and the midpoint of the top curve along the surface of the head, and multiply it by 2 to get the longitudinal size of the head.

[0253] The head size of the target user is determined based on the horizontal size and the vertical size of the head of the target user, so as to determine the size of the EEG cap and the position coordinate arrangement of the brain electrodes according to the head size.

[0254] In some embodiments, the method further includes acquiring a head image of the target user based on a depth sensor;

[0255] The step of determining the shooting posture of the depth sensor includes: obtaining a working plane image corresponding to a working plane and a plane normal vector corresponding to the working plane based on the depth sensor, wherein the working plane includes a plane corresponding to a working platform for monitoring the EEG data of the target user; determining the shooting position and shooting direction of the depth sensor based on the on-site observation distance, the working distance of the depth sensor, and the plane normal vector.

[0256] In some embodiments, a head image of the target user may be captured by a depth sensor, and the head pose and head size of the target user may be calculated based on the head image.

[0257] In some embodiments, the posture setting of the sensor is very important. If the posture setting of the sensor is unreasonable, accurate head posture and head size data cannot be obtained from the captured head image, which may cause unreasonable posture of brain electrode monitoring and inaccurate brain data.

[0258] In some embodiments, the method further includes setting a shooting posture of the sensor: first, the sensor collects a working plane image corresponding to the working plane, and determines the shooting posture of the sensor based on the working plane image.

[0259] In some embodiments, after obtaining the working plane image corresponding to the working plane, the method further includes: extracting multiple internal coordinate point sets from the working plane image based on a preset marking template; converting the coordinate system of the multiple internal coordinate point sets into a world coordinate system, and determining the plane normal vector corresponding to the working plane based on the internal coordinate point set in the world coordinate system.

[0260] Specifically, the target user being inspected lies flat on the work platform, and the work platform is photographed by an RGBD sensor to obtain an image of the work plane. The 2D edge detection / image segmentation algorithm is combined with a marker template of known shape and size to obtain the internal point coordinates of n markers on the work plane and the corresponding depth information. In some embodiments, the coordinates of these points can be recorded as C_POINTS(camera_set_1,camera_set_2,…,camera_set_n), and then the 3D coordinates in the camera coordinate system are converted to the world coordinate system W_POINTS(world_set_1,world_set_2,…,world_set_n) of the bracket base using external parameters.

[0261] Using the coordinates of all points in W_POINTS, we perform plane fitting to obtain the plane normal vector N = [nx, ny, nz]. We then set the optimal observation distance d based on the sensor's working distance and field of view. Based on the set of marker points, we calculate the center position of the work plane, P_c = [x_c, y_c, d_c]. The sensor's image capture position is then P1 = P_c + N*d, perpendicular to the work plane of the work platform. Based on this determined position and orientation, we determine the target sensor pose. The robotic arm then adjusts the sensor to this target pose to capture data from the target user's head.

[0262] The present application also provides a non-contact EEG monitoring system, which includes: a support module, a robotic arm module, an image acquisition module, a work platform module, an EEG cap module and a control module; the image acquisition module is installed on the support module, and is used to acquire a head image of a target user, and / or, is used to acquire a work platform image corresponding to the work platform module; the EEG cap module is installed at the end of the robotic arm, and the robotic arm is used to adjust the posture of the EEG cap, and the brain electrodes in the EEG cap are used to monitor the EEG data of the target user; the work platform module, the work platform module is used to carry the target user; the control module, the control module is used to control the robotic arm to adjust the posture of the EEG cap based on the head posture of the target user, and the brain electrodes in the EEG cap are used to monitor the EEG data of the target user.

[0263] It can be understood that the computer device provided in this application can be a server, and its internal structure diagram can be as follows: Figure 9 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the method provided in the present application is implemented.

[0264] Those skilled in the art will understand that Figure 9The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical storage, etc. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0265] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0266] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for determining user categories based on the auditory brainstem response paradigm and source localization, characterized in that: The method comprises: Acquire auditory brainstem response data and EEG data of the target user when they are stimulated by external sounds; Based on the feature extraction module, feature extraction is performed on the source estimation data corresponding to the auditory brainstem response data and the electroencephalogram data respectively; performing feature fusion based on the features extracted from the auditory brainstem response data and the features extracted from the source estimation data to determine a fusion feature; The category of the target user is determined based on the fusion features and the user category prediction model.

2. The method according to claim 1, characterized in that The feature extraction module includes an auditory brainstem response feature extraction unit, a source localization unit, a whole-brain feature extraction unit and a feature fusion unit; The feature extraction module extracts features from the auditory brainstem response data and the source estimation data corresponding to the EEG data, and performs feature fusion based on the features extracted from the auditory brainstem response data and the features extracted from the source estimation data. Determine fusion features, including: Determining a first feature based on the auditory brainstem response feature extraction unit and the auditory brainstem response data; the first feature is a feature of the auditory brainstem response data; determining the whole-brain source estimation data based on the source localization unit and the EEG data; Determining a second feature based on the whole-brain feature extraction unit, the whole-brain source estimation data, and the brain region of interest; wherein the second feature is a feature of the source estimation data of the brain region of interest; The first feature and the second feature are fused based on the feature fusion unit.

3. The method according to claim 2, wherein: The brain region of interest is a part of all brain regions related to the screening target; The determining of a second feature based on the whole-brain feature extraction unit, the whole-brain source estimation data, and the brain region of interest includes: determining features of whole-brain source estimation based on the whole-brain source estimation data and a brain region of interest; Based on the whole-brain range feature extraction unit, feature extraction is performed on the feature of the whole-brain range source estimation to obtain a second feature.

4. The method according to claim 3, wherein: The brain region of interest includes at least one of Hersch's gyrus, the fusiform gyrus, the superior frontal gyrus, and the inferior frontal gyrus.

5. The method according to claim 1, wherein The step of obtaining auditory brainstem response data and electroencephalogram (EEG) data of the target user when the target user is stimulated by external sound includes: Get external sound material; Randomly arranging the order in which the external sounds are played based on the external sound materials; Based on the collected and randomly arranged external sound materials, they are played to the target user, and the auditory brainstem response data and the electroencephalogram data corresponding to the target user are collected.

6. The method according to claim 5, characterized in that: The external sound material is a sound of different emotions associated with the target user, and the emotion includes at least one of a neutral emotion, a happy emotion, and a sad emotion.

7. The method according to claim 1, wherein: The steps of constructing the user category prediction model include: Obtaining a user's category label, and auditory brainstem response data and electroencephalogram (EEG) data of the user when the user is stimulated by external sound; Determining, based on a feature extraction module, a first feature corresponding to the auditory brainstem response data; Determining source estimation data of the brain region of interest based on the feature extraction module, the EEG data, and the brain region of interest, and extracting a second feature corresponding to the source estimation data of the brain region of interest; fusing the first feature and the second feature to determine the fused feature; A user category prediction model is trained based on the fused features and the category of the user; the user category prediction model is a mapping relationship between the category of the user and the fused features.

8. The method according to claim 1, wherein: The screening target includes whether the target user suffers from autism, and the category of the target user includes one of: the target user suffers from autism and the target user does not suffer from autism.

9. The method according to claim 1, characterized in that The method further comprises: Obtain the head structure image corresponding to the target user; constructing a head model of the target user and a brain source space to be estimated based on the head structural image of the target user; Determining a forward model for source localization based on the electrode layout of the EEG acquisition device, the head model of the target user, and the brain source space to be estimated; wherein the forward model for source localization is used to characterize the physical relationship between the brain source space to be estimated and the EEG data mapped through the head model; The determining of whole-brain source estimation data comprises: The EEG data of the target user is obtained, the EEG data is solved based on an inverse solution model and a forward model of the source localization, and a source estimation result of a source space corresponding to the EEG data is determined.

10. The method according to claim 1, characterized in that The method further comprises: Acquiring the target user's EEG data includes acquiring the target user's EEG data through a contact EEG acquisition device or monitoring the user's EEG data through a non-contact EEG device; Monitor the user's EEG data through non-contact EEG equipment, including: Obtain the head image of the target user at the current moment; Determining the head posture of the target user corresponding to the current moment based on the head image corresponding to the current moment; Based on the head posture at the current moment and the head posture corresponding to the previous moment, the posture of the EEG cap is determined, wherein the EEG cap includes multiple brain electrodes, and the multiple brain electrodes monitor the EEG data of the target user in the posture corresponding to the EEG cap.

Images