Methods, devices, software products and electronic equipment for slow wave detection in electroencephalography (EEG)

Abstract

This disclosure provides a method, apparatus, program product, and electronic device for slow-wave detection in electroencephalography (EEG), relating to the field of computer technology. The method includes: acquiring EEG data of a subject and acquiring background description information corresponding to the EEG data; determining an initial feature map based on the EEG data; processing the initial feature map using a visual model to obtain visual features; the visual model is obtained by transfer learning of a general visual model using an EEG dataset; processing the background description information using a natural language model to obtain background semantic features; and determining the slow-wave detection result of the EEG data based on the visual features and the background semantic features. This disclosure improves the efficiency and accuracy of slow-wave detection.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a method, apparatus, program product, and electronic device for slow wave detection in electroencephalography (EEG). Background Technology

[0002] Brain electrical activity manifests as waves of varying frequencies. EEG data typically includes waves of multiple frequencies, such as fast waves and slow waves. Slow waves, including delta waves (0.5-3 Hz) and theta waves (4-7 Hz), have relatively low frequencies. Slow wave patterns in EEG can reflect the functional state of the brain to some extent. Detecting slow waves can aid in diagnosis and treatment, such as in assessing sleep disorders, screening for central nervous system diseases, evaluating brain damage, and screening for mental illnesses. Currently, slow wave detection still primarily relies on manual interpretation of EEG readings. Summary of the Invention

[0003] However, slow-wave detection using manual EEG interpretation has the following problems: the results are affected by the professional experience of the examiners, resulting in low accuracy and reliability, and are prone to misjudgment and missed detection. Furthermore, inconsistent subjective judgment standards among different examiners make it difficult to compare different results. In addition, manual interpretation is inefficient and cannot meet the needs of detecting massive amounts of EEG data.

[0004] In view of the above problems, this disclosure provides a method, apparatus, program product and electronic device for slow wave detection in electroencephalography (EEG).

[0005] According to a first aspect of this disclosure, a method for slow-wave detection in electroencephalography (EEG) is provided. The method includes: acquiring EEG data of a subject and acquiring background description information corresponding to the EEG data; determining an initial feature map based on the EEG data; processing the initial feature map using a visual model to obtain visual features; the visual model is obtained by transfer learning of a general visual model using an EEG dataset; processing the background description information using a natural language model to obtain background semantic features; and determining the slow-wave detection result of the EEG data based on the visual features and the background semantic features.

[0006] According to a second aspect of this disclosure, a slow-wave detection device for electroencephalography (EEG) is provided. The device includes: a data acquisition module configured to acquire EEG data of a subject and acquire background description information corresponding to the EEG data; a visual processing module configured to determine an initial feature map based on the EEG data and process the initial feature map using a visual model to obtain visual features; wherein the visual model is obtained by transfer learning of a general visual model using an EEG dataset; a natural language processing module configured to process the background description information using a natural language model to obtain background semantic features; and a detection result determination module configured to determine the slow-wave detection result of the EEG data based on the visual features and the background semantic features.

[0007] According to a third aspect of this disclosure, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method of the first aspect described above and its possible implementations.

[0008] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of the first aspect and possible implementations thereof by executing the executable instructions.

[0009] The embodiments disclosed herein have the following technical effects: Firstly, this solution provides an automated slow-wave detection scheme, addressing the inefficiency of manual EEG interpretation and improving detection efficiency. It can rapidly process massive amounts of EEG data, meeting the needs of large-scale testing. Secondly, in addition to extracting visual features from EEG data, background descriptive information is introduced. By combining visual and background semantic features, the EEG state of the tested subject can be comprehensively reflected. Subsequent feature processing yields slow-wave detection results, achieving an objective and unified feature extraction and detection logic. This improves the accuracy of slow-wave detection and resolves issues such as reliance on professional experience and inconsistent subjective judgment standards in manual interpretation, enhancing the reliability and comparability of the detection results. Thirdly, this solution possesses good versatility and scalability, adapting to different devices and EEG data formats, and can be widely applied in various scenarios such as clinical monitoring and home testing. Attached Figure Description

[0010] Figure 1 A schematic diagram of a system architecture according to an embodiment of this disclosure is shown.

[0011] Figure 2 A flowchart illustrating a slow-wave detection method in electroencephalography (EEG) according to an embodiment of this disclosure is shown.

[0012] Figure 3A schematic diagram of a lead system according to an embodiment of this disclosure is shown.

[0013] Figure 4 A schematic diagram of a multichannel electroencephalogram (EEG) according to an embodiment of this disclosure is shown.

[0014] Figure 5 A flowchart illustrating an embodiment of visual feature extraction in this disclosure is shown.

[0015] Figure 6 A flowchart illustrating an embodiment of this disclosure for extracting background semantic features is shown.

[0016] Figure 7 A flowchart illustrating a model training method according to an embodiment of this disclosure is shown.

[0017] Figure 8 A schematic diagram of a slow-wave detection process is shown in an embodiment of this disclosure.

[0018] Figure 9 A schematic diagram of a slow-wave detection device in electroencephalography (EEG) according to an embodiment of this disclosure is shown.

[0019] Figure 10 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0020] The principles and spirit of this disclosure are described in detail below with reference to several representative embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0021] The embodiments of this disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Invention Overview

[0023] Traditional slow-wave EEG detection relies on manual interpretation of EEG images, which suffers from low efficiency and difficulty in guaranteeing accuracy and reliability. With the development and application of artificial intelligence technology, deep learning-based EEG data processing solutions have emerged, including the following three types: 1. Using CNNs (Convolutional Neural Networks) to extract features from EEG and perform classification (e.g., sleep state classification); 2. Using temporal networks, such as RNNs and LSTMs, to extract features from EEG and perform classification; 3. Based on features extracted using CNNs, using temporal networks for further feature processing and classification.

[0024] However, Schemes 1 and 2 can only extract features at a single scale, failing to fully capture effective information from the EEG. Scheme 3 also suffers from incomplete or insufficient features (e.g., insufficient EEG initialization embedding). These issues result in low feature quality. Furthermore, the aforementioned schemes have not been applied to slow-wave detection tasks, and feature extraction methods used in other tasks such as sleep state classification may not be suitable for slow-wave detection. Therefore, there is a lack of effective EEG slow-wave detection schemes in the relevant technologies.

[0025] In view of the above problems, this disclosure provides a method, apparatus, program product, and electronic device for slow-wave detection in electroencephalography (EEG). Specifically, it involves acquiring EEG data of a subject and acquiring background description information corresponding to the EEG data; determining an initial feature map based on the EEG data; processing the initial feature map using a visual model to obtain visual features; the visual model is obtained by transfer learning of a general visual model using an EEG dataset; processing the background description information using a natural language model to obtain background semantic features; and determining the slow-wave detection result of the EEG data based on the visual features and the background semantic features. Thus, in a first aspect, it provides an automated slow-wave detection scheme, solving the problem of low efficiency in manual EEG interpretation, improving detection efficiency, and enabling rapid processing of massive amounts of EEG data to meet large-scale detection needs. Secondly, based on the extraction of visual features from EEG data, background descriptive information is also introduced. By combining visual and background semantic features, the EEG state of the tested subject can be comprehensively reflected. Subsequent feature processing yields slow-wave detection results, achieving an objective and unified feature extraction and detection logic, improving the accuracy of slow-wave detection, and resolving issues such as reliance on professional experience and inconsistent subjective judgment standards in manual interpretation, thus enhancing the reliability and comparability of the detection results. Thirdly, this solution possesses good versatility and scalability, adaptable to different devices and EEG data formats, and can be widely applied in various scenarios such as clinical monitoring and home testing.

[0026] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0027] Application Scenarios Overview

[0028] It should be noted that the following application scenarios are shown only to facilitate understanding of the spirit and principles of this disclosure. The implementation of this disclosure is not limited in this respect and can be applied to any applicable scenario.

[0029] The embodiments disclosed herein can be applied to various slow-wave detection scenarios, such as using the detection results to help assess the sleep state of the subject or to provide auxiliary diagnostic information about the subject. The application scenarios are described in detail below in conjunction with the system architecture.

[0030] Figure 1 A schematic diagram of a slow-wave detection system architecture is shown. This system architecture includes a subject 110, an EEG monitoring device 120, and a processing device 130. The EEG monitoring device 120 is used to collect EEG data. In one embodiment, the EEG monitoring device 120 includes probe electrodes 1201, 1202, and 1203 and a main device 1204. When slow-wave detection is required on the subject 110, the probe electrodes 1201, 1202, and 1203 are fixed to the head of the subject 110 to collect EEG signals. The main device 1204 summarizes and processes the EEG signals to obtain raw or pre-processed EEG data. Of course, Figure 1 The EEG monitoring device 120 shown is merely exemplary and may also include various other components such as helmets, hats, bedding, and encapsulated portable detection electrodes, which are not limited in this disclosure.

[0031] The processing device 130 may be a personal computer, server, or similar device with certain data processing capabilities, used to process EEG data. The EEG monitoring device 120 can communicate with the processing device 130, for example, through a wired or wireless communication link, thereby sending the collected EEG data to the processing device 130. The processing device 130 can execute the slow-wave detection method in this exemplary embodiment to process the acquired EEG data and obtain the slow-wave detection results for the test subject 110. In one embodiment, the processing device 130 may include a display screen, which can display the slow-wave detection results and the EEG data.

[0032] In one embodiment, the EEG monitoring device 120 and the processing device 130 can be integrated into the same device. For example, the main device 1204 of the EEG monitoring device 120 can serve as the processing device 130.

[0033] In one implementation, Figure 1The system architecture shown is suitable for portable scenarios. For example, the test subject 110 can use a portable EEG monitoring device 120 to collect EEG data at home, and then send the EEG data to a remote processing device 130 for slow-wave detection via a data transceiver. In this way, the user can understand their own EEG status without leaving home, and can send the obtained slow-wave detection results to a doctor to help the doctor determine the health status of the test subject 110.

[0034] Exemplary methods

[0035] The following is an exemplary description of the slow wave detection method in electroencephalography (EEG) according to the embodiments of this disclosure. Figure 2 An exemplary flow diagram of the slow-wave detection method is shown, which may include steps S210 to S240. The following describes the method in conjunction with one or more specific embodiments and related figures. Figure 2 Each step in the process will be explained in detail.

[0036] refer to Figure 2 In step S210, the EEG data of the test subject is acquired, as well as the background description information corresponding to the EEG data is acquired.

[0037] The subjects being tested are individuals who require sleep monitoring. Electroencephalogram (EEG) data reflects the electrical activity of the cerebral cortex in the subjects.

[0038] In one implementation, the EEG data includes multi-channel data. Channels may refer to the body parts being tested. Figure 3This diagram illustrates the 10-20 international standard lead system. The points on the head include Nz (nasal root), Iz (external occipital protuberance), Fpz (frontal pole midpoint), Fz (frontal midpoint), Cz (central point), Pz (apex), Oz (occipital point), Fp1 (left frontal pole), F3 (left frontal), C3 (left central), P3 (left parietal), O1 (left occipital), T3 (left temporal midpoint), T5 (left posterior temporal), F7 (left anterior temporal), Fp2 (right frontal pole), F4 (right frontal), C4 (right central), P4 (right parietal), O2 (right occipital), T4 (right temporal midpoint), T6 (right posterior temporal), and F8 (right anterior temporal). The scale values ​​in the diagram represent the proportion of the distance between two points to the length of their respective baselines. For example, the baseline for Nz and Iz is the midline of the head; the distances between Fpz and Nz, and between Oz and Iz, are both 10% of the midline of the head; the distances between Fz and Cz, and between Cz and Pz, are both 20% of the midline of the head; the distance between Fp1 and F7 accounts for 10% of the length of the circular baseline Fpz-T3-Oz-T3-Fpz; and the distance between F3 and F7 accounts for 25% of the length of the baseline F7-Fz-F8. Based on the 10-20 international standard lead system, 19 standard electrodes (i.e., 19 channels) can be set, located at: Fp1, Fp2, F3, F4, F7, F8, Fz, C3, C4, Cz, T3, T4, T5, T6, P3, P4, Pz, O1, O2. Reference electrodes (such as left ear electrode A1, right ear electrode A2) and ground electrodes are also set. The electrical signal data (such as potential) of each electrode is collected using an EEG monitoring device. The relative value (such as potential difference) of each standard electrode and the reference electrode (such as the average value of the left ear electrode and the right ear electrode, or, for the standard electrodes on the left hemisphere and the midline of the head, the left ear electrode is used as the reference electrode, and for the standard electrodes on the right hemisphere, the right ear electrode is used as the reference electrode) is calculated to obtain the first lead data of 19 channels.

[0039] In one implementation, the electroencephalogram (EEG) data includes an electroencephalogram (EEG), which is a graphic obtained by amplifying and recording spontaneous bioelectric potentials of the cerebral cortex from the scalp using sophisticated instruments, characterizing the spontaneous, rhythmic electrical activity of brain cell groups. Figure 4 A schematic diagram of a multichannel electroencephalogram (EEG) is shown, which can be a graphical representation of the first lead data of the aforementioned 19 channels. For example, with time as the horizontal axis and potential as the vertical axis, a planar graph representing potentials at different times is used; this planar graph is the EEG. The frequency (period), amplitude, and phase of brain waves constitute the basic characteristics of an EEG. Of course, the above EEG is usually time-domain data, but frequency-domain and time-frequency combined domain EEG data can also be obtained to draw the corresponding EEG.

[0040] After acquiring EEG data and before further processing, preprocessing can be performed. Preprocessing methods include, but are not limited to: Resampling. Resampling can change the sampling rate of a signal, converting a non-uniformly sampled signal into a uniformly sampled signal. For example, EEG data can be resampled at 500Hz.

[0041] Filtering. Methods such as bandpass filtering and notch filtering can be used to reduce interference from non-EEG signals. For example, bandpass filtering of EEG data in the 0.1~70Hz frequency band can be used to eliminate voltage drift and high-frequency noise, followed by 50Hz notch filtering to reduce environmental noise and AC signal interference.

[0042] Data augmentation. For example, raw EEG leads, average leads, left hemisphere leads, and right hemisphere leads can be acquired, and different types of lead data can be combined (e.g., spliced) to obtain augmented EEG data.

[0043] Remove noisy data. Noisy data may be caused by factors such as poor contact (e.g., poor contact between the EEG monitoring device and the subject's body part, poor contact of the device cable, etc.). Removing noisy data can improve data quality and the accuracy of test results. Filtering can reduce some noise. In addition, noisy data can be identified and removed at the data level through analysis. For example, obvious outliers can be removed, or data from channels with more noise can be removed.

[0044] Numerical standardization. Numerical standardization maps data of different modalities and types to the same suitable numerical range for unified processing. For example, given a standard numerical range, EEG data is dedimensionalized and then normalized to that range. This can be achieved by multiplying by a specific coefficient to perform numerical mapping, thus completing the numerical standardization process.

[0045] In one implementation, the EEG data includes first-lead data from multiple channels. After acquiring the EEG data of the subject, preprocessing can be performed as follows: Filter the data from the first lead; Calculate the average value of the filtered first lead data from multiple channels; The difference between the filtered first lead data and the average value of each channel is determined to obtain the second lead data of multiple channels; The data from the first and second leads are combined to obtain preprocessed EEG data.

[0046] This disclosure does not limit the specific filtering method or parameters. For example, high-pass and low-pass filtering of the first lead data can be performed using a frequency of 0.1-70Hz to eliminate voltage drift and high-frequency noise. Subsequently, a 50Hz notch filter is applied to eliminate environmental noise. After filtering, the difference between the filtered first lead data and the average value of the first lead data for each channel is calculated to obtain the second lead data. The first and second lead data are combined to obtain preprocessed EEG data. For example, the dimensions of the preprocessed EEG data are (38, 15000), where 38 represents the 19 channels of the first and second lead data, and 15000 indicates that each channel includes 15000 data points, which can be a time series formed by 15000 data points.

[0047] The data from the first and second leads can be combined to obtain preprocessed EEG data. Alternatively, the data from the first and second leads can be further preprocessed, such as removing noise and standardizing the values, before being combined to obtain preprocessed EEG data.

[0048] By employing the methods described above, filtering and data augmentation were used to add second-lead data to the first-lead data, thereby improving the comprehensiveness of the EEG data. The second-lead data more clearly demonstrates the differences in EEG signals across different channels, making it easier to identify subtle changes or abnormalities in the EEG signals during subsequent processing. This provides a richer and more effective data source for subsequent processing, contributing to improved accuracy in slow-wave detection.

[0049] In one implementation, the EEG data includes brain electrode data and ear electrode data. After acquiring the EEG data of the subject, preprocessing can be performed in the following manner: Determine the first reference electrode data, and determine the unipolar lead data based on the brain electrode data and the first reference electrode data; With the reference electrode changed, the second reference electrode data is determined based on the ear electrode data, and the bipolar lead data is determined based on the brain electrode data and the second reference electrode data. By combining unipolar and bipolar lead data, preprocessed EEG data is obtained.

[0050] In this study, the first reference electrode data is the reference electrode data in the unipolar lead, and the second reference electrode data is the reference electrode data in the bipolar lead; the reference electrodes used in the two are different. The first reference electrode data can be determined by any one or more of the following methods: averaging the data from different brain electrodes; averaging the data from the left and right ear electrodes; averaging the data from different brain electrodes and the two ear electrodes; or selecting a specific brain electrode as the reference electrode and using its data as the first reference electrode data. The difference (e.g., potential difference) between the brain electrode data and the first reference electrode data is calculated to obtain the unipolar lead data.

[0051] When the reference electrode is changed compared to the monopolar lead, second reference electrode data is determined based on ear electrode data. For example, either left ear electrode data or right ear electrode data can be used as the second reference electrode data. The difference between the brain electrode data and the second reference electrode data is calculated to obtain bipolar lead data.

[0052] In one embodiment, the brain electrode data includes left hemisphere electrode data, midline electrode data, and right hemisphere electrode data; the ear electrode data includes left ear electrode data and right ear electrode data. For the left hemisphere and midline, the right ear electrode data is used as a second reference electrode data, and the differences between the left hemisphere electrode data, midline electrode data, and right ear electrode data are calculated respectively. For the right hemisphere, the left ear electrode data is used as a second reference electrode data, and the differences between the right hemisphere electrode data and left ear electrode data are calculated to obtain bipolar lead data. For example, for left hemisphere electrode data, F7'=F7-A2, F3'=F3-A2, C3'=C3-A2, T3'=T3-A2, P3'=P3-A2, T5'=T5-A2, O1'=O1-A2, yielding data for 7 bipolar lead channels in the left hemisphere; for midline EEG data, Fpz'=Fpz-A2, Fz'=Fz-A2, Cz'=Cz-A2, P... z'=Pz-A2, Oz'=Oz-A2, yielding data for 5 bipolar lead channels in the midline; for the right hemisphere electrode data, F8'=F8-A1, F4'=F4-A1, C4'=C4-A1, T4'=T4-A1, P4'=P4-A1, T6'=T6-A1, O2'=O2-A1, yielding data for 7 bipolar lead channels in the right hemisphere; a total of 19 channels.

[0053] In one implementation, for the left hemisphere and midline, the left ear electrode data is used as the second reference electrode data, and the difference between the left hemisphere electrode data, the midline electrode data and the left ear electrode data is calculated respectively. For the right hemisphere, the right ear electrode data is used as the second reference electrode data, and the difference between the right hemisphere electrode data and the right ear electrode data is calculated to obtain bipolar lead data.

[0054] The monopolar and bipolar lead data are combined to obtain preprocessed EEG data. Alternatively, the monopolar or bipolar lead data can be further preprocessed, such as removing noise and standardizing the values, before being combined to obtain preprocessed EEG data.

[0055] By using the above methods, bipolar lead data can be obtained by changing the reference electrode on the basis of unipolar lead. This can more accurately reflect the discharge status of different parts of the brain, enhance the local characteristics of EEG signals, reduce the influence of irrelevant interference, and help improve the accuracy of slow wave detection.

[0056] Background description information is text-based descriptive information related to EEG data that assists in slow-wave detection. It includes, but is not limited to, one or more of the following: basic information about the subject, such as age, gender, and medical history; physiological state information about the subject, such as current diagnosis and medication information; and preliminary descriptions of the EEG data by the doctor, such as "This segment shows low to medium amplitude, good modulation, dominant occipital rhythm, symmetrical left and right channels, and no background slow waves." Background description information can be obtained manually or automatically to ensure accurate correspondence between EEG data and background description information. For example, professional medical staff can fill in the background description information based on the collected EEG data and the subject's actual situation, including key information related to slow-wave detection. Alternatively, hospital information systems, electronic medical record systems, etc., can automatically link the subject's basic information and physiological state information to the corresponding EEG data, and can combine this with brief annotations from the doctor to form a complete background description.

[0057] Continue to refer to Figure 2 In step S220, an initial feature map is determined based on the EEG data, and the initial feature map is processed using a visual model to obtain visual features; the visual model is obtained by transfer learning of a general visual model using the EEG dataset.

[0058] The initial feature map is feature data that represents information in the EEG data in the form of an image. This disclosure does not limit the specific type of the initial feature map. For example, when acquiring EEG data, the corresponding EEG can be acquired, or after acquiring EEG data, an EEG can be drawn based on the EEG data, and the EEG can be used as the initial feature map.

[0059] In one implementation, the initial feature map includes any one or more of a first initial feature map, a second initial feature map, a third initial feature map, and a fourth initial feature map. The determination of the initial feature map based on EEG data includes any one or more of the following methods: ① Using the number of channels in the EEG data as the height and the dimension in each channel as the width, the EEG data is converted into a first initial feature map.

[0060] For example, the preprocessed 38-channel EEG data is obtained, and the number of channels (38) is used as the height (H) of the feature map, the dimension (15000) of each channel is used as the width (W), and the number of channels (C) is set to 1. The EEG data is converted into a single-channel two-dimensional feature map, namely the first initial feature map, with a dimension of 1×38×15000.

[0061] ② The first initial feature map is transformed in dimensions to obtain the second initial feature map; the difference between the height and width of the second initial feature map is less than the difference between the height and width of the first initial feature map.

[0062] For example, to perform dimensionality transformation on the first initial feature map (1×38×15000), a uniform segmentation reconstruction method can be used. The 15000 data points in the width direction are evenly divided, for example, into 20 segments, each with 750 data points. The 38 channels in the height direction are expanded to 760 (38×20), forming a feature map with dimensions of 1×760×750, i.e., the second initial feature map. Alternatively, based on the input size preferred by the visual model, the 1×760×750 feature map can be padded, adding 8 zero-value pixels in the height direction and 18 zero-value pixels in the width direction, resulting in a second initial feature map with dimensions of 1×768×768.

[0063] By employing the above method, the height and width difference of the first initial feature map are narrowed, making the height and width difference of the second initial feature map smaller than that of the first initial feature map. That is, compared to the first initial feature map, the second initial feature map is closer to a square, facilitating processing by the visual model.

[0064] ③ Convert the EEG data into time-frequency features, and use the time dimension and frequency dimension of the time-frequency features as the height and width to convert the time-frequency features into a third initial feature map.

[0065] Electroencephalogram (EEG) data is typically time-domain data, expressing changes in electrical signals over time, and thus provides relatively simple information. Time-frequency transformation (TFD) can convert EEG data to the frequency domain or a combined time-frequency domain, yielding frequency-domain or time-frequency data, which provides information on the signal's distribution in the frequency domain or its combined distribution in the time and frequency domains. TFD methods include, but are not limited to, short-time Fourier transform (SFT) and wavelet transform. A third initial feature map is then obtained based on video features. For example, short-time Fourier transform (STFT) was performed on the preprocessed 38-channel EEG data to extract time-frequency features. The window function was set as the Hanning window with a window length of 256ms, an overlap rate of 50%, and a sampling frequency of 250Hz. The EEG signal of each channel was processed by STFT to obtain a time-frequency matrix (frequency dimension 64, time dimension 1000). The frequency dimension (64) of the time-frequency features was used as the height, the time dimension (1000) as the width, and the number of channels was set to 38. The time-frequency matrices of the 38 channels were spliced ​​together to obtain a third initial feature map with a dimension of 38×64×1000, which integrates time domain and frequency information.

[0066] ④ Obtain the EEG corresponding to the EEG data, and determine the fourth initial feature map based on the EEG.

[0067] When acquiring EEG data, pre-drawn EEG maps can be obtained, or corresponding EEG maps can be drawn based on the EEG data. Using time as the x-axis and signal amplitude as the y-axis, the EEG data from each channel is plotted as a corresponding waveform curve, resulting in an EEG map. This EEG map can be used as a fourth initial feature map, or it can be processed, such as converting it to grayscale or setting it to a standard size, to obtain the fourth initial feature map. For example, the dimensions of the fourth initial feature map can be 1×768×768 or 1×1024×1024.

[0068] In one implementation, the numerical range of different types of initial feature maps can be unified, such as converting the value of each type of initial feature map to a standard numerical range (e.g., 0~1).

[0069] In one implementation, multiple initial feature maps can be obtained from EEG data. The first initial feature map retains the original channel and temporal correlation information of the EEG data; the second initial feature map is more suitable for the input of a visual model, which is beneficial for improving feature extraction efficiency; the third initial feature map integrates time-frequency information; and the fourth initial feature map represents waveform features in image form. These different types of initial feature maps complement each other, comprehensively covering the basic features, structural features, and time-frequency features of the EEG data. In practical applications, different combinations of initial feature maps can be used according to specific needs, such as combining the second, third, and fourth initial feature maps. This combination based on different types of initial feature maps achieves multi-dimensional representation of EEG signals, overcomes the limitations of incomplete information from a single feature map, lays the foundation for subsequent processing, and further improves the accuracy of slow-wave detection.

[0070] A visual model is a machine learning model used to extract visual features. It can be obtained by transferring learning from a general visual model using an EEG dataset. The general visual model can be a pre-trained 2D model, such as EfficientNet or ResNet. The EEG dataset can include sample EEG images and corresponding classification labels (such as labels for EEG abnormalities, waveforms, and sleep). Transfer learning of the general visual model using the EEG dataset can be performed, for example, by fine-tuning the model on the EEG dataset. Specifically, the EEG dataset is divided into a training set and a validation set. Sample EEG images from the training set are input into the general visual model. A portion of the model's intermediate layers (such as the bottom feature extraction layer) can be frozen. The classification layer and intermediate feature extraction layer are fine-tuned. A loss function is constructed based on the difference between the classification result and the classification label. The model is iteratively trained and optimized until its performance on the validation set meets the requirements. The fine-tuning process is then complete, resulting in a visual model adapted for the EEG slow-wave detection task.

[0071] After inputting the initial feature map into the visual model, the visual model can process the initial feature map through convolution, pooling, fully connected layers, etc. The information output by the final visual model can be used as visual features, or the information output by any one or more intermediate layers (such as the information output by the last intermediate layer before the output layer) can be used as visual features.

[0072] In one implementation, if multiple initial feature maps are obtained based on EEG data, each initial feature map can be output to a visual model for processing to obtain the visual features corresponding to each initial feature. These features are then merged through splicing, weighting, or other methods for subsequent processing.

[0073] In one implementation, reference Figure 5 As shown, the above-described method of processing the initial feature map using a visual model to obtain visual features includes the following steps: S510, Extract key descriptive information associated with the initial feature map from the background description information; S520, Determine location indication information based on key description information; the location indication information is used to indicate the location information of the region of interest in the initial feature map; S530: Extract the feature map of interest from the initial feature map based on the location indication information; S540 inputs the initial feature map and the feature map of interest into the visual model to obtain visual features.

[0074] The key descriptive information refers to the background descriptive information that is directly related to the slow wave distribution, morphology, and location of the initial feature map. For example, text semantic parsing of the background descriptive information can be performed by combining a keyword library related to slow wave detection (such as technical terms including occipital region, frontal region, left hemisphere, right hemisphere, delta wave, theta wave, slow wave, abnormal waveform, high amplitude, low amplitude, etc.) for word segmentation and keyword identification. These keywords are then associated with their context to form one or more phrases or sentences with complete semantics. Natural language understanding algorithms are used to determine the semantics of these phrases or sentences, further filtering out phrases or sentences related to slow waves, and removing duplicate and irrelevant information. The resulting integrated key descriptive information is then obtained. For example, the background description information is "This EEG segment was collected during sleep, with good amplitude modulation. A small number of delta waves were visible in the occipital region, and the waveform in the left frontal region was normal. The subject had no history of epilepsy and had not taken any medication." After word segmentation, it was matched with a keyword database to select keywords such as "occipital region," "delta wave," "left hemisphere," and "frontal region." Each keyword was associated with the context to form two key sentences: "A small number of delta waves were visible in the occipital region" and "The waveform in the left frontal region was normal." Further analysis of its semantics determined that the semantics of "A small number of delta waves were visible in the occipital region" were directly related to slow waves, and "A small number of delta waves were visible in the occipital region" was selected as the key descriptive information.

[0075] The region of interest (ROI) is a key area in the initial feature map closely related to slow-wave detection. It can correspond to slow-wave distribution areas (such as the pinnatif and frontal regions) and anomalous waveform regions mentioned in the background description information, and is a region that should be focused on during visual feature extraction. Structured location indication information is generated based on the key description information to indicate the location of the ROI. For example, based on the distribution of parameters in the width and height directions (e.g., time in the width direction and channel in the height direction) in the initial feature map, the mapping relationship between parameters and pixel positions is determined, such as determining the pixel region coordinates corresponding to each channel in the initial feature map. Location-related information in the key description information, such as information about channels and time periods (e.g., "pinnatif"), is parsed and mapped to pixel region coordinates in the initial feature map to obtain the location indication information. The location indication information can adopt a standardized format of "region name or number + coordinate range".

[0076] Based on the location indication information, the region of interest (ROI) is cropped from the initial feature map to obtain the feature map of interest (MOI). The MOI can be adjusted to a uniform size, such as maintaining the same aspect ratio as the initial feature map. Alternatively, since the original size of the MOI is usually smaller than the initial feature map, it can be upsampled to match the size of the initial feature map, facilitating subsequent uniform processing. Both the initial feature map and the MOI are input into the visual model. The visual model can extract comprehensive features at various scales from the initial feature map and extract key features of the ROI from the MOI. The two feature sets are then merged using methods such as concatenation and weighting to obtain the visual features.

[0077] based on Figure 5 Compared to extracting visual features solely from the initial feature map, this method, by incorporating a feature map of interest (MOI), guides the visual model to focus on regions in the initial feature map related to slow-wave activity. This increases the attention given to these regions and enhances feature extraction, resulting in extracted visual features that encompass comprehensive features across all regions and scales in the initial feature map, as well as key features from the MOI. This balances comprehensiveness and specificity of visual features, providing a higher-quality feature data source for subsequent slow-wave detection. Furthermore, it achieves linkage between the initial feature map and background description information at the initial data or feature level. Guided by key descriptive information in the background description, the MOI is extracted from the initial feature map, making the visual features more closely aligned with the background description, improving the rationality and reliability of the visual features, and promoting high-quality deep fusion of subsequent visual features with background semantic features.

[0078] Continue to refer to Figure 2 In step S230, the background description information is processed using a natural language model to obtain background semantic features.

[0079] Among them, background semantic features are structured data extracted from background description information that contains semantic information related to slow-wave detection. Natural language models are machine learning models capable of processing natural language text. They can transform unstructured background description information into structured background semantic features, adapting to the textual attributes of the background description information. Natural language models include, but are not limited to, traditional natural language processing models such as Large Language Models (LLM) and BERT (Bidirectional Encoder Representations from Transformers).

[0080] Background description information can be directly input into a natural language model, or it can be preprocessed (e.g., removing redundant information, standardizing text formatting, etc.) before being input. The natural language model can encode the input information, extract features, and output background semantic features.

[0081] In one implementation, the natural language model includes a large language model. The above-described processing of background description information using a natural language model to obtain background semantic features includes the following steps: Generate prompt information based on the background description and prompt template; The prompt information is input into a large language model to obtain background semantic features.

[0082] The prompt template is a pre-constructed text template that describes the task information for extracting background semantic features and includes fillable slots for filling in the background description information. For example, the prompt template might read, "You are an EEG clinical expert, proficient in EEG-related clinical knowledge. The background activity information of this patient's EEG segment is: background description information, extract useful slow-wave text feature information…". Filling the prompt template with the background description information yields the prompt information. This prompt information is then input into a large language model, which, after inference, outputs the background semantic features. The large language model possesses strong professional text understanding capabilities, accurately identifying EEG and slow-wave related terms, thereby effectively extracting background language features, reducing feature extraction errors, and improving processing efficiency.

[0083] In one implementation, reference Figure 6 As shown, the above-mentioned use of natural language models to process background description information and obtain background semantic features includes the following steps: S610, classify the waveforms in the initial feature map to obtain the waveform classification results; S620: Obtain waveform description information corresponding to the waveform classification results from a pre-built knowledge base; S630 combines background description information and waveform description information and inputs them into a natural language model to obtain background semantic features.

[0084] Waveform classification is based on features such as frequency, amplitude, and morphology of EEG waves in the initial feature map. This can be achieved by statistically analyzing indicators like frequency and amplitude, or by using a machine learning model for waveform classification. The knowledge base pre-stores waveform descriptions corresponding to different waveform categories. These descriptions, presented in natural language, specifically describe the waveform's morphology or related parameter features. The waveform descriptions complement the background descriptions; combining them and inputting them into the natural language model allows the model to learn more comprehensive and sufficient background semantic features.

[0085] For example, multiple waveform categories and corresponding waveform descriptions for each category are set based on expert experience and stored in a knowledge base. A waveform classification model, such as ViT (Visual Transformer), is pre-built and trained. After determining the initial feature map based on EEG data, the initial feature map is input into the waveform classification model to obtain the waveform classification result, which indicates that the waveform in the initial feature map belongs to a certain waveform category. The waveform description information corresponding to that waveform category is retrieved from the knowledge base. The background description information and waveform description information are combined to fill the prompt template to form the prompt information. The prompt information is input into a large language model to obtain background semantic features.

[0086] based on Figure 6 This method, by combining the waveform description information corresponding to the waveform classification results of the initial feature map, supplements the waveform details that may be missing in the background description information, making the background description information more comprehensive. This results in higher quality extracted background semantic features. Furthermore, it enhances the correlation between background semantic features and visual features, enabling better mining of intermodal relationships during their fusion, improving the quality of the fused features, and thus enhancing the accuracy of slow-wave detection.

[0087] Continue to refer to Figure 2 In step S240, the slow wave detection result of the EEG data is determined based on visual features and background semantic features.

[0088] For example, the extracted visual features and background semantic features can be combined and linearly processed to obtain the slow wave detection result. This disclosure does not limit the specific content or form of the slow wave detection result. For example, basic slow wave detection results (such as whether slow waves are present or not) can be output according to specific needs, or detailed slow wave detection results (such as the type of slow wave, distribution area, amplitude range, and whether there are pathological abnormalities) can be output.

[0089] In one implementation, determining the slow-wave detection result of EEG data based on visual features and background semantic features includes the following steps: A multimodal fusion model is used to fuse visual features and background semantic features to obtain fused features; The fusion features are processed using a classification model to obtain slow-wave detection results.

[0090] Multimodal fusion models refer to machine learning models that can process and fuse features from two or more different modalities (such as visual modal features, text modal features, etc.), such as cross-attention models and gating models. Multimodal fusion models can align and fuse visual features and background semantic features, outputting fused features. These fused features contain key information from both modalities. Classification models refer to machine learning models that can perform dimensionality reduction, inference, and classification on the fused features, such as multilayer perceptrons (MLP) and support vector machines (SVM). The fusion model is input into the classification model, processed through a fully connected layer, and outputs the slow-wave detection result. For example, the slow-wave detection result has a dimension of 4, corresponding to four classification results: "no slow-wave," "delta-wave," "theta-wave," and "mixed slow-wave."

[0091] By using a multimodal fusion model and a classification model to perform the final feature fusion and classification process, the complementary advantages of the two modal features can be fully utilized to improve detection accuracy. Furthermore, the two models can efficiently process massive feature data, thereby improving processing efficiency.

[0092] In one implementation, the multimodal fusion model includes a cross-attention model. The above-described method of fusing visual features and background semantic features using a multimodal fusion model to obtain fused features includes the following steps: Input visual features and background semantic features into the cross-attention model; Query information is extracted from background semantic features by query weights in the cross-attention model, and key information and value information are extracted from visual features by key weights and value weights in the cross-attention model, respectively. The fusion features are determined based on the query information, key information, and value information.

[0093] Cross-attention models, in particular, can fuse features from different modalities based on a cross-attention mechanism. They typically utilize information from one modality to guide feature extraction from another, thereby uncovering deep relationships between the two modalities and achieving high-quality feature fusion. The parameters of a cross-attention model include query weight, key weight, and value weight.

[0094] For example, after inputting visual features and background semantic features into the cross-attention model, the background semantic features are linearly transformed and filtered based on query weights, and redundant dimensions unrelated to slow wave detection can be removed to obtain structured query information. This query information is used to guide the subsequent matching of key and value information. Based on key weights, the visual features are linearly transformed and filtered to extract key dimension information related to slow wave detection, resulting in structured key information. This key information is used to match the query information, reflecting the slow wave-related dimensions in the visual features. Based on value weights, the visual features are linearly transformed and filtered to extract specific features corresponding to the key information, resulting in structured value information. This value information is used to generate fused features in conjunction with the query information. The query information and key information are input into the attention calculation layer of the cross-attention model. Attention weights are obtained based on the correlation between the two, where each element represents the degree of correlation between a certain dimension of the query information and a certain dimension of the key information. The higher the correlation, the larger the element value, used to determine the weight allocation of the value information. Attention weights are multiplied with value information using a matrix multiplication operation, and each dimension of the value information is assigned a corresponding attention weight (higher weights for dimensions with higher relevance), resulting in weighted value information. This weighted value information can then be element-wise added to the query information. After processing with activation functions such as ReLU (Revised Linear Unit), a structured fusion feature is output. This fusion feature incorporates both the specific slow-wave information of visual features and the guiding information of background semantic features, achieving a deep fusion of two modalities.

[0095] In one implementation, the above-mentioned method of fusing visual features and background semantic features using a multimodal fusion model to obtain fused features includes the following steps: A multimodal fusion model is used to align visual features and background semantic features to obtain the first intermediate feature corresponding to the visual features and the second intermediate feature corresponding to the background semantic features. Determine the image quality of the initial feature map and the effective information length of the background description information; The fusion weights are determined based on image quality and effective information length. The fused feature is obtained by fusing the first and second intermediate features based on the fusion weight.

[0096] The multimodal fusion model uses feature alignment to map visual features and background semantic features to the same feature space, eliminating intermodal differences and obtaining corresponding first and second intermediate features. For example, the multimodal fusion model includes a feature alignment layer, which can employ modality adaptive alignment algorithms, cosine similarity alignment algorithms, etc., to map visual features and background semantic features to the same feature space, obtaining the first intermediate feature corresponding to the visual features and the second intermediate feature corresponding to the background semantic features.

[0097] The image quality of the initial feature map can include one or more of the following metrics: sharpness, noise content, and recognizability of slow-wave features. The effective information length of the background description information refers to the text length (e.g., number of sentences, number of characters) of key information related to slow-wave detection within the background description information, used to characterize the effectiveness and richness of the background description information. It should be understood that the higher the image quality of the initial feature map, the more important the visual features and their corresponding first intermediate features are to the slow-wave detection process, and the higher the proportion of the first intermediate features should be during feature fusion. The greater the effective information length of the background description information, the more important the background semantic features and their corresponding second intermediate features are to the slow-wave detection process, and the higher the proportion of the second intermediate features should be during feature fusion. Therefore, the fusion weights can be determined based on image quality and effective information length, including the respective proportions of the first and second intermediate features during feature fusion. Based on the fusion weights, the first and second intermediate features are fused in an appropriate ratio to obtain high-quality fused features.

[0098] The process of calculating the fusion weights and the two fusion features is explained in detail below: In one implementation, an image quality assessment system is constructed, where sharpness has a weight of 0.4, noise content has a weight of 0.3, and slow-wave feature recognizability has a weight of 0.3. Traditional detection methods are used to determine the sharpness and noise content of the initial feature map, and a pre-trained image assessment model is used to determine the recognizability of the slow-wave features in the initial feature map. Based on the weighted result of the three indicators, an image quality score for the initial feature map is obtained. The background description information is segmented into words, and the number of keywords and related sentences is counted. These indicators are then combined to obtain the effective information length. A normalization algorithm is then used to normalize the image quality score and the effective information length to the [0,1] interval, respectively, to obtain the normalized image quality value and the normalized effective information length value. The weights corresponding to the normalized image quality value (e.g., 0.6) and the normalized effective information length value (e.g., 0.4) are pre-set. Based on these weights, a fusion weight is calculated, including the weights of the first intermediate feature and the second intermediate feature. For example, if the image quality score is 8.3, the normalized value is 8.3 / 10 = 0.83; the effective information length is 2, and the maximum effective information length is set to 3, then the normalized value of the effective information length is 0.67; substituting into the formula, the weight α of the first intermediate feature is 0.65, and the weight β of the second intermediate feature is 0.35. Finally, the first and second intermediate features are fused according to the above weights. For example, a gating mechanism (the parameters of which include fusion weights) can be used to fuse the first and second intermediate features to obtain the fused feature.

[0099] By employing the above method and feature alignment processing, visual features and background semantic features are mapped to the same feature space. This eliminates the differences in numerical distribution and feature dimensionality between the two different modalities, ensuring smooth fusion of the first and second intermediate features and improving the integrity of the fused features. Furthermore, adaptive allocation of fusion weights is achieved, aligning with the key indicators of image quality and effective information length of background description in the initial feature map. This eliminates the need for manually preset fixed weights, adapts to the differences in data source quality across different detection scenarios, and makes the fused features more focused on slow-wave information, providing more accurate input for subsequent classification and detection, thereby improving the accuracy of slow-wave detection.

[0100] In one implementation, reference Figure 7 As shown, the slow-wave detection method also includes the following steps: S710 acquires multiple sets of sample data, each set of sample data including sample EEG data, corresponding sample background description information and slow wave detection annotation information; S720: Determine the initial feature map of the sample based on the sample EEG data, input the initial feature map of the sample into the visual model, and obtain the visual features of the sample. S730 uses a natural language model to process the sample background description information to obtain the sample background semantic features; S740, input the sample visual features and sample background semantic features into the multimodal fusion model to obtain the first sample intermediate feature corresponding to the sample visual features and the second sample intermediate feature corresponding to the sample background semantic features, and fuse the first sample intermediate feature and the second sample intermediate feature to obtain the sample fusion feature; S750 inputs the sample fusion features into the classification model to obtain the sample slow wave detection results; S760, Based on the correlation between the intermediate features of the first sample and the intermediate features of the second sample corresponding to the same set of sample data, and the correlation between the intermediate features of the first sample and the intermediate features of the second sample corresponding to different sets of sample data, determine the multimodal alignment loss function; S770, determine the classification loss function based on the sample slow wave detection results and slow wave detection annotation information; S780 updates the multimodal fusion model and classification model based on the multimodal alignment loss function and classification loss function.

[0101] In this process, experts can manually annotate the sample EEG data to obtain standard slow-wave detection results, including whether slow waves exist in the sample EEG data and the type of slow waves. This serves as slow-wave detection annotation information, providing a supervisory basis for model training. Each set of sample data corresponds one-to-one with the sample EEG data, sample background description information, and slow-wave detection annotation information.

[0102] The same processing method as described above for EEG data and background description information is adopted, using the model to be trained to process the sample EEG data and sample background description information. Specifically, an initial feature map is determined based on the sample EEG data, such as the EEG, frequency domain map, or video joint domain feature map corresponding to the sample EEG data. The initial feature map is input into the visual model to obtain the sample visual features. Alternatively, key descriptive information of the sample is determined based on the sample background description information, and then the sample location indication information is determined. A feature map of interest is extracted from the initial feature map, and the initial feature map and the feature map of interest are input into the visual model to obtain the sample visual features. The sample background description information is processed using a natural language model to obtain the sample background semantic features. For example, the sample waveform classification result in the initial feature map is determined, and the corresponding sample waveform description information is obtained. The sample background description information and the sample waveform description information are combined and input into the natural language model to obtain the sample background semantic features. The sample visual features and sample background semantic features are aligned through a multimodal fusion model to obtain the first and second intermediate features of the sample. The first and second intermediate features of the sample can be output by the intermediate layer of the multimodal fusion model. Then, the multimodal fusion model further fuses the intermediate features of the first and second samples. The output layer of the multimodal fusion model outputs the fused sample features, or the fusion weights are determined based on the image quality of the initial feature maps of the samples and the effective information length of the sample background description information. The first and second intermediate features are then fused based on these fusion weights to obtain the fused sample features. Finally, the fused sample features are input into the classification model to obtain the slow-wave detection results of the samples.

[0103] In the above process, the features of the two modalities were aligned, and the fused features were classified. Based on this, two loss functions can be constructed: ① The multimodal alignment loss function is a loss function used to measure the degree of alignment between two modal features (sample visual features and sample background semantic features). The first and second sample intermediate features reside in the same feature space, and their correlation can be directly calculated using methods such as cosine similarity and Euclidean distance (the smaller the distance, the higher the correlation). Pairing the first and second sample intermediate features generated from a large amount of sample data creates multiple data pairs (first sample intermediate feature, second sample intermediate feature). If the first and second sample intermediate features in a data pair originate from the same set of sample data (i.e., the initial sample EEG data corresponding to the first sample intermediate feature and the initial sample background description information corresponding to the second sample intermediate feature belong to the same set of sample data), then the correlation between the first and second sample intermediate features should be high (e.g., approaching 1). If the first and second intermediate features in a data pair originate from different groups of sample data (i.e., the initial EEG data corresponding to the first intermediate feature and the initial background description information corresponding to the second intermediate feature belong to different groups of sample data), then the correlation between the first and second intermediate features should be low (e.g., close to 0). Therefore, a multimodal alignment loss function can be constructed based on whether the correlation between the first and second intermediate features corresponding to the same group of sample data approaches 1, and whether the correlation between the first and second intermediate features corresponding to different groups of sample data approaches 0. The correlation calculation results of a large number of data pairs are then substituted into this function to obtain the loss function value.

[0104] For example, the multimodal alignment loss function can be represented by the following expression: ;Formula (1) Where L1 represents the multimodal alignment loss function, sim i,j τ represents the similarity between the intermediate features of the first sample corresponding to the i-th sample data and the intermediate features of the second sample corresponding to the j-th sample data. τ is a learnable temperature coefficient used to scale the similarity and control the sharpness of the probability distribution. N represents the number of sample data groups.

[0105] ② The classification loss function is used to measure the degree of deviation between the slow-wave detection results and the slow-wave detection annotation information. For example, the cross-entropy loss function can be used, and its expression is as follows: ;Formula (2) Where L2 represents the classification loss function, m is the number of classification categories detected by slow wave, and y i p represents the slow-wave detection annotation information for the i-th category. iThis represents the probability value of the i-th category in the slow-wave detection results of the sample.

[0106] The multimodal fusion model and the classification model can be updated based on the multimodal alignment loss function and the classification loss function. For example, updating the multimodal fusion model based on the multimodal alignment loss function and the classification loss function involves weighting the two loss functions according to preset weights to obtain a comprehensive loss function, calculating the parameter update gradient of the multimodal fusion model based on the comprehensive loss function, and then updating the parameters of the multimodal fusion model. Similarly, updating the classification model based on the classification loss function involves calculating the parameter update gradient of the classification model based on the classification loss function, and then updating the parameters of the classification model. In one implementation, at least one of the visual model and the natural language model can also be updated based on the multimodal alignment loss function and the classification loss function.

[0107] based on Figure 7 The proposed method provides a model training scheme that, through joint optimization of the multimodal alignment loss function and the classification loss function, not only ensures the effective alignment of sample visual features and sample background semantic features, solving the problem of fusion deviation of different modal features and improving the fusion accuracy of the multimodal fusion model, but also ensures that the output of the classification model is highly matched with the labeled information, thereby improving the accuracy and reliability of slow wave detection.

[0108] In one implementation, the slow-wave detection process can be referenced. Figure 8 As shown, a general visual model is pre-learned through transfer learning to obtain a visual model suitable for EEG visual feature extraction. EEG data is collected to obtain multi-channel EEG images, which are then input into the visual model to obtain visual features. Background description information corresponding to the EEG data is obtained and input into a semantically adapted LLM for feature encoding and extraction to obtain background semantic features. A cross-attention model is used to fuse visual features and background semantic features to obtain fused features. The fused features are then input into a classification model to obtain slow-wave detection results.

[0109] Exemplary device

[0110] The slow-wave detection device in electroencephalography (EEG) according to the embodiments of this disclosure will now be described. (See reference...) Figure 9 As shown, the slow-wave detection device 900 may include the following modules: The data acquisition module 910 is configured to acquire the electroencephalogram (EEG) data of the test subject and to acquire the background description information corresponding to the EEG data. The visual processing module 920 is configured to determine an initial feature map based on the EEG data, and process the initial feature map using a visual model to obtain visual features; the visual model is obtained by transfer learning of a general visual model using the EEG dataset. Natural language processing module 930 is configured to process the background description information using a natural language model to obtain background semantic features; The detection result determination module 940 is configured to determine the slow wave detection result of the EEG data based on the visual features and the background semantic features.

[0111] In one embodiment, the EEG data includes first lead data from multiple channels; the data acquisition module 910 is configured to preprocess the EEG data of the subject by: filtering the first lead data; calculating the average value of the filtered first lead data from multiple channels; determining the difference between the filtered first lead data of each channel and the average value to obtain second lead data from multiple channels; and combining the first lead data and the second lead data to obtain preprocessed EEG data. The step of determining an initial feature map based on the EEG data includes: determining an initial feature map based on the preprocessed EEG data.

[0112] In one implementation, the natural language model includes a large language model; the step of processing the background description information using the natural language model to obtain background semantic features includes: generating prompt information based on the background description information and the prompt template; and inputting the prompt information into the large language model to obtain the background semantic features.

[0113] In one embodiment, determining the slow-wave detection result of the EEG data based on the visual features and the background semantic features includes: fusing the visual features and the background semantic features using a multimodal fusion model to obtain fused features; and processing the fused features using a classification model to obtain the slow-wave detection result.

[0114] In one embodiment, the multimodal fusion model includes a cross-attention model; the step of fusing the visual features and the background semantic features using the multimodal fusion model to obtain fused features includes: inputting the visual features and the background semantic features into the cross-attention model; extracting query information from the background semantic features using query weights in the cross-attention model; extracting key information and value information from the visual features using key weights and value weights in the cross-attention model, respectively; and determining the fused features based on the query information, the key information, and the value information.

[0115] In one embodiment, fusing the visual features and the background semantic features using a multimodal fusion model to obtain fused features includes: aligning the visual features and the background semantic features using a multimodal fusion model to obtain a first intermediate feature corresponding to the visual features and a second intermediate feature corresponding to the background semantic features; determining the image quality of the initial feature map and determining the effective information length of the background description information; determining fusion weights based on the image quality and the effective information length; and fusing the first intermediate feature and the second intermediate feature based on the fusion weights to obtain the fused features.

[0116] In one embodiment, the device is further configured to: acquire multiple sets of sample data, each set of sample data including sample EEG data, corresponding sample background description information, and slow wave detection annotation information; determine an initial feature map of the sample based on the sample EEG data; input the initial feature map of the sample into a visual model to obtain sample visual features; process the sample background description information using a natural language model to obtain sample background semantic features; input the sample visual features and the sample background semantic features into the multimodal fusion model to obtain a first sample intermediate feature corresponding to the sample visual features and a second sample intermediate feature corresponding to the sample background semantic features. The intermediate features of the first sample and the intermediate features of the second sample are fused to obtain sample fusion features; the sample fusion features are input into the classification model to obtain sample slow wave detection results; a multimodal alignment loss function is determined based on the correlation between the intermediate features of the first sample and the intermediate features of the second sample corresponding to the same set of sample data, and the correlation between the intermediate features of the first sample and the intermediate features of the second sample corresponding to different sets of sample data; a classification loss function is determined based on the sample slow wave detection results and the slow wave detection annotation information; the multimodal fusion model and the classification model are updated based on the multimodal alignment loss function and the classification loss function.

[0117] In one embodiment, processing the initial feature map using a visual model to obtain visual features includes: extracting key descriptive information associated with the initial feature map from the background descriptive information; determining location indication information based on the key descriptive information; the location indication information being used to indicate the location information of the region of interest in the initial feature map; extracting the feature map of interest from the initial feature map based on the location indication information; and inputting the initial feature map and the feature map of interest into the visual model to obtain the visual features.

[0118] In one implementation, the step of processing the background description information using a natural language model to obtain background semantic features includes: classifying the waveforms in the initial feature map to obtain waveform classification results; obtaining waveform description information corresponding to the waveform classification results from a pre-built knowledge base; and combining the background description information and the waveform description information and inputting them into the natural language model to obtain the background semantic features.

[0119] Furthermore, other specific details of the embodiments of this disclosure have been described in detail in the embodiments of the above methods, and will not be repeated here.

[0120] Exemplary program product

[0121] The computer program product in the embodiments of this disclosure will now be described. The computer program product includes a computer program that, when executed by a processor, implements the methods described above in this disclosure.

[0122] In one implementation, the computer program product can be a tangible product, such as a computer-readable storage medium storing a computer program. The readable storage medium can be based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, and includes, but is not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. Exemplarily, the computer program product can be a non-volatile storage medium storing a computer program, such as read-only memory, NAND flash memory, etc.

[0123] In one implementation, the computer program product can be an intangible product. For example, the computer program product can be a virtual digital product, such as an executable file or installation package containing a computer program.

[0124] Computer program code can be written in one or more programming languages. Programming languages ​​include, but are not limited to, C, Java, and C++. Program code can execute entirely on the user's computing device, or partially on the user's computing device, or as a standalone software package, or partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).

[0125] Computer programs can be carried or transmitted via signals such as electrical, magnetic, optical, electromagnetic, and infrared rays. Electronic devices can convert the signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, to be executed by the processor of the electronic device) the method steps of various embodiments of this disclosure, such as... Figure 2 The steps are shown.

[0126] By executing the above methods and steps through a computer program, the following technical effects can be achieved: First, an automated slow-wave detection scheme is provided, solving the problem of low efficiency in manual EEG interpretation, improving detection efficiency, and enabling rapid processing of massive amounts of EEG data to meet large-scale testing needs. Second, based on the extraction of visual features from EEG data, background description information is also introduced. Through both visual and background semantic features, the EEG state of the tested subject can be comprehensively reflected. Subsequent feature processing yields slow-wave detection results, achieving an objective and unified feature extraction and detection logic, improving the accuracy of slow-wave detection, and solving problems such as reliance on professional experience and inconsistent subjective judgment standards in manual interpretation, thus enhancing the reliability and comparability of detection results. Third, this scheme possesses good versatility and scalability, adapting to different devices and EEG data formats, and can be widely applied in various scenarios such as clinical monitoring and home testing.

[0127] Exemplary electronic devices

[0128] The electronic device in the embodiments of this disclosure will now be described. The electronic device may be... Figure 1 Any device in the present disclosure. An electronic device includes a processor and a memory for storing executable instructions of the processor. The processor is configured to perform the methods described above by executing the executable instructions.

[0129] refer to Figure 10 An electronic device according to embodiments of the present disclosure will be described by way of example. Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0130] like Figure 10 As shown, the electronic device 1000 is presented in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: a processor 1010, a memory 1020, a bus 1030 connecting different system components (including the memory 1020 and the processor 1010), an I / O (input / output) interface 1040, and a network adapter 1050.

[0131] The memory 1020 stores program code, which can be executed by the processor 1010, causing the processor 1010 to perform the method steps of this embodiment. For example... Figure 2 The steps are shown.

[0132] By executing the above method steps through processor 1010, the following technical effects can be achieved: First, an automated slow-wave detection scheme is provided, solving the problem of low efficiency in manual EEG interpretation, improving detection efficiency, and enabling rapid processing of massive EEG data to meet large-scale detection needs. Second, based on the extraction of visual features from EEG data, background description information is also introduced. Through both visual and background semantic features, the EEG state of the tested subject can be comprehensively reflected. Subsequent feature processing yields slow-wave detection results, achieving an objective and unified feature extraction and detection logic, improving the accuracy of slow-wave detection, and solving problems such as reliance on professional experience and inconsistent subjective judgment standards in manual interpretation, thus enhancing the reliability and comparability of detection results. Third, this scheme has good versatility and scalability, adapting to different devices and different formats of EEG data, and can be widely applied in various scenarios such as clinical monitoring and home testing.

[0133] The memory 1020 may include volatile memory, such as random access memory (RAM) 1021 and / or cache unit 1022, and may also include non-volatile memory, such as read-only memory (ROM) unit 1023. The memory 1020 may also include one or more program modules 1024, such program modules 1024 including, but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. For example, program module 1024 may include the modules described above.

[0134] The processor 1010 may include processing units such as AP (Application Processor), modem processor, GPU (Graphics Processing Unit), ISP (Image Signal Processor), controller, encoder, decoder, DSP (Digital Signal Processor), baseband processor and / or NPU (Neural-Network Processing Unit).

[0135] Bus 1030 is used to connect different parts of electronic device 1000, and may include data bus, address bus and control bus, etc.

[0136] The electronic device 1000 can also communicate with one or more external devices 1100 (such as keyboards, pointing devices, Bluetooth devices, etc.), and this communication can be performed through the I / O interface 1040.

[0137] Electronic device 1000 can also communicate with one or more networks via network adapter 1050. For example, network adapter 1050 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication. Network adapter 1050 can communicate with other modules of electronic device 1000 via bus 1030.

[0138] Although not shown in the figure, other hardware and / or software modules may be installed on the electronic device 1000, including but not limited to: display, microcode, device driver, redundant processing unit, external disk drive array, RAID (Redundant Arrays of Independent Disks) system, tape drive, and data backup storage system.

[0139] It should be noted that although several modules or sub-modules of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0140] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0141] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A slow wave detection method in electroencephalography, characterized by, The method includes: Acquire the electroencephalogram (EEG) data of the test subject, and acquire the background description information corresponding to the EEG data; An initial feature map is determined based on the EEG data, and the initial feature map is processed using a visual model to obtain visual features; the visual model is obtained by transfer learning of a general visual model using the EEG dataset. The background description information is processed using a natural language model to obtain background semantic features; The slow-wave detection results of the EEG data are determined based on the visual features and the background semantic features. The step of processing the initial feature map using a visual model to obtain visual features includes: Extract key descriptive information associated with the initial feature map from the background description information; Location indication information is determined based on the key description information; the location indication information is used to indicate the location information of the region of interest in the initial feature map; Extract the feature map of interest from the initial feature map based on the location indication information; The initial feature map and the feature map of interest are input into the visual model to obtain the visual features; The step of processing the background description information using a natural language model to obtain background semantic features includes: classifying the waveforms in the initial feature map to obtain waveform classification results; obtaining waveform description information corresponding to the waveform classification results from a pre-built knowledge base; and combining the background description information and the waveform description information and inputting them into the natural language model to obtain the background semantic features. The step of determining the slow-wave detection result of the EEG data based on the visual features and the background semantic features includes: fusing the visual features and the background semantic features using a multimodal fusion model to obtain fused features; and processing the fused features using a classification model to obtain the slow-wave detection result. The step of fusing the visual features and the background semantic features using a multimodal fusion model to obtain fused features includes: aligning the visual features and the background semantic features using a multimodal fusion model to obtain a first intermediate feature corresponding to the visual features and a second intermediate feature corresponding to the background semantic features; determining the image quality of the initial feature map and determining the effective information length of the background description information; determining fusion weights based on the image quality and the effective information length; and fusing the first intermediate feature and the second intermediate feature based on the fusion weights to obtain the fused features.

2. The method according to claim 1, characterized in that, The EEG data includes first-lead data from multiple channels; after acquiring the EEG data of the subject, it is preprocessed in the following manner: Filter the data from the first lead; Calculate the average value of the filtered first lead data from multiple channels; The difference between the filtered first lead data of each channel and the average value is determined to obtain the second lead data of multiple channels; The data from the first lead and the data from the second lead are combined to obtain preprocessed EEG data; The step of determining the initial feature map based on the EEG data includes: Initial feature maps are determined based on preprocessed EEG data.

3. The method according to claim 1, characterized in that, The natural language model includes a large language model; the process of using the natural language model to process the background description information to obtain background semantic features includes: Based on the background description information and the prompt template, generate prompt information; The prompt information is input into the large language model to obtain background semantic features.

4. The method according to claim 1, characterized in that, The multimodal fusion model includes a cross-attention model; the process of fusing the visual features and the background semantic features using the multimodal fusion model to obtain fused features includes: The visual features and the background semantic features are input into the cross-attention model; Query information is extracted from the background semantic features using the query weights in the cross-attention model, and key information and value information are extracted from the visual features using the key weights and value weights in the cross-attention model, respectively. The fusion feature is determined based on the query information, the key information, and the value information.

5. The method according to claim 1, characterized in that, The method further includes: Multiple sets of sample data were acquired. Each set of sample data included sample EEG data, corresponding sample background description information, and slow wave detection annotation information. Based on the sample EEG data, an initial feature map of the sample is determined, and the initial feature map of the sample is input into a visual model to obtain the visual features of the sample. The sample background description information is processed using a natural language model to obtain sample background semantic features; The sample visual features and the sample background semantic features are input into the multimodal fusion model to obtain the first sample intermediate feature corresponding to the sample visual features and the second sample intermediate feature corresponding to the sample background semantic features. The first sample intermediate feature and the second sample intermediate feature are fused to obtain the sample fusion feature. The sample fusion features are input into the classification model to obtain the sample slow wave detection results; The multimodal alignment loss function is determined based on the correlation between the intermediate features of the first and second samples corresponding to the same set of sample data, and the correlation between the intermediate features of the first and second samples corresponding to different sets of sample data. Based on the slow wave detection results of the samples and the slow wave detection annotation information, determine the classification loss function; The multimodal fusion model and the classification model are updated based on the multimodal alignment loss function and the classification loss function.

6. The method according to any one of claims 1 to 5, characterized in that, The initial feature map includes any one or more of the first initial feature map, the second initial feature map, the third initial feature map, and the fourth initial feature map; the determination of the initial feature map based on the EEG data includes any one or more of the following methods: Using the number of channels in the EEG data as the height and the dimension in each channel as the width, the EEG data is converted into a first initial feature map; The first initial feature map is dimensionally transformed to obtain a second initial feature map; the difference between the height and width of the second initial feature map is smaller than the difference between the height and width of the first initial feature map. The EEG data is converted into time-frequency features, and the time and frequency dimensions of the time-frequency features are used as the height and width, respectively, to convert the time-frequency features into a third initial feature map. Obtain the electroencephalogram (EEG) corresponding to the EEG data, and determine the fourth initial feature map based on the EEG.

7. A slow-wave detection device for electroencephalography (EEG), characterized in that, The device includes: The data acquisition module is configured to acquire the electroencephalogram (EEG) data of the test subject and to acquire the background description information corresponding to the EEG data. The visual processing module is configured to determine an initial feature map based on the EEG data, and process the initial feature map using a visual model to obtain visual features; the visual model is obtained by transfer learning of a general visual model using the EEG dataset. The natural language processing module is configured to process the background description information using a natural language model to obtain background semantic features; The detection result determination module is configured to determine the slow wave detection result of the EEG data based on the visual features and the background semantic features; The step of processing the initial feature map using a visual model to obtain visual features includes: Extract key descriptive information associated with the initial feature map from the background description information; Location indication information is determined based on the key description information; the location indication information is used to indicate the location information of the region of interest in the initial feature map; Extract the feature map of interest from the initial feature map based on the location indication information; The initial feature map and the feature map of interest are input into the visual model to obtain the visual features; The step of processing the background description information using a natural language model to obtain background semantic features includes: classifying the waveforms in the initial feature map to obtain waveform classification results; obtaining waveform description information corresponding to the waveform classification results from a pre-built knowledge base; and combining the background description information and the waveform description information and inputting them into the natural language model to obtain the background semantic features. The step of determining the slow-wave detection result of the EEG data based on the visual features and the background semantic features includes: fusing the visual features and the background semantic features using a multimodal fusion model to obtain fused features; and processing the fused features using a classification model to obtain the slow-wave detection result. The step of fusing the visual features and the background semantic features using a multimodal fusion model to obtain fused features includes: aligning the visual features and the background semantic features using a multimodal fusion model to obtain a first intermediate feature corresponding to the visual features and a second intermediate feature corresponding to the background semantic features; determining the image quality of the initial feature map and determining the effective information length of the background description information; determining fusion weights based on the image quality and the effective information length; and fusing the first intermediate feature and the second intermediate feature based on the fusion weights to obtain the fused features.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 6 by executing the executable instructions.

Images