Sleep detection method and device based on electroencephalogram, program product and electronic equipment

Abstract

The invention provides a sleep detection method and device based on electroencephalogram, a program product and electronic equipment, and relates to the technical field of computers. The method comprises the following steps: acquiring electroencephalogram data of a tested object; determining at least two initial feature maps according to the electroencephalogram data; processing the at least two initial feature maps by using a visual model to obtain a first intermediate feature; performing cascade expansion convolution processing on the first intermediate feature to obtain a second intermediate feature; and determining a sleep detection result corresponding to the electroencephalogram data according to the second intermediate feature. According to the invention, automatic sleep detection is realized, the detection efficiency is improved, and the accuracy and reliability of the detection result are improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a method, apparatus, program product, and electronic device for sleep detection based on electroencephalography (EEG). Background Technology

[0002] Sleep testing, which involves detecting the sleep stage or state of a subject, is a core component in assessing sleep quality and diagnosing sleep disorders. Furthermore, sleep testing results can serve as important health information for the subject, providing a reference for the diagnosis and treatment of related diseases. With increasing societal attention to sleep health and the continuous development of medical testing technology, the demand for sleep testing and related research is growing. Currently, sleep testing still primarily relies on manual analysis of electroencephalograms (EEGs). Summary of the Invention

[0003] However, using manual EEG analysis for sleep monitoring has the following problems: the sleep monitoring process generates a large amount of EEG data, and manual identification and analysis are time-consuming, labor-intensive, and inefficient, making it difficult to meet the needs of large-scale screening or long-term monitoring. Furthermore, the results of manual analysis are subject to subjectivity, and their accuracy is difficult to guarantee. In particular, the results analyzed by different experts may have low consistency, affecting the reliability and comparability of the test results.

[0004] In view of the above problems, this disclosure provides a method, device, program product and electronic device for sleep detection based on electroencephalography (EEG).

[0005] According to a first aspect of this disclosure, a sleep detection method based on electroencephalography (EEG) is provided. The method includes: acquiring EEG data of a subject; determining at least two initial feature maps based on the EEG data; processing the at least two initial feature maps using a visual model to obtain a first intermediate feature; performing cascaded dilation convolution processing on the first intermediate feature to obtain a second intermediate feature; and determining a sleep detection result corresponding to the EEG data based on the second intermediate feature.

[0006] According to a second aspect of this disclosure, a sleep detection device based on electroencephalography (EEG) is provided. The device includes: an EEG data acquisition module configured to acquire EEG data of a subject; an initial feature map determination module configured to determine at least two initial feature maps based on the EEG data; a visual processing module configured to process the at least two initial feature maps using a visual model to obtain a first intermediate feature; a dilated convolution module configured to perform cascaded dilated convolution processing on the first intermediate feature to obtain a second intermediate feature; and a detection result determination module configured to determine a sleep detection result corresponding to the EEG data based on the second intermediate feature.

[0007] According to a third aspect of this disclosure, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method of the first aspect described above and possible implementations thereof.

[0008] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of the first aspect and possible implementations thereof by executing the executable instructions.

[0009] The embodiments disclosed herein have the following technical effects: Firstly, this solution provides an automated sleep detection method, addressing the time-consuming, labor-intensive, and inefficient nature of manual EEG analysis. It improves sleep detection efficiency, enabling rapid processing of large-scale EEG data, shortening sleep detection time, and meeting the needs of large-scale sleep disorder screening and long-term sleep monitoring. Secondly, it determines at least two initial feature maps based on EEG data, improving the comprehensiveness of initial feature information. Standardized feature initialization is achieved through a visual model, and multi-scale features are captured through cascaded dilated convolution. Subsequent feature processing yields sleep detection results, realizing an objective and unified feature extraction and detection logic. This overcomes the limitations of incomplete and insufficient feature extraction in related technologies, improving the accuracy of sleep detection and addressing the issues of strong subjectivity and low consistency in manual analysis, thus enhancing the reliability and comparability of detection results. Thirdly, this solution possesses good versatility and scalability, adapting to different devices and EEG data formats, and can be widely applied in various scenarios such as sleep disorder diagnosis and home sleep monitoring. Attached Figure Description

[0010] Figure 1 A schematic diagram of a system architecture according to an embodiment of this disclosure is shown.

[0011] Figure 2 A flowchart of a sleep detection method according to an embodiment of this disclosure is shown.

[0012] Figure 3 A schematic diagram of a lead system according to an embodiment of this disclosure is shown.

[0013] Figure 4 A schematic diagram of a multichannel electroencephalogram (EEG) according to an embodiment of this disclosure is shown.

[0014] Figure 5 A schematic diagram of a cascaded dilated convolution is shown in an embodiment of this disclosure.

[0015] Figure 6 A flowchart illustrating one method for determining sleep test results according to an embodiment of this disclosure is shown.

[0016] Figure 7 A schematic flowchart of a sleep detection method according to an embodiment of the present disclosure is shown.

[0017] Figure 8 A flowchart illustrating a training model in an embodiment of this disclosure is shown.

[0018] Figure 9 A schematic diagram of the structure of a sleep detection device according to an embodiment of the present disclosure is shown.

[0019] Figure 10 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0020] The principles and spirit of this disclosure are described in detail below with reference to several representative embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0021] The embodiments of this disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Invention Overview

[0023] Traditional sleep monitoring relies on manual analysis of electroencephalograms (EEGs), which suffers from low efficiency and difficulty in guaranteeing accuracy and reliability. With the development and application of artificial intelligence (AI) technology, automated sleep monitoring solutions based on deep learning have emerged, including the following three types: 1. Using CNNs (Convolutional Neural Networks) to extract features from EEGs and perform classification; 2. Using temporal networks, such as RNNs and LSTMs, to extract features from EEGs and perform classification; 3. Based on features extracted using CNNs, using temporal networks for further feature processing and classification.

[0024] However, Scheme 1 and Scheme 2 can only extract features at a single scale and cannot fully obtain effective information from the EEG. Scheme 3 also suffers from incomplete or insufficient feature extraction (such as insufficient EEG initialization embedding). These problems lead to low accuracy in the final sleep classification.

[0025] In view of the above problems, this disclosure provides a method, device, program product, and electronic device for sleep detection based on electroencephalography (EEG). Specifically, EEG data of the subject is acquired, at least two initial feature maps are determined based on the EEG data, the at least two initial feature maps are input into a visual model to obtain a first intermediate feature, the first intermediate feature is subjected to cascaded dilation convolution processing to obtain a second intermediate feature, and the sleep detection result corresponding to the EEG data is determined based on the second intermediate feature. Thus, in a first aspect, an automated sleep detection scheme is provided, which solves the problems of time-consuming, labor-intensive, and inefficient manual analysis of EEG, improves the efficiency of sleep detection, can quickly process large-scale EEG data, shortens the sleep detection time, and meets the needs of large-scale sleep disorder screening and long-term sleep monitoring. Secondly, by determining at least two initial feature maps based on EEG data, the comprehensiveness of the initial feature information is improved. Standardized feature initialization is achieved through a visual model, multi-scale features are captured through cascaded dilated convolution, and sleep detection results are obtained through subsequent feature processing. This achieves an objective and unified feature extraction and detection logic, overcoming the limitations of incomplete and insufficient feature extraction in related technologies, improving the accuracy of sleep detection, and solving the problems of strong subjectivity and low consistency in manual analysis, thus improving the reliability and comparability of detection results. Thirdly, this solution has good versatility and scalability, can be adapted to different devices and different formats of EEG data, and can be widely applied to various scenarios such as sleep disorder diagnosis and home sleep monitoring.

[0026] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0027] Application Scenarios Overview

[0028] It should be noted that the following application scenarios are shown only to facilitate understanding of the spirit and principles of this disclosure. The implementation of this disclosure is not limited in this respect and can be applied to any applicable scenario.

[0029] The embodiments disclosed herein can be applied to general sleep testing scenarios, where the test results are used to assess the sleep quality of the tested subject. Alternatively, they can also be applied to scenarios related to the detection and auxiliary diagnosis of relevant diseases. For example, if a patient is suspected of having epilepsy, sleep testing can be performed on the patient, and the test results can assist doctors in diagnosing and treating the patient. The application scenarios are described in detail below with reference to the system architecture.

[0030] Figure 1A schematic diagram of a sleep monitoring system architecture is shown. This system architecture includes a subject 110, an EEG monitoring device 120, and a processing device 130. The EEG monitoring device 120 is used to collect EEG data. In one embodiment, the EEG monitoring device 120 includes probe electrodes 1201, 1202, and 1203 and a main device 1204. When sleep monitoring is required on the subject 110, the probe electrodes 1201, 1202, and 1203 are fixed to the subject's head to collect EEG signals. The main device 1204 summarizes and processes the EEG signals to obtain raw or pre-processed EEG data. Of course, Figure 1 The EEG monitoring device 120 shown is merely exemplary and may also include various other components such as helmets, hats, bedding, and encapsulated portable detection electrodes, which are not limited in this disclosure.

[0031] The processing device 130 may be a personal computer, server, or similar device with certain data processing capabilities, used to process EEG data. The EEG monitoring device 120 can communicate with the processing device 130, for example, through a wired or wireless communication link, thereby sending the collected EEG data to the processing device 130. The processing device 130 can execute the sleep detection method in this exemplary embodiment to process the acquired EEG data and obtain the sleep detection results of the subject 110. In one embodiment, the processing device 130 may include a display screen, which can display the sleep detection results and the EEG data.

[0032] In one embodiment, the EEG monitoring device 120 and the processing device 130 can be integrated into the same device. For example, the main device 1204 of the EEG monitoring device 120 can serve as the processing device 130.

[0033] In one implementation, Figure 1 The system architecture shown is suitable for portable scenarios. For example, the test subject 110 can use a portable EEG monitoring device 120 to collect EEG data at home, and then send the EEG data to a remote processing device 130 for sleep monitoring via a data transceiver. In this way, the user can understand their own sleep status without leaving home, and can send the obtained sleep test results to a doctor to help the doctor determine the health status of the test subject 110.

[0034] Exemplary methods

[0035] The following is an exemplary description of the EEG-based sleep detection method in the embodiments of this disclosure. Figure 2 An exemplary flow diagram of the sleep detection method is shown, which may include steps S210 to S250. The following describes... Figure 2 Each step in the process will be explained in detail.

[0036] refer to Figure 2 In step S210, the electroencephalogram (EEG) data of the subject is acquired.

[0037] The subjects being tested are individuals who require sleep monitoring. Electroencephalogram (EEG) data reflects the electrical activity of the cerebral cortex in the subjects.

[0038] In one implementation, the EEG data includes multi-channel data. Channels may refer to the body parts being tested. Figure 3 This diagram illustrates the 10-20 international standard lead system. The points on the head include Nz (nasal root), Iz (external occipital protuberance), Fpz (frontal pole midpoint), Fz (frontal midpoint), Cz (central point), Pz (apex), Oz (occipital point), Fp1 (left frontal pole), F3 (left frontal), C3 (left central), P3 (left parietal), O1 (left occipital), T3 (left temporal midpoint), T5 (left posterior temporal), F7 (left anterior temporal), Fp2 (right frontal pole), F4 (right frontal), C4 (right central), P4 (right parietal), O2 (right occipital), T4 (right temporal midpoint), T6 (right posterior temporal), and F8 (right anterior temporal). The scale values ​​in the diagram represent the proportion of the distance between two points to the length of their respective baselines. For example, the baseline for Nz and Iz is the midline of the head; the distances between Fpz and Nz, and between Oz and Iz, are both 10% of the midline of the head; the distances between Fz and Cz, and between Cz and Pz, are both 20% of the midline of the head; the distance between Fp1 and F7 accounts for 10% of the length of the circular baseline Fpz-T3-Oz-T3-Fpz; and the distance between F3 and F7 accounts for 25% of the length of the baseline F7-Fz-F8. Based on the 10-20 international standard lead system, 19 standard electrodes (i.e., 19 channels) can be set, located at: Fp1, Fp2, F3, F4, F7, F8, Fz, C3, C4, Cz, T3, T4, T5, T6, P3, P4, Pz, O1, O2. Reference electrodes (such as left ear electrode A1, right ear electrode A2) and ground electrodes are also set. The electrical signal data (such as potential) of each electrode was collected using an EEG monitoring device. The relative value (such as potential difference) of each standard electrode and the reference electrode (such as the average value of the left ear electrode and the right ear electrode, or, for the standard electrodes on the left hemisphere and the midline of the head, the left ear electrode is used as the reference electrode, and for the standard electrodes on the right hemisphere, the right ear electrode is used as the reference electrode) was calculated to obtain EEG data of 19 channels.

[0039] In one implementation, the electroencephalogram (EEG) data includes an electroencephalogram (EEG), which is a graphic obtained by amplifying and recording spontaneous bioelectric potentials of the cerebral cortex from the scalp using sophisticated instruments, characterizing the spontaneous, rhythmic electrical activity of brain cell groups. Figure 4The diagram illustrates a multi-channel electroencephalogram (EEG), a graphical representation of the EEG data from the aforementioned 19 channels. For example, with time on the horizontal axis and electrical potential on the vertical axis, a planar graph represents the electrical potentials at different times; this planar graph is the EEG. The frequency (period), amplitude, and phase of the brain waves constitute the basic characteristics of an EEG. Of course, the above EEG data is typically in the time domain; however, frequency domain and combined time-frequency domain EEG data can also be acquired to create corresponding EEG maps.

[0040] After acquiring EEG data and before further processing, preprocessing can be performed. Preprocessing methods include, but are not limited to: Resampling. Resampling can change the sampling rate of a signal, converting a non-uniformly sampled signal into a uniformly sampled signal. For example, EEG data can be resampled at 500Hz.

[0041] Filtering. Methods such as bandpass filtering and notch filtering can be used to reduce interference from non-EEG signals. For example, bandpass filtering of EEG data in the 0.1~70Hz frequency band can be used to eliminate voltage drift and high-frequency noise, followed by 50Hz notch filtering to reduce environmental noise and AC signal interference.

[0042] Data augmentation. For example, raw EEG leads, average leads, left hemisphere leads, and right hemisphere leads can be acquired, and different types of lead data can be combined (e.g., spliced) to obtain augmented EEG data.

[0043] Remove noisy data. Noisy data may be caused by factors such as poor contact (e.g., poor contact between the EEG monitoring device and the subject's body part, poor contact of the device cable, etc.). Removing noisy data can improve data quality and the accuracy of test results. Filtering can reduce some noise. In addition, noisy data can be identified and removed at the data level through analysis. For example, obvious outliers can be removed, or data from channels with more noise can be removed.

[0044] Numerical standardization. Numerical standardization maps data of different modalities and types to the same suitable numerical range for unified processing. For example, given a standard numerical range, EEG data is dedimensionalized and then normalized to that range. This can be achieved by multiplying by a specific coefficient to perform numerical mapping, thus completing the numerical standardization process.

[0045] In one implementation, the EEG data includes brain electrode data and ear electrode data, for example, referring to... Figure 3 As shown, the brain electrode data includes data collected from 19 standard electrodes, and the ear electrode data includes data collected from the left ear electrode A1 and the right ear electrode A2. After acquiring the subject's EEG data, preprocessing can be performed in the following ways: Determine the first reference electrode data, and determine the unipolar lead data based on the brain electrode data and the first reference electrode data; With the reference electrode changed, the second reference electrode data is determined based on the ear electrode data, and the bipolar lead data is determined based on the brain electrode data and the second reference electrode data. By combining unipolar and bipolar lead data, preprocessed EEG data is obtained.

[0046] In this study, the first reference electrode data is the reference electrode data in the unipolar lead, and the second reference electrode data is the reference electrode data in the bipolar lead; the reference electrodes used in the two are different. The first reference electrode data can be determined by any one or more of the following methods: averaging the data from different brain electrodes; averaging the data from the left and right ear electrodes; averaging the data from different brain electrodes and the two ear electrodes; or selecting a specific brain electrode as the reference electrode and using its data as the first reference electrode data. The difference (e.g., potential difference) between the brain electrode data and the first reference electrode data is calculated to obtain the unipolar lead data.

[0047] When the reference electrode is changed compared to the monopolar lead, second reference electrode data is determined based on ear electrode data. For example, either left ear electrode data or right ear electrode data can be used as the second reference electrode data. The difference between the brain electrode data and the second reference electrode data is calculated to obtain bipolar lead data.

[0048] In one embodiment, the brain electrode data includes left hemisphere electrode data, midline electrode data, and right hemisphere electrode data; the ear electrode data includes left ear electrode data and right ear electrode data. For the left hemisphere electrode data and midline electrode data, using the right ear electrode data as a second reference electrode data, the differences between the left hemisphere electrode data, midline electrode data, and right ear electrode data are calculated respectively. For the right hemisphere electrode data, using the left ear electrode data as a second reference electrode data, the differences between the right hemisphere electrode data and left ear electrode data are calculated to obtain bipolar lead data. For example, for left hemisphere electrode data, F7'=F7-A2, F3'=F3-A2, C3'=C3-A2, T3'=T3-A2, P3'=P3-A2, T5'=T5-A2, O1'=O1-A2, yielding data for 7 bipolar lead channels in the left hemisphere; for midline EEG data, Fpz'=Fpz-A2, Fz'=Fz-A2, Cz'=Cz-A2, P... z'=Pz-A2, Oz'=Oz-A2, yielding data for 5 bipolar lead channels in the midline; for the right hemisphere electrode data, F8'=F8-A1, F4'=F4-A1, C4'=C4-A1, T4'=T4-A1, P4'=P4-A1, T6'=T6-A1, O2'=O2-A1, yielding data for 7 bipolar lead channels in the right hemisphere; a total of 19 channels.

[0049] In one implementation, for the left hemisphere and midline, the left ear electrode data is used as the second reference electrode data, and the difference between the left hemisphere electrode data, the midline electrode data and the left ear electrode data is calculated respectively. For the right hemisphere, the right ear electrode data is used as the second reference electrode data, and the difference between the right hemisphere electrode data and the right ear electrode data is calculated to obtain bipolar lead data.

[0050] By using the above methods, bipolar lead data can be obtained by changing the reference electrode on the basis of unipolar leads. This can more accurately reflect the discharge status of different parts of the brain, enhance the local characteristics of EEG signals, reduce the influence of irrelevant interference, and provide a richer and more accurate data source for subsequent multi-scale feature extraction, which is conducive to improving the accuracy of sleep detection.

[0051] The monopolar and bipolar lead data are combined to obtain preprocessed EEG data. Alternatively, the monopolar or bipolar lead data can be further preprocessed, such as removing noise and standardizing the values, before being combined to obtain preprocessed EEG data.

[0052] In one implementation, electroencephalogram (EEG) data, including brain electrode data and ear electrode data, is acquired. The brain electrode data and ear electrode data are then filtered to obtain unipolar lead data and bipolar lead data, each with 19 channels. The two sets of data are then combined (e.g., spliced) and subjected to numerical standardization to obtain preprocessed EEG data. For example, the dimensions of the preprocessed EEG data are (38, 15000), where 38 represents the 19 channels of the unipolar lead and the 19 channels of the bipolar lead, and 15000 represents that each channel includes 15000 data points, which can be a time series formed by 15000 data points.

[0053] Subsequent processing, such as feature extraction, can be performed on the preprocessed EEG data to improve feature quality and the accuracy of the final sleep test results.

[0054] Continue to refer to Figure 2 In step S220, at least two initial feature maps are determined based on the EEG data.

[0055] Initial feature maps are feature data that represent information in EEG data in image form. This disclosure uses at least two types of initial feature maps to represent information in EEG data. Different types of initial feature maps have different information dimensions or sizes, which allows for the focused representation of different information in the EEG data. By combining at least two initial feature maps, the comprehensiveness of information representation is improved, which is beneficial for obtaining accurate sleep detection results. If the EEG data is preprocessed after acquisition, at least two initial feature maps can be determined based on the preprocessed EEG data in step S220.

[0056] In one implementation, the aforementioned at least two initial feature maps include at least two of a first initial feature map, a second initial feature map, a third initial feature map, and a fourth initial feature map. The determination of at least two initial feature maps based on EEG data includes at least two of the following methods: ① Using the number of channels in the EEG data as the height and the dimension in each channel as the width, the EEG data is converted into a first initial feature map.

[0057] For example, the preprocessed 38-channel EEG data is obtained, and the number of channels (38) is used as the height (H) of the feature map, the dimension (15000) of each channel is used as the width (W), and the number of channels (C) is set to 1. The EEG data is converted into a single-channel two-dimensional feature map, namely the first initial feature map, with a dimension of 1×38×15000.

[0058] ② The first initial feature map is transformed in dimensions to obtain the second initial feature map; the difference between the height and width of the second initial feature map is less than the difference between the height and width of the first initial feature map.

[0059] For example, to perform dimensionality transformation on the first initial feature map (1×38×15000), a uniform segmentation reconstruction method can be used. The 15000 data points in the width direction are evenly divided, for example, into 20 segments, each with 750 data points. The 38 channels in the height direction are expanded to 760 (38×20), forming a feature map with dimensions of 1×760×750, i.e., the second initial feature map. Alternatively, based on the input size preferred by the visual model, the 1×760×750 feature map can be padded, adding 8 zero-value pixels in the height direction and 18 zero-value pixels in the width direction, resulting in a second initial feature map with dimensions of 1×768×768.

[0060] By employing the above method, the height and width difference of the first initial feature map are narrowed, making the height and width difference of the second initial feature map smaller than that of the first initial feature map. That is, compared to the first initial feature map, the second initial feature map is closer to a square, facilitating processing by the visual model.

[0061] ③ Convert the EEG data into time-frequency features, and use the time dimension and frequency dimension of the time-frequency features as the height and width to convert the time-frequency features into a third initial feature map.

[0062] Electroencephalogram (EEG) data is typically time-domain data, expressing changes in electrical signals over time, and thus provides relatively simple information. Time-frequency transformation (TFD) can convert EEG data to the frequency domain or a combined time-frequency domain, yielding frequency-domain or time-frequency data. This provides information on the signal's distribution in the frequency domain or its combined distribution in the time and frequency domains. TFD methods include, but are not limited to, short-time Fourier transform (SFT) and wavelet transform. A third initial feature map is then obtained based on the time-frequency characteristics. For example, short-time Fourier transform (STFT) was performed on the preprocessed 38-channel EEG data to extract time-frequency features. The window function was set as the Hanning window with a window length of 256ms, an overlap rate of 50%, and a sampling frequency of 250Hz. The EEG signal of each channel was processed by STFT to obtain a time-frequency matrix (frequency dimension 64, time dimension 1000). The frequency dimension (64) of the time-frequency features was used as the height, the time dimension (1000) as the width, and the number of channels was set to 38. The time-frequency matrices of the 38 channels were spliced ​​together to obtain a third initial feature map with a dimension of 38×64×1000, which integrates time domain and frequency information.

[0063] ④ Obtain the EEG corresponding to the EEG data, and determine the fourth initial feature map based on the EEG.

[0064] When acquiring EEG data, pre-drawn EEG maps can be obtained, or corresponding EEG maps can be drawn based on the EEG data. Using time as the x-axis and signal amplitude as the y-axis, the EEG data from each channel is plotted as a corresponding waveform curve, resulting in an EEG map. This EEG map can be used as a fourth initial feature map, or it can be processed, such as converting it to grayscale or setting it to a standard size, to obtain the fourth initial feature map. For example, the dimensions of the fourth initial feature map can be 1×768×768 or 1×1024×1024.

[0065] In one implementation, the numerical range of different types of initial feature maps can be unified, such as converting the value of each type of initial feature map to a standard numerical range (e.g., 0~1).

[0066] Based on the combination of the aforementioned initial feature maps, a multi-dimensional representation of EEG signals is achieved, overcoming the limitations of incomplete information from a single feature map. Specifically, the first initial feature map retains the original channel and temporal correlation information from the EEG data; the second initial feature map is more adaptable to the input of the visual model, which is beneficial for improving feature extraction efficiency; the third initial feature map integrates time-frequency information; and the fourth initial feature map represents waveform features in image form. These different types of initial feature maps complement each other, comprehensively covering the basic features, structural features, and time-frequency features of the EEG data. In practical applications, different combinations of initial feature maps can be used according to specific needs, such as combining the second, third, and fourth initial feature maps. This lays the foundation for subsequent multi-scale feature fusion, further improving the accuracy of sleep detection.

[0067] Continue to refer to Figure 2 In step S230, the visual model is used to process the above-mentioned at least two initial feature maps to obtain the first intermediate feature.

[0068] The visual model is a machine learning model used to extract visual features. A pre-trained 2D model, such as EfficientNet, can be used as the visual model. Alternatively, a pre-trained model can be fine-tuned using a specific dataset to obtain a usable visual model. After inputting the initial feature map into the visual model, the model can process the initial feature map through convolution, pooling, fully connected layers, etc. The final output of the visual model can be used as the first intermediate feature, or the output of any one or more of its intermediate layers can be used as the first intermediate feature.

[0069] In one implementation, the above-described processing of at least two initial feature maps using a visual model to obtain a first intermediate feature includes the following steps: Each initial feature map is processed using a visual model to obtain the visual features corresponding to each initial feature map; The visual features corresponding to each initial feature map are spliced ​​together to form the first intermediate feature.

[0070] For example, using EfficientNetB0, the original classification layer in the model is removed, and the layer before the classification layer is used as the output layer to obtain the visual model. The second, third, and fourth initial feature maps are input into the visual model respectively. After processing steps such as convolution, pooling, and attention-based feature extraction, the visual features corresponding to each initial feature map are output. The dimensions can be 1×256×32×32 (1 is the batch size, 256 is the number of channels, and 32 is the height and width of the feature). The number of channels can be different for visual features corresponding to different initial feature maps. The visual features corresponding to each initial feature map are concatenated, such as along the channels, to form the first intermediate feature. In one implementation, the concatenated features can also be batch normalized to obtain the first intermediate feature, which can unify the feature distribution and provide a more stable input for subsequent cascaded dilated convolutions.

[0071] By extracting features independently from each initial feature map in the above manner, information interference can be avoided. The concatenation operation realizes the complementary fusion of multi-dimensional features. Compared with a single initial feature map, it can significantly improve the information richness of the first intermediate feature, laying the foundation for subsequent multi-scale feature mining.

[0072] Continue to refer to Figure 2 In step S240, the first intermediate feature is subjected to cascaded dilation convolution to obtain the second intermediate feature.

[0073] Dilated convolution involves inserting holes into a standard convolution kernel, effectively "expanding" the kernel and expanding its field of view without increasing the number of parameters or computational cost. Cascaded dilated convolution involves chaining multiple dilated convolutional layers with different dilation rates together to form a processing module. This allows for the sequential extraction of features at different scales.

[0074] In one implementation, a cascaded dilated convolution module can be set up, which includes multiple cascaded dilated convolution layers with different dilation rates. The first intermediate feature is input into the module to obtain the second intermediate feature.

[0075] In one implementation, the above-described cascaded dilated convolution process on the first intermediate feature to obtain the second intermediate feature includes the following steps: Obtain one or more sequences of expansion parameters, each sequence including one expansion rate or multiple gradually increasing expansion rates; Each dilation parameter sequence is used to perform serial dilation convolution on the first intermediate feature to obtain the corresponding convolution result; The convolution results corresponding to each dilation parameter sequence are fused to obtain the second intermediate feature.

[0076] Here, the dilation parameter sequence refers to an ordered set consisting of one or more dilation rates. One dilation parameter sequence or multiple different dilation parameter sequences can be set; different dilation parameter sequences have different dilation rate values ​​or different numbers of dilation rates. Each dilation parameter sequence is used to perform sequential dilation convolution processing on the first intermediate feature, that is, the dilation convolution operation is performed layer by layer on the first intermediate feature according to each dilation rate in the dilation parameter sequence, resulting in the convolution result corresponding to each dilation parameter sequence. Figure 5 A schematic diagram of serial dilated convolution processing using the dilation parameter sequence {1,2,3} is shown. It can be seen that, with a constant kernel size, dilated convolution using a smaller dilation rate focuses on features within a smaller range (such as adjacent features). As the dilation rate increases, the range of feature extraction expands, resulting in features at a larger scale. In one implementation, the convolution results corresponding to different dilation rates can be weighted, as shown below: F = weight1 × F1 + weight2 × F2 + weight3 × F3; Formula (1) Where F1, F2, and F3 represent the convolution results corresponding to an inflation rate of 1 (e.g., ...). Figure 5 The result of the first convolutional layer), the convolution result corresponding to the dilation rate 2 (e.g.) Figure 5 The result of the second convolution layer), the convolution result corresponding to the dilation rate of 3 (e.g.) Figure 5 The output of the third convolutional layer (weight1, weight2, and weight3) represents the weights corresponding to the three convolutional results. These can be manually set hyperparameters or learnable parameters, satisfying weight1 + weight2 + weight3 = 1. F represents the convolutional result corresponding to the dilation parameter sequence. If only one dilation parameter sequence is set, the convolutional result is the second intermediate feature. If multiple dilation parameter sequences are set, the convolutional results corresponding to each dilation parameter sequence are obtained, and then the convolutional results corresponding to different dilation parameter sequences are fused through weighting, concatenation, etc., to obtain the second intermediate feature.

[0077] In one implementation, the first intermediate feature includes the first intermediate feature corresponding to each initial feature map. The above-described method of fusing the convolution results corresponding to each dilation parameter sequence to obtain the second intermediate feature includes the following steps: The convolution results corresponding to different initial feature maps and the same dilation parameter sequence are fused at the first level, and then the first-level fusion results corresponding to different dilation parameter sequences are fused at the second level to obtain the second intermediate feature.

[0078] The first-level fusion is for the same dilation parameter sequence, fusing information from different initial feature maps; the second-level fusion is for different dilation parameter sequences, integrating multi-scale features.

[0079] For example, based on EEG data, second, third, and fourth initial feature maps are determined. These are then processed using a visual model to obtain their corresponding first intermediate features, denoted as F1a (dimension 1×256×32×32, from the second initial feature map), F1b (dimension 1×256×32×32, from the second initial feature map), and F1c (dimension 1×256×32×32, from the third initial feature map). Three dilation parameter sequences are used, each containing only one dilation rate: 1, 2, and 3. Each dilation parameter sequence is used to perform dilated convolution processing on the first intermediate features corresponding to each initial feature map, resulting in convolution results for each initial feature map and each dilation parameter sequence, denoted as F1a1, F1a2, F1a3, F1b1, F1b2, F1b3, F1c1, F1c2, and F1c3. Here, F1a1 represents the convolution result of the first intermediate feature F1a after processing with dilation parameter sequence 1. The convolution results F1a1, F1b1, and F1c1 corresponding to the dilation parameter sequence 1 are fused at the first level. The fusion method includes, but is not limited to, concatenation, addition, and weighting, to obtain the first-level fusion result F1-1. Similarly, the first-level fusion results F1-2 and F1-3 corresponding to the dilation parameter sequence 2 and dilation parameter sequence 3 can be obtained. The first-level fusion can enhance the feature intensity at a single scale and eliminate the information bias of different initial feature maps. Then, the three first-level fusion results are fused at the second level. The fusion method includes, but is not limited to, concatenation, addition, and weighting, to finally obtain the second intermediate feature F2, realizing the deep integration of features from multiple sources and at multiple scales.

[0080] Continue to refer to Figure 2 In step S240, the sleep detection result corresponding to the EEG data is determined based on the second intermediate feature.

[0081] The second intermediate feature can be used for further classification to obtain the corresponding sleep test results. The sleep test results can refer to sleep stages, such as W stage (Wake), N1 stage (light sleep), N2 stage (light sleep), N3 stage (deep sleep), and REM sleep stage.

[0082] In one implementation, reference Figure 6 As shown, the above method for determining the sleep test results corresponding to EEG data based on the second intermediate feature includes the following steps: S610, the second intermediate feature is converted into a one-dimensional feature and position embedding information is added to form the third intermediate feature; S620, input the third intermediate feature into the Transformer, and determine the fourth intermediate feature based on the information output by the Transformer; S630 uses a classification model to process the fourth intermediate feature to obtain sleep detection results.

[0083] In this model, the one-dimensional feature can be a linear vector flattened from the second intermediate feature. Positional embedding information represents the spatial relationship between features in the second intermediate feature, and the third intermediate feature is a sequence feature adapted to the Transformer input. Adding positional embedding information compensates for the Transformer's insensitivity to positional information. The Transformer uses a global attention mechanism to capture long-distance dependencies between features. The fourth intermediate feature is the global feature integrated by the Transformer. The classification model can be any type of machine learning model, such as an MLP (Multilayer Perceptron), which includes one or more fully connected layers. The classification model performs the final fusion processing on the fourth intermediate feature, outputting the sleep detection result, i.e., the classification result regarding sleep stages.

[0084] In one implementation, the second intermediate feature can be converted into a one-dimensional feature, and location embedding information and a classification token can be added to form a third intermediate feature. Information corresponding to the classification token is extracted from the Transformer output as a fourth intermediate feature. The classification token (cls token) is a learnable or fixed vector used to integrate the global features output by the Transformer. The third intermediate feature consists of the one-dimensional feature, location embedding information, and classification token. After the Transformer processes the third intermediate feature, the information corresponding to the classification token is extracted from its output as the fourth intermediate feature, which simplifies the classification input and reduces redundant interference.

[0085] For example, the second intermediate feature F2 is flattened into a one-dimensional feature to generate a learnable classification label. This label can be initialized using a normal distribution and updated synchronously with the model parameters during training. Learnable positional embedding information is generated and concatenated or added element-wise with the one-dimensional feature. The classification label is then added to the beginning or end of the feature sequence as a carrier of global information, resulting in the third intermediate feature F3. This third intermediate feature F3 is input into a Transformer. The Transformer module (which may include 6 blocks and 8 self-attention heads) performs global attention calculations on F3, allowing the classification label to interact with features at each position in the feature sequence, gradually summarizing global information. The Transformer outputs a complete feature sequence. From this output, the feature vector at the classification label position is extracted as the fourth intermediate feature F4, which accurately integrates global correlation information. Finally, F4 is input into the classification model, which outputs the sleep detection result.

[0086] In one implementation, the procedure for the sleep detection method can be referred to Figure 7 As shown. After acquiring and preprocessing EEG data, multiple initial feature maps are determined. These initial feature maps are input into a visual model to obtain the first intermediate feature. The first intermediate feature is then input into a cascaded dilated convolution module to obtain the second intermediate feature. The second intermediate feature is converted into a one-dimensional feature, and positional embedding information is added. A classification label can also be added to form the third intermediate feature. The third intermediate feature is input into a Transformer, and the information corresponding to the classification label is extracted from the Transformer's output as the fourth intermediate feature. If no classification label was previously added, the Transformer's output is subjected to global pooling (e.g., global average pooling) to obtain the fourth intermediate feature. Finally, the fourth intermediate feature is input into a classification model to obtain the sleep detection result.

[0087] In one implementation, reference Figure 8 As shown, the sleep detection method also includes the following steps: S810, obtain sample EEG data and the sleep tags corresponding to the sample EEG data; S820: Determine at least two initial feature maps of the samples based on the sample EEG data, and input them into the visual model to obtain the intermediate features of the first sample; S830, perform cascaded dilation convolution on the intermediate features of the first sample to obtain the intermediate features of the second sample; S840 converts the intermediate features of the second sample into one-dimensional features and adds position embedding information to form the intermediate features of the third sample; S850: Input the intermediate features of the third sample into the Transformer, and determine the intermediate features of the fourth sample based on the information output by the Transformer. S860 uses a classification model to process the intermediate features of the fourth sample to obtain the sample sleep detection results; S870, based on the sample sleep detection results and sleep labels, determines the loss function value; S880 updates the parameters of at least one of the visual model, Transformer, and classification model based on the loss function value.

[0088] Figure 8This describes the training process for visual models, Transformers, classification models, etc. For example, multi-channel EEG data from different test subjects can be acquired first as sample EEG data. This sample EEG data is then manually annotated by experts (multiple experts can annotate separately, and only those with consistent annotations are used) to obtain sleep labels corresponding to the sample EEG data. Based on the sample EEG data, three initial feature maps (such as a second and third initial feature map) are determined and input into the pre-trained EfficientNetB0 to obtain the first sample intermediate feature. This first intermediate feature is then processed through cascaded dilated convolution to obtain the second sample intermediate feature. This second intermediate feature is flattened into a one-dimensional feature and learnable positional embedding information is added to form the third sample intermediate feature. The third sample intermediate feature is then input into a Transformer to obtain the fourth sample intermediate feature. Finally, the fourth sample intermediate feature is input into a classification model to obtain the sample sleep detection results.

[0089] The loss function value can be determined based on the deviation between the sample sleep detection results and the sleep labels. In one embodiment, determining the loss function value based on the sample sleep detection results and the sleep labels includes the following steps: Based on the deviation between the sample sleep detection results and sleep labels, and using focus loss and L2 regularization loss, a classification loss function is constructed. A sleep state transition loss function is constructed based on the sleep detection results of multiple samples of EEG data with temporal relationship. The loss function value is determined based on the classification loss function and the sleep state transition loss function.

[0090] The classification loss function represents the loss in classifying sleep states and can be constructed based on focus loss and L2 regularization loss. For example, the classification loss function is shown below: ;Formula (2) Among them, L Class L represents the classification loss function. Focal L2 represents the focus loss, and L2 represents the L2 regularization loss. λ is the L2 regularization coefficient, which can be set to 0.001 to penalize all parameters of the visual model, Transformer, and MLP to prevent overfitting. Furthermore, the focus loss involves related parameters, such as setting a balance factor α=0.25 and a focus parameter γ=2 to reduce the weight of easily classified samples and focus on difficult-to-classify samples (such as samples from deep sleep).

[0091] Multiple EEG data samples exhibiting a temporal relationship means that the EEG data samples are consecutive in time. For example, EEG data A is from 23:00 to 23:05, and EEG data B is from 23:05 to 23:10; they are consecutive in time. Generally, sleep states change gradually, such as from N1 to N2, and then from N2 to N3, rather than directly from N1 to N3 or from REM sleep to N3. Therefore, the state transition relationship between multiple sleep test results can be constrained based on human sleep physiology. For example, if the sleep test result corresponding to EEG data A is sleep state N1, and the sleep test result corresponding to EEG data B is either sleep state N1 or N2, then the state transition relationship between them conforms to the constraint. If the sleep test result corresponding to EEG data A is sleep state N1, and the sleep test result corresponding to EEG data B is sleep state N3, then the state transition relationship between them does not conform to the constraint. A loss can be applied to cases that do not conform to the constraint.

[0092] For example, the following state transition matrix is ​​set for five sleep states: W, N1, N2, N3, and REM: ;Formula (3) This matrix represents the sleep state transition rules. If a direct transition between two sleep states is possible, the corresponding weight is 1. For example, the first row and first column of the matrix represent the weight of transitioning from W to W, and the first row and second column represent the weight of transitioning from W to N1. Since transitioning from W to W and from W to N1 conforms to the physiological laws of sleep, the weight is 1. If a direct transition between two sleep states is not possible, the corresponding weight is 0. For example, the first row and third column of the matrix represent the weight of transitioning from W to N2. Since a direct transition from W to N2 is usually not possible, this weight is 0.

[0093] For two temporally consecutive samples of EEG data x i x j The corresponding sleep test results for each sample are denoted as p. i p j The sleep state transition loss function can be expressed as follows: ;Formula (4) Where Ltrans represents the sleep state transition loss function, N represents the number of sample data, and m(p i ,p j ) represents the state transition matrix M trans p i p j The weight value of the position.

[0094] The total loss function value is determined based on the classification loss function and the sleep state transition loss function, such as by weighting the two according to a preset weight. At least one of the visual model, Transformer, and classification model is trained based on the total loss function value. After each training round, the loss function value is calculated, and the parameters of the visual model, Transformer, and classification model are updated through backpropagation. This iterative training is performed for multiple rounds until the loss function value converges (e.g., the loss fluctuation is less than 1e-5 for 10 consecutive rounds). Then, the training effect of the visual model, Transformer, and classification model is verified using a validation set. If the effect meets the target, the training is complete.

[0095] By incorporating classification loss and sleep state transition loss into the loss function, the trained model can accurately classify data while conforming to human sleep physiology to a certain extent, thus further improving the accuracy of sleep detection results.

[0096] In one implementation, in addition to EEG data, other sleep-related monitoring data of the subject can be acquired to provide more comprehensive information and help achieve more accurate sleep detection. Other sleep-related monitoring data includes, but is not limited to: Electrocardiogram (ECG) data can be obtained through a multi-lead ECG machine. An ECG is a graphical representation of the changes in electrical potentials observed in the heart during each cardiac cycle, caused by the successive excitation of the pacemaker, atria, and ventricles. These changes are captured from the body surface by an electrocardiogram recorder and represent various forms of potential changes. The ECG is an objective indicator of the occurrence, propagation, and recovery of cardiac excitation.

[0097] Electromyographic data (such as electromyography (EMG)) is the temporal and spatial superposition of action potentials of motor units in numerous muscle fibers, and can be obtained by attaching electromyographic sensors to the skin.

[0098] Electrooculogram (EOG) data is a bioelectrical signal generated by the potential difference between the cornea and retina of the eye. It is very easy to collect and can be completed with a small number of electrodes.

[0099] Gastric electrical data (such as electrogastrogram (EGG)) are electrical signals generated by the contraction of stomach muscles and can be collected using electrodes on the surface of the abdominal skin.

[0100] Non-electrical signal data, such as respiratory data (e.g., respiratory rate), blood pressure, etc.

[0101] For electrical signal data other than EEG data, it can be processed in a similar manner to EEG data and combined with the features corresponding to the EEG data (including but not limited to the initial feature map, first intermediate feature, second intermediate feature, third intermediate feature, and fourth intermediate feature). The sleep detection result is determined based on the combined features. For non-electrical signal data, features can be extracted in any way (such as using an encoder) and multimodal fusion can be performed with the features corresponding to the electrical signal data (including but not limited to the initial feature map, first intermediate feature, second intermediate feature, third intermediate feature, and fourth intermediate feature). The sleep detection result is determined based on the fused features.

[0102] For example, when performing sleep testing on a subject, EEG data, ECG data, and respiration data are acquired. On one hand, at least two initial feature maps are determined based on the EEG data, and at least two initial feature maps are determined based on the ECG data. These two initial feature maps are input into a visual model to obtain the first intermediate feature corresponding to the EEG data and the first intermediate feature corresponding to the ECG data. The first intermediate feature corresponding to the EEG data and the first intermediate feature corresponding to the ECG data are combined along the channel dimension. The combined first intermediate feature is then subjected to cascaded dilation convolution processing to obtain the second intermediate feature. The second intermediate feature is converted into a one-dimensional feature, and positional embedding information is added to form the third intermediate feature. The third intermediate feature is input into a Transformer, and the fourth intermediate feature is determined based on the Transformer's output. On the other hand, a feature extraction model is used to extract features from the respiration data, denoted as the fifth intermediate feature. A multimodal fusion model (such as an attention-based model) is used to fuse the fourth and fifth intermediate features, and then a classification model is used to process the fused features to obtain the final sleep test result.

[0103] Exemplary device

[0104] The EEG-based sleep monitoring device according to the embodiments of this disclosure will now be described. (See reference...) Figure 9 As shown, the sleep detection device 900 may include the following modules: The EEG data acquisition module 910 is configured to acquire the EEG data of the test subject. The initial feature map determination module 920 is configured to determine at least two initial feature maps based on the electroencephalogram data. The visual processing module 930 is configured to process the at least two initial feature maps using a visual model to obtain a first intermediate feature; The dilated convolution module 940 is configured to perform cascaded dilated convolution processing on the first intermediate feature to obtain the second intermediate feature; The detection result determination module 950 is configured to determine the sleep detection result corresponding to the EEG data based on the second intermediate feature.

[0105] The above modules can be program modules or hardware modules.

[0106] In one embodiment, the EEG data includes brain electrode data and ear electrode data; the EEG data acquisition module 910 is configured to preprocess the EEG data of the subject by: determining first reference electrode data; determining unipolar lead data based on the brain electrode data and the first reference electrode data; determining second reference electrode data based on the ear electrode data when the reference electrode is changed; determining bipolar lead data based on the brain electrode data and the second reference electrode data; and combining the unipolar lead data and the bipolar lead data to obtain preprocessed EEG data. The step of determining at least two initial feature maps based on the EEG data includes: determining at least two initial feature maps based on the preprocessed EEG data.

[0107] In one embodiment, the brain electrode data includes left hemisphere electrode data, midline electrode data, and right hemisphere electrode data; the ear electrode data includes left ear electrode data and right ear electrode data; the step of determining second reference electrode data based on the ear electrode data when the reference electrode is changed, and determining bipolar lead data based on the brain electrode data and the second reference electrode data, includes: for the left hemisphere electrode data and midline electrode data, using the right ear electrode data as the second reference electrode data, calculating the difference between the left hemisphere electrode data, the midline electrode data, and the right ear electrode data respectively; for the right hemisphere electrode data, using the left ear electrode data as the second reference electrode data, calculating the difference between the right hemisphere electrode data and the left ear electrode data, to obtain the bipolar lead data.

[0108] In one embodiment, the at least two initial feature maps include at least two of a first initial feature map, a second initial feature map, a third initial feature map, and a fourth initial feature map; determining the at least two initial feature maps based on the EEG data includes at least the following at least two methods: converting the EEG data into a first initial feature map using the number of channels of the EEG data as the height and the dimension in each channel as the width; performing a dimension transformation on the first initial feature map to obtain a second initial feature map; the difference between the height and width of the second initial feature map is less than the difference between the height and width of the first initial feature map; converting the EEG data into a time-frequency feature, and converting the time-frequency feature into a third initial feature map using the time dimension and frequency dimension of the time-frequency feature as the height and width; obtaining the EEG corresponding to the EEG data, and determining the fourth initial feature map based on the EEG.

[0109] In one embodiment, the step of processing the at least two initial feature maps using a visual model to obtain a first intermediate feature includes: processing each initial feature map using a visual model to obtain a visual feature corresponding to each initial feature map; and concatenating the visual features corresponding to each initial feature map to form the first intermediate feature.

[0110] In one embodiment, the step of performing cascaded dilation convolution processing on the first intermediate feature to obtain the second intermediate feature includes: acquiring one or more dilation parameter sequences, each dilation parameter sequence including one dilation rate or multiple gradually increasing dilation rates; performing serial dilation convolution processing on the first intermediate feature using each dilation parameter sequence to obtain the corresponding convolution result; and fusing the convolution results corresponding to each dilation parameter sequence to obtain the second intermediate feature.

[0111] In one embodiment, the first intermediate feature includes a first intermediate feature corresponding to each initial feature map; the step of fusing the convolution results corresponding to each dilation parameter sequence to obtain the second intermediate feature includes: performing a first-level fusion of the convolution results corresponding to different initial feature maps and the same dilation parameter sequence, and then performing a second-level fusion of the first-level fusion results corresponding to different dilation parameter sequences to obtain the second intermediate feature.

[0112] In one implementation, determining the sleep detection result corresponding to the EEG data based on the second intermediate feature includes: converting the second intermediate feature into a one-dimensional feature and adding location embedding information to form a third intermediate feature; inputting the third intermediate feature into a Transformer and determining a fourth intermediate feature based on the information output by the Transformer; and processing the fourth intermediate feature using a classification model to obtain the sleep detection result.

[0113] In one implementation, the step of converting the second intermediate feature into a one-dimensional feature and adding position embedding information to form a third intermediate feature includes: converting the second intermediate feature into a one-dimensional feature and adding position embedding information and a classification label to form a third intermediate feature; the step of determining a fourth intermediate feature based on the information output by the Transformer includes: extracting the information corresponding to the classification label from the information output by the Transformer as the fourth intermediate feature.

[0114] In one embodiment, the device is further configured to: acquire sample EEG data and sleep tags corresponding to the sample EEG data; determine at least two initial feature maps of the samples based on the sample EEG data, and input them into the visual model to obtain a first sample intermediate feature; perform cascaded dilated convolution processing on the first sample intermediate feature to obtain a second sample intermediate feature; convert the second sample intermediate feature into a one-dimensional feature and add position embedding information to form a third sample intermediate feature; input the third sample intermediate feature into the Transformer, and determine a fourth sample intermediate feature based on the information output by the Transformer; process the fourth sample intermediate feature using the classification model to obtain a sample sleep detection result; determine a loss function value based on the sample sleep detection result and the sleep tag; and update the parameters of at least one of the visual model, the Transformer, and the classification model based on the loss function value.

[0115] In one implementation, determining the loss function value based on the sample sleep detection results and the sleep label includes: constructing a classification loss function based on the deviation between the sample sleep detection results and the sleep label, and based on focus loss and L2 regularization loss; constructing a sleep state transition loss function based on the sleep detection results of multiple sample EEG data corresponding to multiple samples with temporal relationships; and determining the loss function value based on the classification loss function and the sleep state transition loss function.

[0116] Furthermore, other specific details of the embodiments of this disclosure have been described in detail in the embodiments of the above methods, and will not be repeated here.

[0117] Exemplary program product

[0118] The computer program product in the embodiments of this disclosure will now be described. The computer program product includes a computer program that, when executed by a processor, implements the methods described above in this disclosure.

[0119] In one implementation, the computer program product can be a tangible product, such as a computer-readable storage medium storing a computer program. The readable storage medium can be based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, and includes, but is not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. Exemplarily, the computer program product can be a non-volatile storage medium storing a computer program, such as read-only memory, NAND flash memory, etc.

[0120] In one implementation, the computer program product can be an intangible product. For example, the computer program product can be a virtual digital product, such as an executable file or installation package containing a computer program.

[0121] Computer program code can be written in one or more programming languages. Programming languages ​​include, but are not limited to, C, Java, and C++. Program code can execute entirely on the user's computing device, or partially on the user's computing device, or as a standalone software package, or partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).

[0122] Computer programs can be carried or transmitted via signals such as electrical, magnetic, optical, electromagnetic, and infrared rays. Electronic devices can convert the signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, to be executed by the processor of the electronic device) the method steps of various embodiments of this disclosure, such as... Figure 2 The steps are shown.

[0123] By executing the above methods and steps through a computer program, the following technical effects can be achieved: First, it provides an automated sleep detection solution, solving the problems of time-consuming, labor-intensive, and inefficient manual analysis of EEG data, thus improving sleep detection efficiency. It can quickly process large-scale EEG data, shorten sleep detection time, and meet the needs of large-scale sleep disorder screening and long-term sleep monitoring. Second, it determines at least two initial feature maps based on EEG data, improving the comprehensiveness of initial feature information. Standardized feature initialization is achieved through a visual model, multi-scale features are captured through cascaded dilated convolution, and sleep detection results are obtained through subsequent feature processing. This achieves an objective and unified feature extraction and detection logic, overcoming the limitations of incomplete and insufficient feature extraction in related technologies, improving the accuracy of sleep detection, and solving the problems of strong subjectivity and low consistency in manual analysis, thereby improving the reliability and comparability of detection results. Third, this solution has good versatility and scalability, adapting to different devices and different formats of EEG data, and can be widely applied to various scenarios such as sleep disorder diagnosis and home sleep monitoring.

[0124] Exemplary electronic devices

[0125] The electronic device in the embodiments of this disclosure will now be described. The electronic device may be... Figure 1Any device in the present disclosure. An electronic device includes a processor and a memory for storing executable instructions of the processor. The processor is configured to perform the methods described above by executing the executable instructions.

[0126] refer to Figure 10 An electronic device according to embodiments of the present disclosure will be described by way of example. Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0127] like Figure 10 As shown, the electronic device 1000 is presented in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: a processor 1010, a memory 1020, a bus 1030 connecting different system components (including the memory 1020 and the processor 1010), an I / O (input / output) interface 1040, and a network adapter 1050.

[0128] The memory 1020 stores program code, which can be executed by the processor 1010, causing the processor 1010 to perform the method steps of this embodiment. For example... Figure 2 The steps are shown.

[0129] By executing the above method steps through processor 1010, the following technical effects can be achieved: First, an automated sleep detection solution is provided, solving the problems of time-consuming, labor-intensive, and inefficient manual analysis of EEG, improving sleep detection efficiency, enabling rapid processing of large-scale EEG data, shortening sleep detection time, and meeting the needs of large-scale sleep disorder screening and long-term sleep monitoring. Second, at least two initial feature maps are determined based on EEG data, improving the comprehensiveness of initial feature information. Standardized feature initialization is achieved through a visual model, multi-scale features are captured through cascaded dilated convolution, and sleep detection results are obtained through subsequent feature processing. This achieves an objective and unified feature extraction and detection logic, overcoming the limitations of incomplete and insufficient feature extraction in related technologies, improving the accuracy of sleep detection, and solving the problems of strong subjectivity and low consistency in manual analysis, thus improving the reliability and comparability of detection results. Third, this solution has good versatility and scalability, can be adapted to different devices and different formats of EEG data, and can be widely applied to various scenarios such as sleep disorder diagnosis and home sleep monitoring.

[0130] The memory 1020 may include volatile memory, such as random access memory (RAM) 1021 and / or cache unit 1022, and may also include non-volatile memory, such as read-only memory (ROM) unit 1023. The memory 1020 may also include one or more program modules 1024, such program modules 1024 including, but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. For example, program module 1024 may include the modules described above.

[0131] The processor 1010 may include processing units such as AP (Application Processor), modem processor, GPU (Graphics Processing Unit), ISP (Image Signal Processor), controller, encoder, decoder, DSP (Digital Signal Processor), baseband processor and / or NPU (Neural-Network Processing Unit).

[0132] Bus 1030 is used to connect different parts of electronic device 1000, and may include data bus, address bus and control bus, etc.

[0133] The electronic device 1000 can also communicate with one or more external devices 1100 (such as keyboards, pointing devices, Bluetooth devices, etc.), and this communication can be performed through the I / O interface 1040.

[0134] Electronic device 1000 can also communicate with one or more networks via network adapter 1050. For example, network adapter 1050 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication. Network adapter 1050 can communicate with other modules of electronic device 1000 via bus 1030.

[0135] Although not shown in the figure, other hardware and / or software modules may be installed on the electronic device 1000, including but not limited to: display, microcode, device driver, redundant processing unit, external disk drive array, RAID (Redundant Arrays of Independent Disks) system, tape drive, and data backup storage system.

[0136] It should be noted that although several modules or sub-modules of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0137] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0138] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A sleep detection method based on electroencephalography (EEG), characterized in that, The method includes: Acquire electroencephalogram (EEG) data from the test subjects; At least two initial feature maps are determined based on the electroencephalogram data; The at least two initial feature maps are processed using a visual model to obtain a first intermediate feature; The first intermediate feature is subjected to cascaded dilatation convolution to obtain the second intermediate feature; The sleep detection result corresponding to the EEG data is determined based on the second intermediate feature.

2. The method according to claim 1, characterized in that, The EEG data includes brain electrode data and ear electrode data; after acquiring the EEG data of the subject, it is preprocessed in the following manner: Determine the first reference electrode data, and determine the unipolar lead data based on the brain electrode data and the first reference electrode data; When the reference electrode is changed, second reference electrode data is determined based on the ear electrode data, and bipolar lead data is determined based on the brain electrode data and the second reference electrode data. The monopolar lead data and the bipolar lead data are combined to obtain preprocessed EEG data; The determination of at least two initial feature maps based on the EEG data includes: At least two initial feature maps are determined based on the preprocessed EEG data.

3. The method according to claim 2, characterized in that, The brain electrode data includes left hemisphere electrode data, midline electrode data, and right hemisphere electrode data; the ear electrode data includes left ear electrode data and right ear electrode data; the step of determining second reference electrode data based on the ear electrode data when the reference electrode is changed, and determining bipolar lead data based on the brain electrode data and the second reference electrode data, includes: For the left hemisphere electrode data and midline electrode data, with the right ear electrode data as the second reference electrode data, the difference between the left hemisphere electrode data, the midline electrode data and the right ear electrode data is calculated respectively. For the right hemisphere electrode data, with the left ear electrode data as the second reference electrode data, the difference between the right hemisphere electrode data and the left ear electrode data is calculated to obtain the bipolar lead data.

4. The method according to claim 1, characterized in that, The at least two initial feature maps include at least two of a first initial feature map, a second initial feature map, a third initial feature map, and a fourth initial feature map; the determination of at least two initial feature maps based on the EEG data includes at least two of the following methods: Using the number of channels in the EEG data as the height and the dimension in each channel as the width, the EEG data is converted into a first initial feature map; The first initial feature map is dimensionally transformed to obtain a second initial feature map; the difference between the height and width of the second initial feature map is smaller than the difference between the height and width of the first initial feature map. The EEG data is converted into time-frequency features, and the time and frequency dimensions of the time-frequency features are used as the height and width to convert the time-frequency features into a third initial feature map. Obtain the electroencephalogram (EEG) corresponding to the EEG data, and determine the fourth initial feature map based on the EEG.

5. The method according to claim 1, characterized in that, The process of using a visual model to process the at least two initial feature maps to obtain a first intermediate feature includes: Each initial feature map is processed using a visual model to obtain the visual features corresponding to each initial feature map; The visual features corresponding to each initial feature map are spliced ​​together to form the first intermediate feature.

6. The method according to claim 1, characterized in that, The step of performing cascaded dilated convolution on the first intermediate feature to obtain the second intermediate feature includes: Obtain one or more sequences of expansion parameters, each sequence including one expansion rate or multiple gradually increasing expansion rates; Each dilation parameter sequence is used to perform a sequential dilation convolution process on the first intermediate feature to obtain the corresponding convolution result; The convolution results corresponding to each dilation parameter sequence are fused to obtain the second intermediate feature.

7. The method according to claim 6, characterized in that, The first intermediate feature includes the first intermediate feature corresponding to each initial feature map; The step of fusing the convolution results corresponding to each dilation parameter sequence to obtain the second intermediate feature includes: The convolution results corresponding to different initial feature maps and the same dilation parameter sequence are fused at the first level, and then the first-level fusion results corresponding to different dilation parameter sequences are fused at the second level to obtain the second intermediate feature.

8. The method according to claim 1, characterized in that, The step of determining the sleep detection result corresponding to the EEG data based on the second intermediate feature includes: The second intermediate feature is converted into a one-dimensional feature, and positional embedding information is added to form the third intermediate feature; The third intermediate feature is input into the Transformer, and the fourth intermediate feature is determined based on the information output by the Transformer. The fourth intermediate feature is processed using a classification model to obtain the sleep detection result.

9. The method according to claim 8, characterized in that, The step of converting the second intermediate feature into a one-dimensional feature and adding positional embedding information to form the third intermediate feature includes: The second intermediate feature is converted into a one-dimensional feature, and position embedding information and classification labels are added to form the third intermediate feature; The step of determining the fourth intermediate feature based on the information output by the Transformer includes: The information corresponding to the classification label is extracted from the information output by the Transformer and used as the fourth intermediate feature.

10. The method according to claim 8, characterized in that, The method further includes: Obtain sample EEG data and the corresponding sleep tags for the sample EEG data; Based on the sample EEG data, at least two initial feature maps for the samples are determined and input into the visual model to obtain the intermediate features of the first sample. The intermediate features of the first sample are subjected to cascaded dilated convolution to obtain the intermediate features of the second sample. The intermediate features of the second sample are converted into one-dimensional features, and position embedding information is added to form the intermediate features of the third sample. The intermediate features of the third sample are input into the Transformer, and the intermediate features of the fourth sample are determined based on the information output by the Transformer. The intermediate features of the fourth sample are processed using the classification model to obtain the sample sleep detection results; The loss function value is determined based on the sample sleep detection results and the sleep tags; The parameters of at least one of the visual model, the Transformer, and the classification model are updated based on the loss function value.

11. The method according to claim 10, characterized in that, The step of determining the loss function value based on the sample sleep detection results and the sleep tags includes: Based on the deviation between the sample sleep detection results and the sleep labels, and using focus loss and L2 regularization loss, a classification loss function is constructed. A sleep state transition loss function is constructed based on the sleep detection results of multiple samples of EEG data with temporal relationship. The loss function value is determined based on the classification loss function and the sleep state transition loss function.

12. A sleep detection device based on electroencephalography (EEG), characterized in that, The device includes: The EEG data acquisition module is configured to acquire the EEG data of the test subject; An initial feature map determination module is configured to determine at least two initial feature maps based on the EEG data; The visual processing module is configured to process the at least two initial feature maps using a visual model to obtain a first intermediate feature; The dilated convolution module is configured to perform cascaded dilated convolution processing on the first intermediate feature to obtain the second intermediate feature; The detection result determination module is configured to determine the sleep detection result corresponding to the EEG data based on the second intermediate feature.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 11.

14. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 11 by executing the executable instructions.

Images