Brain abnormal discharge detection method and device, storage medium and electronic equipment

Abstract

The invention relates to the field of electroencephalogram detection, in particular to a brain abnormal discharge detection method and device, a storage medium and electronic equipment. The abnormal brain discharge detection method comprises the following steps: acquiring electroencephalogram signal data of a detected object, and acquiring a state video of the detected object at the same time; identifying attitude information and face information of the detection object in the state video to obtain key point space trajectory data; and inputting the electroencephalogram signal data and the key point space trajectory data into a pre-trained anomaly prediction model to obtain an electroencephalogram information anomaly prediction result output by the anomaly prediction model, and taking the electroencephalogram information anomaly prediction result as a brain abnormal discharge detection result. The brain abnormal discharge detection method provided by the invention can improve the accuracy and efficiency of the brain abnormal discharge detection result.

Description

Technical Field

[0001] This disclosure relates to the field of electroencephalography (EEG) detection, specifically to a method, device, storage medium, and electronic device for detecting abnormal brain discharges. Background Technology

[0002] Intracranial electroencephalography (EEG) technology uses sophisticated electronic instruments to amplify and record spontaneous bioelectric potentials in the brain from the scalp. The recorded information has great analytical value and helps to assess changes in the brain electrical activity of the person being tested.

[0003] However, current data analysis of acquired intracranial electroencephalogram (EEG) data generally relies on doctors' experience. Furthermore, automatic identification of intracranial EEG information is quite challenging; besides the need for extensive sample identification of the discharge characteristics themselves, artifact interference is also a major factor. Therefore, existing methods for detecting abnormal brain discharges are not only costly and time-consuming, but also prone to misinterpretation, and the consistency of judgments among different doctors is low.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a method, device, storage medium, and electronic device for detecting abnormal brain discharges, with the aim of improving the accuracy of abnormal brain discharge detection results.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of the present disclosure, a method for detecting abnormal brain discharges is provided, comprising: acquiring electroencephalogram (EEG) signal data of a detection subject and simultaneously acquiring a state video of the detection subject; identifying posture information and facial information of the detection subject in the state video to obtain key point spatial trajectory data; and inputting the EEG signal data and the key point spatial trajectory data into a pre-trained abnormal prediction model to obtain an abnormal prediction result of EEG information output by the abnormal prediction model as a result of detecting abnormal brain discharges.

[0008] According to some embodiments of this disclosure, based on the foregoing scheme, the key point spatial trajectory data includes posture key point spatial data and facial key point spatial data of each frame image in the state video; the step of identifying the posture information and facial information of the detected object in the state video to obtain key point spatial trajectory data includes: creating a human posture instance and a facial point instance, and loading the state video; identifying the posture key points of the detected object based on the human posture instance, and obtaining the spatial coordinate data of the posture key points as the posture key point spatial data; and identifying the facial key points of the detected object based on the facial point instance, and obtaining the spatial coordinate data of the facial key points as the facial key point spatial data.

[0009] According to some embodiments of this disclosure, based on the foregoing scheme, the method further includes: identifying the posture key points based on the human posture monitoring model in MediaPipe to obtain the posture key point spatial data; and identifying the facial key points based on the facial point monitoring model in MediaPipe to obtain the facial key point spatial data.

[0010] According to some embodiments of this disclosure, based on the foregoing scheme, the method further includes: applying a moving average filter to smooth the key point spatial trajectory data to obtain first key point spatial trajectory data, so as to update the key point spatial trajectory data.

[0011] According to some embodiments of this disclosure, based on the foregoing scheme, the method further includes: performing non-maximum suppression on the key point spatial trajectory data to obtain second key point spatial trajectory data, so as to update the key point spatial trajectory data.

[0012] According to some embodiments of this disclosure, based on the foregoing scheme, the method further includes pre-training the anomaly prediction model. Pre-training the anomaly prediction model includes: acquiring training samples; wherein the training samples include training data of the electroencephalogram (EEG) signals of the detection object, training data of key point spatial trajectories, and labeling results; the labeling results are used to characterize the abnormal brain discharge detection results of the training samples; inputting the EEG signal training data and the key point spatial trajectory training data into an initial anomaly prediction model to calculate an abnormal EEG information prediction result; calculating a loss function based on the labeling results and the abnormal EEG information prediction result; and adjusting the model parameters of the initial anomaly prediction model based on the loss function to obtain a trained anomaly prediction model.

[0013] According to some embodiments of this disclosure, based on the foregoing scheme, the method further includes: acquiring historical state video; determining whether the detected object is abnormal based on the historical state video to obtain a detection result; and marking the historical state video based on the detection result to obtain the historical state video with the marked result.

[0014] According to a second aspect of the present disclosure, a device for detecting abnormal brain discharges is provided, comprising: a data acquisition module for acquiring electroencephalogram (EEG) signal data of a detection subject and simultaneously acquiring a state video of the detection subject; a recognition module for recognizing posture information and facial information of the detection subject in the state video to obtain key point spatial trajectory data; and a prediction module for detecting abnormal brain discharges by inputting the EEG signal data and the key point spatial trajectory data into a pre-trained abnormal prediction model to obtain an abnormal prediction result of the EEG information output by the abnormal prediction model as the detection result of abnormal brain discharges.

[0015] According to a third aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the brain abnormal discharge detection method as described in the above embodiments.

[0016] According to a fourth aspect of the present disclosure, an electronic device is provided, characterized in that it includes: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the brain abnormal discharge detection method as described in the above embodiments.

[0017] The exemplary embodiments disclosed herein may have some or all of the following beneficial effects:

[0018] In some embodiments of this disclosure, the technical solutions provided include, on the one hand, improving the accuracy and efficiency of abnormal brain discharge detection results by jointly analyzing the EEG signal data and state video of the detection subject and reducing the interference of artifacts using the state video; on the other hand, providing an automated method for detecting abnormal brain discharge from information acquisition to information processing, adopting a unified evaluation standard, which, compared with manual data analysis, can reduce human bias, improve accuracy, shorten detection time, and improve detection efficiency.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0021] Figure 1 The schematic diagram illustrates a flowchart of a method for detecting abnormal brain discharges in an exemplary embodiment of this disclosure;

[0022] Figure 2 A schematic diagram illustrating a multichannel electroencephalogram (EEG) waveform in an exemplary embodiment of the present disclosure is shown.

[0023] Figure 3 This schematic diagram illustrates a key point space diagram in an exemplary embodiment of the present disclosure;

[0024] Figure 4 This illustration schematically shows a diagram of training an anomaly prediction model in an exemplary embodiment of the present disclosure;

[0025] Figure 5 This illustration schematically shows a method for obtaining detection results of abnormal brain discharges in an exemplary embodiment of the present disclosure;

[0026] Figure 6 This schematic diagram illustrates the composition of a brain abnormal discharge detection device according to an exemplary embodiment of the present disclosure;

[0027] Figure 7 This schematic diagram illustrates a computer-readable storage medium according to an exemplary embodiment of the present disclosure;

[0028] Figure 8 The schematic diagram illustrates the structure of a computer system of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0030] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0032] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0033] Intracranial electroencephalography (EEG) technology uses sophisticated electronic instruments to amplify and record spontaneous bioelectric potentials in the brain from the scalp. The recorded information has great analytical value and helps to assess changes in the brain electrical activity of the person being tested.

[0034] However, in the current process of analyzing the acquired intracranial electroencephalogram (EEG) data, it is generally done by doctors based on their experience. This is not only costly and time-consuming, but also prone to misjudgment, and the consistency of judgments among different doctors is low.

[0035] Furthermore, the automatic identification of intracranial electroencephalogram (EEG) information is quite challenging. Besides the need for a large number of samples to identify the characteristics of the discharges themselves, artifacts are a major contributing factor. This is because the subjects being tested are inevitably subject to interference from electromyography (EMG) and motor artifacts when awake with eyes closed or in a state of impaired consciousness. They are also susceptible to environmental influences, which further affect the accuracy of the assessment.

[0036] Therefore, in view of the problems in the existing technology, the demand for automatic EEG analysis is increasing. Therefore, this disclosure provides a method for detecting abnormal brain discharges. By using video to further eliminate motion and environmental artifacts, and by combining modeling and machine learning, it is possible to establish standardized evaluation rules, reduce the workload of manual analysis, reduce human bias, and improve accuracy.

[0037] The implementation details of the technical solutions of the embodiments of this disclosure are described in detail below.

[0038] Figure 1 This illustration schematically shows a flowchart of a method for detecting abnormal brain discharges according to an exemplary embodiment of this disclosure. Figure 1 As shown, the method for detecting abnormal brain discharges includes steps S101 to S103:

[0039] Step S101: Collect the electroencephalogram (EEG) signal data of the test subject, and simultaneously collect the state video of the test subject;

[0040] Step S102: Identify the pose information and facial information of the detected object in the state video to obtain key point spatial trajectory data;

[0041] Step S103: Input the EEG signal data and the key point spatial trajectory data into the pre-trained anomaly prediction model to obtain the EEG information anomaly prediction result output by the anomaly prediction model as the brain abnormal discharge detection result.

[0042] Based on the above scheme, on the one hand, by jointly analyzing the EEG signal data and state video of the test subjects, the interference of artifacts can be reduced by utilizing the state video, which can improve the accuracy and efficiency of abnormal brain discharge detection results; on the other hand, an automated method for detecting abnormal brain discharge from information acquisition to information processing is provided. It adopts a unified evaluation standard, which, compared with manual data analysis methods, can reduce human bias, improve accuracy, shorten detection time, and improve detection efficiency.

[0043] The steps of the brain abnormal discharge detection method in this exemplary embodiment will now be described in more detail with reference to the accompanying drawings and embodiments.

[0044] In step S101, the electroencephalogram (EEG) signal data of the test subject is collected, and the state video of the test subject is collected simultaneously.

[0045] Specifically, brainwave signal data of the subject can be acquired using a multichannel electroencephalogram (EEG). A multichannel EEG is an advanced medical device with multiple electrode channels, capable of simultaneously recording the electrical activity of multiple brain regions, thus providing more comprehensive information on brain function.

[0046] Figure 2 This diagram schematically illustrates a multi-channel electroencephalogram (EEG) waveform according to an exemplary embodiment of this disclosure. EEG signals are acquired using a multi-channel EEG machine, and the acquired real-time signals are visualized as follows. Figure 2 As shown.

[0047] While collecting EEG signal data, it is also necessary to collect video footage of the subject's state. This video footage mainly includes the subject's facial and torso movements over time. For example, a camera can be used to capture video footage of the subject's face and torso.

[0048] Step S102: Identify the pose information and facial information of the detected object in the state video to obtain key point spatial trajectory data.

[0049] Specifically, in one embodiment of this disclosure, the specific steps of identifying key point spatial trajectory data in step S102 include: creating a human posture instance and a facial point instance, and loading the state video; identifying the posture key points of the detected object based on the human posture instance, and obtaining the spatial coordinate data of the posture key points as the posture key point spatial data; and identifying the facial key points of the detected object based on the facial point instance, and obtaining the spatial coordinate data of the facial key points as the facial key point spatial data.

[0050] In one embodiment of this disclosure, the method further includes: identifying the posture key points based on a human posture monitoring model in MediaPipe to obtain spatial data of the posture key points; and identifying the facial key points based on a facial point monitoring model in MediaPipe to obtain spatial data of the facial key points.

[0051] Specifically, MediaPipe can be used for recognition. Each frame of the image is processed using the human pose detection model in MediaPipe, identifying pose keypoints and plotting the pose results to obtain spatial data of the pose keypoints. Similarly, each frame of the image is processed using the facial landmark detection model in MediaPipe, identifying facial keypoints and plotting the facial landmarks to obtain spatial data of the facial landmarks.

[0052] The specific process of identifying key point spatial trajectory data using the mediapipe method is as follows:

[0053] (1) Create instances of Mediapipe for human pose and facial points;

[0054] (2) Initialize the human body posture and facial positioning modules;

[0055] (3) Load video using OpenCV;

[0056] (4) Use the human pose monitoring model in MediaPipe to process each frame of image to obtain pose key point spatial data pose_landmarks;

[0057] (5) Use the face landmark detection model in MediaPipe to process each frame of the image to obtain the spatial data of pose key points face_landmarks.

[0058] Figure 3 This illustration schematically depicts a keypoint space diagram in an exemplary embodiment of the present disclosure. For example... Figure 3 As shown, the key points include pose key points and facial key points. Each frame of the image records the x and y coordinates of each key point in real time. By stringing together each frame, the spatial trajectory data of the key points can be obtained.

[0059] In one embodiment of this disclosure, the method further includes: applying a moving average filter to smooth the key point spatial trajectory data to obtain first key point spatial trajectory data, so as to update the key point spatial trajectory data.

[0060] Specifically, the identified landmark spatial data can be smoothed. Moving average filters are primarily used to smooth the data, reducing the impact of random fluctuations and making the results more stable. In the case of landmarks, moving average filters can be used to predict their locations, reducing jitter caused by noise or false detections.

[0061] In one embodiment of this disclosure, the method further includes: performing non-maximum suppression on the key point spatial trajectory data to obtain second key point spatial trajectory data, so as to update the key point spatial trajectory data.

[0062] Specifically, non-maximum suppression (NMS) post-processing can be applied to the identified landmark spatial data. NMS is mainly used to remove redundant detection boxes or keypoints, preserving the best detection results. In landmark detection, NMS can help remove overlapping or redundant keypoints, ensuring the accuracy of the final result.

[0063] It should be noted that the smoothing post-processing and non-maximum suppression post-processing of the moving average filter can also be applied to the spatial trajectory data of key points simultaneously, and there is no limitation on the order of processing. Those skilled in the art will readily realize that the spatial trajectory data of key points can be smoothed first and then subjected to non-maximum suppression, or non-maximum suppression can be performed first and then smoothed.

[0064] Step S103: Input the EEG signal data and the key point spatial trajectory data into the pre-trained anomaly prediction model to obtain the EEG information anomaly prediction result output by the anomaly prediction model as the brain abnormal discharge detection result.

[0065] Specifically, abnormal prediction results of EEG information can be obtained through an anomaly prediction model. Therefore, before step S103, the method further includes pre-training the anomaly prediction model. This specifically includes the following steps: obtaining training samples; wherein, the training samples include EEG signal training data of the detection object, key point spatial trajectory training data, and labeling results; the labeling results are used to characterize the abnormal brain discharge detection results of the training samples; inputting the EEG signal training data and the key point spatial trajectory training data into an initial anomaly prediction model to calculate the abnormal prediction results of EEG information; calculating a loss function based on the labeling results and the abnormal prediction results of EEG information; adjusting the model parameters of the initial anomaly prediction model based on the loss function to obtain a trained anomaly prediction model.

[0066] The training samples are pre-collected and include training data of the EEG signals of the test subjects, training data of spatial trajectories of key points, and labeling results.

[0067] The EEG signal training data and key point spatial trajectory training data can be obtained by referring to steps S101 and S102, which will not be elaborated further here. The labeling result is the detection result of abnormal brain discharges of the detection object under this type of EEG signal data and key point spatial trajectory data, including whether the EEG information is normal or abnormal. Specifically, the method further includes: acquiring a training state video corresponding to the EEG signal training data; and labeling whether the EEG information is abnormal according to the training state video to obtain the labeling result. That is to say, the labeling can be done in advance according to the training state video as a real abnormal brain discharge detection result for subsequent model training and parameter correction.

[0068] It should be noted that, in addition to constructing training samples as described above, relevant information of the test subjects that have stored abnormal brain discharge detection results can also be retrieved from historical data, and then the training sample data can be obtained by processing the relevant information.

[0069] The initial anomaly prediction model is the basic neural network model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), etc. This disclosure does not make any specific limitations.

[0070] Figure 4 The illustration shows a schematic diagram of a training anomaly prediction model in an exemplary embodiment of the present disclosure, specifically including the following:

[0071] First, training samples are acquired, including EEG data, trajectory data, and labeling results. These are then input into a neural network model, which outputs prediction results. Next, a loss is calculated based on the prediction output and labeling results. Finally, the model parameters of the neural network are adjusted based on the loss until training is complete, resulting in a final trained neural network model, which serves as the anomaly prediction model. The input to the anomaly prediction model is EEG signal data and keypoint spatial trajectory data; the output is the anomaly prediction result based on EEG information.

[0072] Therefore, after the abnormality prediction model is pre-trained, when the EEG signal data is obtained through step S101 and the key point spatial trajectory data is identified through step S102, the two data are input into the abnormality prediction model, and the abnormal prediction result of the EEG information output by the abnormality prediction model can be obtained as the detection result of abnormal brain discharge of the detection object.

[0073] Figure 5 This schematic diagram illustrates an exemplary embodiment of the present disclosure of a method for obtaining detection results of abnormal brain discharges, such as... Figure 5 As shown, the complete process for obtaining the detection results of abnormal brain discharges is as follows:

[0074] First, an electroencephalogram (EEG) machine is used to collect EEG signal data from the subject, obtaining multi-channel EEG information. Simultaneously, a camera is used to capture video of the subject's state, including the face and torso. Then, a posture and facial key point monitoring model is used to identify the coordinates of key points, such as the eyes, mouth, head, and limbs. Next, the EEG signal data and key point spatial trajectory data are input into an anomaly prediction model in segments of n seconds, and the output EEG information anomaly prediction result is used as the detection result of abnormal brain discharge in the subject.

[0075] Therefore, based on the above methods, this disclosure establishes a multimodal artificial intelligence EEG interpretation model. By combining the state video of the detected object, it improves the accuracy of existing artificial intelligence models, increasing both the efficiency and accuracy of anomaly detection. Anomaly detection results can provide auxiliary support for preoperative localization of various diseases, helping to increase surgical success rates, improve patients' quality of life, and reduce the burden on society and families.

[0076] Figure 6 This schematic diagram illustrates the composition of a brain abnormal discharge detection device according to an exemplary embodiment of the present disclosure, such as... Figure 6 As shown, the brain abnormal discharge detection device 600 may include a data acquisition module 601, a recognition module 602, and a prediction module 603. Wherein:

[0077] The acquisition module 600 is used to acquire the electroencephalogram (EEG) signal data of the detection object and simultaneously acquire the state video of the detection object;

[0078] The recognition module 601 is used to recognize the posture information and facial information of the detected object in the state video to obtain key point spatial trajectory data;

[0079] The prediction module 602 is used to detect the abnormality prediction results of the brain electrical signal data and the spatial trajectory data of the key points by inputting them into a pre-trained abnormality prediction model, so as to obtain the abnormality prediction results of the brain electrical information output by the abnormality prediction model as the detection results of abnormal brain discharge.

[0080] According to an exemplary embodiment of this disclosure, the key point spatial trajectory data includes posture key point spatial data and facial key point spatial data of each frame image in the state video; the recognition module 601 is further configured to create human posture instances and facial point instances, and load the state video; recognize the posture key points of the detected object based on the human posture instances, and obtain the spatial coordinate data of the posture key points as the posture key point spatial data; and recognize the facial key points of the detected object based on the facial point instances, and obtain the spatial coordinate data of the facial key points as the facial key point spatial data.

[0081] According to an exemplary embodiment of this disclosure, the identification module 601 is further configured to identify the posture key points based on the human posture monitoring model in MediaPipe to obtain the posture key point spatial data; and to identify the facial key points based on the facial point monitoring model in MediaPipe to obtain the facial key point spatial data.

[0082] According to an exemplary embodiment of the present disclosure, the identification module 601 is further configured to apply a moving average filter to smooth the key point spatial trajectory data to obtain first key point spatial trajectory data, so as to update the key point spatial trajectory data.

[0083] According to an exemplary embodiment of the present disclosure, the identification module 601 is further configured to perform non-maximum suppression on the key point spatial trajectory data to obtain second key point spatial trajectory data, so as to update the key point spatial trajectory data.

[0084] According to an exemplary embodiment of this disclosure, the method further includes pre-training the abnormality prediction model, and the brain abnormal discharge detection device 600 further includes a training module for acquiring training samples; wherein, the training samples include EEG signal training data of the detection object, key point spatial trajectory training data, and labeling results; the labeling results are used to characterize the brain abnormal discharge detection results of the training samples; the EEG signal training data and the key point spatial trajectory training data are input into an initial abnormality prediction model to calculate the EEG information abnormality prediction result; a loss function is calculated based on the labeling results and the EEG information abnormality prediction result; the model parameters of the initial abnormality prediction model are adjusted based on the loss function to obtain a trained abnormality prediction model.

[0085] According to an exemplary embodiment of this disclosure, the training module is further configured to acquire a training state video corresponding to the EEG signal training data; and to mark whether the EEG information is abnormal based on the training state video to obtain the marking result.

[0086] The specific details of each module in the aforementioned brain abnormal discharge detection device 600 have been described in detail in the corresponding brain abnormal discharge detection method, so they will not be repeated here.

[0087] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0088] In an exemplary embodiment of this disclosure, a storage medium capable of implementing the above-described method is also provided. Figure 7 This schematic diagram illustrates a computer-readable storage medium according to an exemplary embodiment of the present disclosure, such as... Figure 7 As shown, a program product 700 for implementing the above-described method according to an embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a mobile phone. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0089] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided. Figure 8 The schematic diagram illustrates the structure of a computer system of an electronic device according to an exemplary embodiment of the present disclosure.

[0090] It should be noted that, Figure 8 The computer system 800 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0091] like Figure 8 As shown, the computer system 800 includes a Central Processing Unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 802 or programs loaded from storage section 808 into Random Access Memory (RAM) 803. The RAM 803 also stores various programs and data required for system operation. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An Input / Output (I / O) interface 805 is also connected to the bus 804.

[0092] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0093] In particular, according to embodiments of this disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit (CPU) 801, it performs various functions defined in the system of this disclosure.

[0094] It should be noted that the computer-readable medium shown in the embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] The units described in the embodiments of this disclosure can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0097] In another aspect, this disclosure also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0098] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0099] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0100] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0101] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for detecting abnormal brain discharges, characterized in that: include: Collecting EEG signal data of the detected object and simultaneously collecting a status video of the detected object; Identify the posture information and facial information of the detection object in the status video to obtain key point spatial trajectory data; The EEG signal data and the key point spatial trajectory data are input into a pre-trained abnormality prediction model to obtain an abnormal EEG information prediction result output by the abnormality prediction model as an abnormal brain discharge detection result.

2. The method for detecting abnormal brain discharge according to claim 1, characterized in that: The key point spatial trajectory data includes the posture key point spatial data and the facial key point spatial data of each frame image in the state video; The identifying the posture information and facial information of the detection object in the status video to obtain key point spatial trajectory data includes: Create a human body posture instance and a facial point instance, and load the state video; Identifying the posture key points of the detection object based on the human body posture instance, and acquiring the spatial coordinate data of the posture key points as the posture key point spatial data; and The facial key points of the detection object are identified based on the facial point instances, and the spatial coordinate data of the facial key points are acquired as the facial key point spatial data.

3. The abnormal brain discharge detection method according to claim 2, characterized in that: The method further comprises: Identify the posture key points based on the human posture monitoring model in MediaPipe to obtain the posture key point spatial data; and The facial key points are identified based on the facial point monitoring model in MediaPipe to obtain the facial key point spatial data.

4. The method for detecting abnormal brain discharge according to claim 1, characterized in that: The method further comprises: A moving average filter is applied to smooth the key point spatial trajectory data to obtain first key point spatial trajectory data, so as to update the key point spatial trajectory data.

5. The abnormal brain discharge detection method according to claim 1, characterized in that: The method further comprises: Non-maximum suppression is performed on the key point space trajectory data to obtain second key point space trajectory data to update the key point space trajectory data.

6. The abnormal brain discharge detection method according to claim 1, characterized in that: The method further includes pre-training the abnormality prediction model, wherein the pre-training the abnormality prediction model includes: Acquire training samples; wherein the training samples include EEG signal training data of the detection object, key point spatial trajectory training data and labeling results; the labeling results are used to characterize the abnormal brain discharge detection results of the training samples; Inputting the EEG signal training data and the key point spatial trajectory training data into an initial abnormality prediction model to calculate an abnormality prediction result of EEG information; Calculating a loss function based on the labeling result and the abnormal EEG information prediction result; The model parameters of the initial abnormality prediction model are adjusted based on the loss function to obtain a trained abnormality prediction model.

7. The abnormal brain discharge detection method according to claim 6, characterized in that: The method further comprises: Obtaining a training state video corresponding to the EEG signal training data; The marking result is obtained by marking whether the EEG information is abnormal according to the training status video.

8. A device for detecting abnormal brain discharge, characterized in that: include: An acquisition module is used to acquire EEG signal data of a detection object and simultaneously acquire a status video of the detection object; A recognition module, used for recognizing the posture information and facial information of the detection object in the status video to obtain key point spatial trajectory data; The prediction module is used to detect and input the EEG signal data and the key point spatial trajectory data into a pre-trained abnormal prediction model to obtain the EEG information abnormality prediction result output by the abnormal prediction model as the abnormal brain discharge detection result.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the abnormal brain discharge detection method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the abnormal brain discharge detection method as described in any one of claims 1 to 7.

Images