Brain discharge anomaly detection method and device, storage medium and electronic equipment

Abstract

The invention relates to the field of electroencephalogram detection, in particular to a brain discharge anomaly detection method and device, a storage medium and electronic equipment. The brain discharge anomaly detection method comprises the steps that electroencephalogram signal data and key point space trajectory data of a detection object are collected through target equipment; the key points comprise face key point space trajectory data and body key point space trajectory data; collecting and detecting comprehensive information; the detection comprehensive information comprises basic information of the target equipment, basic information of the detection object and basic information of a detection environment where the detection object is located; and inputting the electroencephalogram signal data, the key point space trajectory data and the detection comprehensive information into a pre-trained anomaly prediction model to obtain an anomaly prediction result output by the anomaly prediction model, and taking the anomaly prediction result as a brain discharge anomaly detection result. According to the brain discharge anomaly detection method provided by the invention, the accuracy and efficiency of a brain discharge anomaly detection result can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of electroencephalogram (EEG) detection, and in particular to a method, device, storage medium and electronic device for detecting abnormal brain discharge. Background Art

[0002] EEG data has important data value in the field of clinical medicine. In recent years, as the quality of life of people in China has improved and medical standards have improved, the number of EEG monitoring has also increased.

[0003] Existing EEG data analysis is mostly done manually, which is arduous and time-consuming. In addition, experts' judgments on EEG epileptic discharges are inconsistent and have low accuracy.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a method, device, storage medium and electronic device for detecting abnormal brain discharge, aiming to improve the accuracy and efficiency of abnormal brain discharge detection results.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of an embodiment of the present disclosure, a method for detecting abnormal brain discharge is provided, comprising: collecting EEG signal data and key point spatial trajectory data of a detection object through a target device; the key points include facial key point spatial trajectory data and body key point spatial trajectory data; and collecting comprehensive detection information; the comprehensive detection information includes basic information of the target device, basic information of the detection object and basic information of the detection environment where the detection object is located; the EEG signal data, the key point spatial trajectory data and the comprehensive detection information are input into a pre-trained abnormal prediction model to obtain an abnormal prediction result output by the abnormal prediction model as a brain discharge abnormal detection result.

[0008] According to some embodiments of the present disclosure, based on the aforementioned scheme, the collecting of EEG signal data and key point spatial trajectory data of the detection object through the target device includes: collecting EEG signal data of the detection object through an electroencephalograph; the collecting of EEG signal data of the detection object through the electroencephalograph includes: fixing the detection electrode of the electroencephalograph at the target position of the detection object; connecting the collection device interface of the electroencephalograph with the detection electrode, so as to collect the EEG signal data through the collection device interface.

[0009] According to some embodiments of the present disclosure, based on the aforementioned scheme, the collecting of EEG signal data and key point spatial trajectory data of the detection object through the target device includes: collecting the facial key point spatial trajectory data of the detection object through the facial video device; the collecting of the facial key point spatial trajectory data of the detection object through the facial video device includes: collecting a facial video of the detection object based on the facial video device; wherein the facial video device is a facial motion capture helmet; identifying facial key points in the facial video, and obtaining the spatial coordinate data of the facial key points that change with time as the facial key point spatial trajectory data.

[0010] According to some embodiments of the present disclosure, based on the aforementioned scheme, the collecting of EEG signal data and key point spatial trajectory data of the detection object through the target device includes: collecting the body key point spatial trajectory data of the detection object through the sensor device; the collecting of the body key point spatial trajectory data of the detection object through the sensor device includes: based on the sensor device pre-set at the body key points of the detection object, obtaining the spatial coordinate data of the body key points of the detection object that changes with time, as the body key point spatial trajectory data.

[0011] According to some embodiments of the present disclosure, based on the aforementioned scheme, the collecting of EEG signal data and key point spatial trajectory data of the detection object through the target device includes: collecting the body key point spatial trajectory data of the detection object through the human video device; the collecting of the body key point spatial trajectory data of the detection object through the human video device includes: collecting the posture video of the detection object based on the human video device; identifying the body key points in the posture video, and obtaining the spatial coordinate data of the body key points that change with time as the body key point spatial trajectory data.

[0012] According to some embodiments of the present disclosure, based on the aforementioned scheme, the method also includes: pre-training the abnormality prediction model; the pre-training of the abnormality prediction model includes: obtaining training samples; wherein the training samples include historical detection data of the detection object and the annotation results corresponding to the historical detection data, and the historical detection data include EEG signal data, key point spatial trajectory data and comprehensive detection information; inputting the detection data into the initial abnormality prediction model to calculate the brain discharge abnormality detection result; calculating the loss function according to the annotation results and the brain discharge abnormality detection result; adjusting the model parameters of the initial abnormality prediction model based on the loss function to obtain a trained abnormality prediction model.

[0013] According to some embodiments of the present disclosure, based on the aforementioned scheme, the method also includes: acquiring historical detection data; generating a status video of the detection object based on key point spatial trajectory data in the historical detection data; and determining whether the brain discharge of the detection object is abnormal as the labeling result based on the status video.

[0014] According to a second aspect of an embodiment of the present disclosure, a device for detecting abnormal brain discharge is provided, comprising: a first acquisition module, used to acquire EEG signal data and key point spatial trajectory data of a detection object through a target device; the key points include facial key point spatial trajectory data and body key point spatial trajectory data; and a second acquisition module, used to acquire comprehensive detection information; the comprehensive detection information includes basic information of the target device, basic information of the detection object and basic information of the detection environment where the detection object is located; a prediction module, used to input the EEG signal data, the key point spatial trajectory data and the comprehensive detection information into a pre-trained abnormal prediction model, and obtain an abnormal prediction result output by the abnormal prediction model as an abnormal brain discharge detection result.

[0015] According to a third aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the abnormal brain discharge detection method as in the above-mentioned embodiment is implemented.

[0016] According to the fourth aspect of the embodiments of the present disclosure, an electronic device is provided, characterized in that it includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the abnormal brain discharge detection method as in the above-mentioned embodiments.

[0017] The exemplary embodiments of the present disclosure may have some or all of the following beneficial effects:

[0018] In the technical solutions provided in some embodiments of the present disclosure, on the one hand, while jointly detecting and analyzing EEG signal data and key point spatial trajectory data, basic information of the target device, the detection object and the detection environment is also taken into account to reduce the interference of artifacts, and to improve the accuracy and efficiency of the EEG information anomaly detection results; on the other hand, a set of automated EEG information anomaly detection methods from information acquisition to information processing is provided, which, compared with manual data analysis, can reduce human bias and improve accuracy while shortening detection time and improving detection efficiency, and makes minor changes to the original technology, making the technology more universal, and can be used in different testing centers. In clinical practice, it can benefit more hospitals and medical staff.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0021] Figure 1 A schematic diagram schematically illustrates a flow chart of a method for detecting abnormal brain discharge in an exemplary embodiment of the present disclosure;

[0022] Figure 2 A schematic diagram schematically illustrates a multi-channel EEG waveform in an exemplary embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram schematically illustrates a facial motion capture helmet in an exemplary embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram schematically illustrates a sensing device in an exemplary embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram schematically shows a hand joint capturing glove in an exemplary embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram schematically illustrating a key point in an exemplary embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram schematically illustrates a training abnormality prediction model in an exemplary embodiment of the present disclosure;

[0028] Figure 8 A schematic diagram schematically illustrates a method of obtaining abnormal brain discharge detection results in an exemplary embodiment of the present disclosure;

[0029] Fig. 9 A schematic diagram schematically illustrates a scenario of abnormal brain discharge detection in an exemplary embodiment of the present disclosure;

[0030] Fig.10 A schematic diagram schematically illustrates another abnormal brain discharge detection scenario in an exemplary embodiment of the present disclosure;

[0031] Fig.11 A schematic diagram schematically shows the composition of a device for detecting abnormal brain discharge in an exemplary embodiment of the present disclosure;

[0032] Fig.12 A schematic diagram schematically illustrates a computer-readable storage medium in an exemplary embodiment of the present disclosure;

[0033] Fig.13 The structure diagram of a computer system of an electronic device in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0035] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0037] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0038] The implementation details of the technical solution of the embodiment of the present disclosure are elaborated in detail below.

[0039] Figure 1 The flowchart of a method for detecting abnormal brain discharge in an exemplary embodiment of the present disclosure is schematically shown. Figure 1 As shown, the abnormal brain discharge detection method includes steps S101 to S103:

[0040] Step S101, collecting EEG signal data and key point spatial trajectory data of the detection object through the target device; the key points include facial key point spatial trajectory data and body key point spatial trajectory data; and

[0041] Step S102, collecting comprehensive detection information; the comprehensive detection information includes basic information of the target device, basic information of the detection object, and basic information of the detection environment where the detection object is located;

[0042] Step S103, inputting the EEG signal data, the key point spatial trajectory data and the comprehensive detection information into a pre-trained abnormality prediction model, and obtaining the abnormality prediction result output by the abnormality prediction model as the brain discharge abnormality detection result.

[0043] In the technical solutions provided in some embodiments of the present disclosure, on the one hand, while jointly detecting and analyzing EEG signal data and key point spatial trajectory data, basic information of the target device, the detection object and the detection environment is also taken into account to reduce the interference of artifacts, and to improve the accuracy and efficiency of the EEG information anomaly detection results; on the other hand, a set of automated EEG information anomaly detection methods from information acquisition to information processing is provided, which, compared with manual data analysis, can reduce human bias and improve accuracy while shortening detection time and improving detection efficiency, and makes minor changes to the original technology, making the technology more universal, and can be used in different testing centers. In clinical practice, it can benefit more hospitals and medical staff.

[0044] Below, each step of the abnormal brain discharge detection method in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0045] In step S101, the EEG signal data and key point spatial trajectory data of the detection object are collected through the target device; the key points include facial key point spatial trajectory data and body key point spatial trajectory data.

[0046] Regarding EEG signal data, in one embodiment of the present disclosure, in step S101, the EEG signal data of the detection object can be collected by an EEG machine; specifically including: fixing the detection electrode of the EEG machine at the target position of the detection object; connecting the collection device interface of the EEG machine to the detection electrode, so as to collect the EEG signal data through the collection device interface.

[0047] Specifically, a multi-channel electroencephalograph can be used to collect EEG signal data. When collecting real-time signal data, the lead information of different centers should be adapted so that the corresponding relationship of the leads uses a unified standard. Of course, the electroencephalograph is only exemplary, and may also include a helmet, a hat, bedding, a packaged portable detection electrode, and other various devices, which are not limited in this disclosure.

[0048] Figure 2 A schematic diagram of a multi-channel EEG waveform in an exemplary embodiment of the present disclosure is schematically shown. The EEG signal is collected by a multi-channel EEG machine, and the collected real-time signal is visualized as shown in FIG. Figure 2 shown.

[0049] Regarding the key point spatial trajectory data, the key points include the facial key points and the body key points of the detection object, so the key point spatial trajectory data includes the facial key point spatial trajectory data and the body key point spatial trajectory data.

[0050] When detecting abnormal brain discharge, it is necessary to collect the spatial trajectory data of the key points on the face of the detection object.

[0051] In one embodiment of the present disclosure, in step S101, the facial key point spatial trajectory data of the detection object can be collected by a facial video device; specifically including: collecting a facial video of the detection object based on the facial video device; wherein the facial video device is a facial motion capture helmet; identifying the facial key points in the facial video, and obtaining the spatial coordinate data of the facial key points changing with time as the facial key point spatial trajectory data.

[0052] Specifically, the facial video of the detection object can be collected through a facial motion capture helmet, and then the facial key points in each frame can be identified based on the facial video, and the positions of the facial key points in each frame can be recorded to obtain the spatial trajectory data of the facial key points.

[0053] When recognizing facial key points based on video, you can use the Mediapipe solution to do the recognition. Use the facial point monitoring model in Mediapipe to process each frame of the image, identify the facial key points, and draw the facial point results to obtain the facial key point spatial data.

[0054] Figure 3 The following is a schematic diagram of a facial motion capture helmet in an exemplary embodiment of the present disclosure. Figure 3 As shown in the figure, a is the main part of the helmet. b is a device with a camera, which can be a mobile phone, a depth camera, etc., connected to the processing module by wire or wireless means. c is a fixed bracket for the camera device, which can keep the camera facing the face regardless of how the head moves.

[0055] When detecting abnormal brain discharges, it is also necessary to collect spatial trajectory data of key points on the body of the detection object.

[0056] In one embodiment of the present disclosure, in step S101, the spatial trajectory data of the key points of the body of the detection object can be collected by a sensor device; specifically, it includes: based on the sensor device pre-set at the key points of the body of the detection object, obtaining the spatial coordinate data of the key points of the body of the detection object that change with time as the spatial trajectory data of the key points of the body.

[0057] The sensing device comprises a hand joint capturing glove, which comprises six sensors respectively located at the ends of five fingers and the center of the palm.

[0058] Specifically, in order to collect the spatial coordinate data of the key points of the body of the detection object, the sensor equipment can be worn on the key parts of the body of the detection object in advance, and then the sensor equipment can be used to record the spatial coordinate data of each key point changing over time, and then transmitted to the processing equipment wirelessly or wiredly for real-time collection and processing of the coordinates of each part, thereby obtaining the spatial trajectory data of the key points of the body.

[0059] Figure 4 A schematic diagram of a sensor device in an exemplary embodiment of the present disclosure is schematically shown. Figure 4 As shown, the sensor devices are arranged at the upper arm, the lower arm, the thigh, and the lower leg. Of course, according to actual needs, sensors can be arranged at multiple other positions, such as the waist, the foot, etc., which are not limited in this disclosure.

[0060] In addition, the key point trajectory of the hand also has a great influence on the detection of abnormal brain discharge, so the hand of the test object can be made to wear hand joint capture gloves. Figure 5 The following is a schematic diagram of a hand joint capturing glove in an exemplary embodiment of the present disclosure. Figure 5 As shown, the sensing device is a hand joint capture glove, which includes six sensors, which are located at the ends of five fingers and the center of the palm. The sensors are transmitted to the processing device by wireless or wired means to collect and process the coordinates of each part in real time.

[0061] In one embodiment of the present disclosure, in step S101, the spatial trajectory data of the key points of the body of the detection object can also be collected by a human video device; specifically including: collecting a posture video of the detection object based on the human video device; identifying the key points of the body in the posture video, and obtaining the spatial coordinate data of the key points of the body that change with time as the spatial trajectory data of the key points of the body.

[0062] Specifically, when obtaining the spatial trajectory data of the key points of the body, the spatial trajectory data of the key points of the body can also be obtained by first capturing a video including the torso, and then identifying the key points of the body based on the video and recording their spatial trajectories.

[0063] Based on video recognition, body key point recognition can be performed using Mediapipe's solution. The human posture monitoring model in Mediapipe is used to process each frame of the image, identify the posture key points, and draw the posture results to obtain the posture key point spatial data.

[0064] In one embodiment of the present disclosure, both facial key points and body key points can be identified using videos. Figure 6 A schematic diagram schematically illustrates a key point in an exemplary embodiment of the present disclosure. Figure 6 As shown, it shows a schematic diagram of the positions of key points, including facial key points 1-10 and body key points 11-32.

[0065] It should be noted that the key point recognition method disclosed in the present invention is only an exemplary description. In addition to using the Mediapipe solution for recognition, yolov5 (a single-stage target detection algorithm), PoseNet, MoveNet, OpenPose, OpenMMD and other methods can also be used for facial or torso posture recognition. The present invention will not go into details here.

[0066] In step S102, comprehensive detection information is collected; the comprehensive detection information includes basic information of the target device, basic information of the detection object, and basic information of the detection environment where the detection object is located.

[0067] Specifically, the comprehensive detection information includes the basic information of the target device. The target device is the device used to collect the EEG signal data and key point spatial trajectory data of the detected object. The results collected by different devices may affect the abnormal brain discharge detection results, so the basic information of the target device is taken into consideration for detection.

[0068] In the present disclosure, the EEG signal data of the detection object can be collected by an EEG machine; the spatial trajectory data of the key points of the face of the detection object can be collected by a face video device; the spatial trajectory data of the key points of the body of the detection object can be collected by a sensor device; and the spatial trajectory data of the key points of the body of the detection object can be collected by a human body video device. Therefore, the target device may be an EEG machine, a video device, and a sensor device.

[0069] Different devices have different basic information, and the collected basic information can be set according to the properties of the device.

[0070] Regarding EEG machines, common parameters include device model, channel configuration, sampling rate, etc. The device model of the EEG machine may vary due to different brands, models and production batches. Channel configuration refers to the number of EEG signals that can be collected simultaneously; the more channels there are, the more comprehensive the EEG information that can be collected, which is more beneficial for the diagnosis of complex cases. The sampling rate refers to the number of samples collected by the EEG machine per second, usually expressed in Hz (Hertz); a high sampling rate can more accurately capture subtle changes in EEG signals, but it will also increase the difficulty of data storage and processing.

[0071] Regarding video equipment, common parameters include video resolution and frame rate. Video resolution refers to the number of pixels in the video picture, usually expressed in the form of horizontal pixels × vertical pixels. Frame rate refers to the number of pictures displayed per second, usually in fps (frames per second).

[0072] Regarding sensor devices, common parameters include sensitivity, accuracy, response time, etc. Sensitivity refers to the relationship between the change in the sensor output signal and the change in the input quantity, or the ratio of the output signal change to the input signal change when the output signal reaches stability; high sensitivity usually means that the sensor has a strong response to small changes. Accuracy refers to the deviation between the sensor output value and the true value of the measured value, which is a comprehensive indicator of the reaction system error and random error; it is usually expressed as a percentage of the full-scale output or an absolute error value. Response time refers to the time it takes for the sensor to detect a change in the input signal and generate a response, usually in milliseconds or seconds.

[0073] The comprehensive test information also includes the basic information of the test object. In practical tests, the test objects may be people of different ages and genders. The different attributes of the test objects will also affect the results of abnormal brain discharge detection. For example, infants and young children need to be held by others during the test, while others do not need to be held, or there are certain differences in the movement of the trunk when testing teenagers and the elderly. Common basic information of the test object includes age, gender and other information.

[0074] The comprehensive test information also includes the basic information of the test environment where the test object is located. In practical tests, the test may be conducted in different test centers, and the environmental information may also affect the results of abnormal brain discharge detection. Therefore, considering this factor, it is also necessary to collect the basic information of the test environment where the test object is located, such as the test address, test site parameters, etc.

[0075] It should be noted that the basic information of the target device, the basic information of the detection object, and the basic information of the detection environment where the detection object is located mentioned in the present disclosure are merely exemplary descriptions and cannot limit the present disclosure.

[0076] In step S103, the EEG signal data, the key point spatial trajectory data and the comprehensive detection information are input into a pre-trained abnormality prediction model to obtain an abnormality prediction result output by the abnormality prediction model as a brain discharge abnormality detection result.

[0077] In one embodiment of the present disclosure, the abnormality prediction model is pre-trained. Therefore, the method further includes: pre-training the abnormality prediction model; specifically including: obtaining training samples; inputting the detection data into the initial abnormality prediction model to calculate the abnormal brain discharge detection result; calculating the loss function according to the annotation result and the abnormal brain discharge detection result; adjusting the model parameters of the initial abnormality prediction model based on the loss function to obtain the trained abnormality prediction model.

[0078] The training samples are collected in advance, including historical detection data of the detection object and the annotation results corresponding to the historical detection data, and the historical detection data includes EEG signal data, key point spatial trajectory data and comprehensive detection information.

[0079] The historical detection data can be obtained by referring to step S101 and step S102, which will not be described in detail here.

[0080] The labeling result is the abnormal brain discharge detection result of the detected object under such historical detection data, including whether the EEG information is normal or abnormal. Specifically, the method further includes: obtaining historical detection data; generating a state video of the detected object based on the key point spatial trajectory data in the historical detection data; and determining whether the brain discharge of the detected object is abnormal based on the state video as the labeling result.

[0081] In other words, the status video obtained based on the historical detection data can be annotated in advance as the real abnormal brain discharge detection result, so as to be used for subsequent model training and parameter correction.

[0082] The initial anomaly prediction model is a basic neural network model, which may be, for example, a convolutional neural network (CNN) or a recurrent neural network (RNN), etc., and the present disclosure does not make any specific limitation.

[0083] Figure 7 A schematic diagram of a training abnormality prediction model in an exemplary embodiment of the present disclosure is schematically shown, which specifically includes the following contents:

[0084] First, the training samples are obtained; then they are input into the neural network model, the prediction results are output, and then the loss is calculated based on the prediction output results and the labeling results. Finally, the model parameters of the neural network model are adjusted according to the loss until the training is completed. The final trained neural network model is obtained as the abnormality prediction model. The input of the abnormality prediction model is the EEG signal data and the key point spatial trajectory data, and the output is the abnormality prediction result of the EEG information.

[0085] Therefore, after pre-training the abnormality prediction model, when the EEG signal data and key point spatial trajectory data are obtained through step S101, and the comprehensive detection information is obtained through step S102, the data is input into the abnormality prediction model, and the abnormality prediction result output by the abnormality prediction model can be obtained as the brain discharge abnormality detection result of the detection object.

[0086] Figure 8 A schematic diagram schematically shows a method of obtaining abnormal brain discharge detection results in an exemplary embodiment of the present disclosure, such as Figure 8 As shown, the complete process of obtaining abnormal brain discharge detection results is as follows:

[0087] First, training samples are obtained, including: ① EEG signals collected by an electroencephalograph; ② facial videos collected by a head camera, and then spatial trajectory data of facial key points obtained based on the facial video, including real-time spatial coordinate trajectories of facial key points such as eyes, mouth, nose, ears, and chin; ③ spatial trajectory data of body key points collected by body sensors, including real-time spatial coordinate trajectories of body key points such as finger joints, upper arms, forearms, thighs, calves, and feet; ④ comprehensive detection information, including basic information of the target device, basic information of the detection object, and basic information of the detection environment where the detection object is located.

[0088] Then, the above data is input into the abnormal prediction model in segments of n seconds to obtain the abnormal prediction result output by the abnormal prediction model as the abnormal brain discharge detection result of the detection object.

[0089] Therefore, based on the above method, the present disclosure combines EEG, ECG, EMG and rich video features, as well as comprehensive detection information of equipment, detection objects and detection environment, thereby improving the accuracy and reliability of abnormal EEG discharge detection, while also avoiding the error of manual identification. In addition, the present disclosure makes minor changes to the original technology, making the technology more universal and can be used in different detection centers. In clinical practice, it can benefit more hospitals and medical staff.

[0090] Fig. 9 The following is a schematic diagram schematically showing a brain discharge abnormality detection scenario in an exemplary embodiment of the present disclosure. Fig. 9 As shown, the multi-channel EEG signals of the human body 101 can be collected by the EEG signal collection device 107. During the collection process, the detection electrodes 103a, 103b, and 103c can be fixed at various positions on the head of the human body 101. After the interface 105 of the EEG signal collection device 107 is connected to the detection electrodes 103a, 103b, and 103c, the EEG signal collection device 107 can collect the multi-channel EEG signals of the human body 101. Of course, the EEG signal collection device 107 is only exemplary, and can also include a helmet, a hat, bedding, a packaged portable detection electrode, and other various devices, which are not limited in this application.

[0091] At the same time, the camera 116 is used to synchronously capture the patient's facial video, and the sensors 117a, 117b, and 117c on the whole body and inside the gloves record the user's body movement status. The EEG signal acquisition device 107 can also be coupled with the abnormal discharge detection device 109. Fig. 9 This is only an exemplary coupling method. The EEG signal acquisition device 107 and the abnormal discharge detection device 109 can also be integrated into the same device. The device can include, for example, a camera, a helmet, a hat, bedding, a packaged portable detection electrode, etc. This application does not limit this. The abnormal discharge detection device 109 can use the abnormal brain discharge detection method provided in any embodiment of this application to perform multi-modal processing on the multi-channel EEG signals acquired by the EEG acquisition device 107 and the user video acquired by the camera 116.

[0092] After obtaining the abnormal discharge detection result, Fig. 9 As shown, the abnormal discharge detection result can be displayed on the display 111, which can facilitate relevant personnel to understand the specific detection result.

[0093] Fig.10 A schematic diagram schematically illustrates another abnormal brain discharge detection scenario in an exemplary embodiment of the present disclosure. Fig.10 As shown, the collected EEG signals and videos can also be uploaded to the server for processing. This scenario is more suitable for portable scenarios. Users can use portable EEG signal collection equipment 107 and camera 116 to collect multi-channel EEG signals and videos at home, office, and other places.

[0094] Fig.10 The portable EEG signal acquisition device 107 is only exemplary, and may also include a camera, a helmet, a hat, bedding, a packaged portable detection electrode, and other devices. Fig.10As shown, the EEG signal acquisition device 107 is coupled with the data transceiver 109. After the EEG signal acquisition device 107 acquires the multi-channel EEG signal of the human body 101, the data transceiver 109 can send the multi-channel EEG signal to the server 115. The server 115 can include any form of data processing server such as a cloud server and a distributed server. After receiving the multi-channel EEG signal, the server 115 can process the multi-channel EEG signal using the abnormal brain discharge detection method provided in each embodiment of the present application, and obtain the abnormal discharge result of the human body 101. Then, the server 115 can send the abnormal discharge result to the data transceiver 107. After receiving the abnormal discharge result, the data transceiver 107 can display the abnormal discharge result on the display 111, such as a client display such as a user's mobile phone or computer.

[0095] Through the above scenarios, users can learn about the abnormal brain discharge detection results without leaving home. Therefore, when encountering suspected epilepsy and other problems based on the test results, the obtained abnormal brain discharge detection results can be sent to the doctor to facilitate the doctor to determine the diagnosis results.

[0096] Fig.11 The following is a schematic diagram showing the composition of a device for detecting abnormal brain discharge in an exemplary embodiment of the present disclosure. Fig.11 As shown, the abnormal brain discharge detection device 1100 may include a first acquisition module 1101, a second acquisition module 1102 and a prediction module 1103. Among them:

[0097] A first acquisition module 1101 is used to acquire EEG signal data and key point spatial trajectory data of a detection object through a target device; the key points include facial key point spatial trajectory data and body key point spatial trajectory data; and

[0098] The second acquisition module 1102 is used to collect comprehensive detection information; the comprehensive detection information includes basic information of the target device, basic information of the detection object, and basic information of the detection environment where the detection object is located;

[0099] The prediction module 1103 is used to input the EEG signal data, the key point spatial trajectory data and the comprehensive detection information into a pre-trained abnormality prediction model to obtain the abnormality prediction result output by the abnormality prediction model as the brain discharge abnormality detection result.

[0100] According to an exemplary embodiment of the present disclosure, the first acquisition module 1101 is used to acquire EEG signal data of a detection object through an electroencephalograph, including: fixing the detection electrode of the electroencephalograph at the target position of the detection object; connecting the acquisition device interface of the electroencephalograph to the detection electrode to acquire the EEG signal data through the acquisition device interface.

[0101] According to an exemplary embodiment of the present disclosure, the first acquisition module 1101 is used to acquire spatial trajectory data of facial key points of a detection object through a facial video device, including: acquiring a facial video of the detection object based on the facial video device; wherein the facial video device is a facial motion capture helmet; identifying facial key points in the facial video, and acquiring spatial coordinate data of the facial key points that change with time as the facial key points spatial trajectory data.

[0102] According to an exemplary embodiment of the present disclosure, the first acquisition module 1101 is used to collect the spatial trajectory data of the body key points of the detection object through the sensor device, including: based on the sensor device pre-set at the body key points of the detection object, obtaining the spatial coordinate data of the body key points of the detection object that change with time as the spatial trajectory data of the body key points.

[0103] According to an exemplary embodiment of the present disclosure, the first acquisition module 1101 is used to acquire spatial trajectory data of body key points of the detection object through a human body video device, including: acquiring a posture video of the detection object based on the human body video device; identifying body key points in the posture video, and obtaining spatial coordinate data of the body key points that change with time as the spatial trajectory data of the body key points.

[0104] According to an exemplary embodiment of the present disclosure, the brain discharge abnormality detection device 1100 also includes a training module for pre-training the abnormality prediction model, including: obtaining training samples; wherein the training samples include historical detection data of the detection object and the annotation results corresponding to the historical detection data, and the historical detection data include EEG signal data, key point spatial trajectory data and comprehensive detection information; inputting the detection data into the initial abnormality prediction model to calculate the brain discharge abnormality detection result; calculating the loss function according to the annotation results and the brain discharge abnormality detection result; adjusting the model parameters of the initial abnormality prediction model based on the loss function to obtain a trained abnormality prediction model.

[0105] According to an exemplary embodiment of the present disclosure, the training module is also used to acquire historical detection data; generate a status video of the detection object based on key point spatial trajectory data in the historical detection data; and determine whether the brain discharge of the detection object is abnormal as the labeling result based on the status video.

[0106] The specific details of each module in the above-mentioned abnormal brain discharge detection device 1100 have been described in detail in the corresponding abnormal brain discharge detection method, so they will not be repeated here.

[0107] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0108] In an exemplary embodiment of the present disclosure, a storage medium capable of implementing the above method is also provided. Fig.12 A schematic diagram schematically illustrates a computer-readable storage medium in an exemplary embodiment of the present disclosure, such as Fig.12 As shown, a program product 1200 for implementing the above method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a mobile phone. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0109] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. Fig.13 The structure diagram of a computer system of an electronic device in an exemplary embodiment of the present disclosure is schematically shown.

[0110] It should be noted that Fig.13 The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0111] like Fig.13As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1302 or the program loaded from the storage part 1308 to the random access memory (RAM) 1303. Various programs and data required for system operation are also stored in the RAM 1303. The CPU 1301, ROM 1302 and RAM 1303 are connected to each other through a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0112] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0113] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1309, and / or installed from a removable medium 1311. When the computer program is executed by a central processing unit (CPU) 1301, various functions defined in the system of the present disclosure are executed.

[0114] It should be noted that the computer-readable medium shown in the embodiment of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0115] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0116] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0117] As another aspect, the present disclosure further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0118] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0119] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0120] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0121] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for detecting abnormal brain discharge, characterized in that: include: Collecting brain wave signal data and key point spatial trajectory data of the detection object through the target device; the key points include facial key point spatial trajectory data and body key point spatial trajectory data; as well as Collect comprehensive detection information; the comprehensive detection information includes basic information of the target device, basic information of the detection object, and basic information of the detection environment where the detection object is located; The EEG signal data, the key point spatial trajectory data and the comprehensive detection information are input into a pre-trained abnormality prediction model to obtain an abnormality prediction result output by the abnormality prediction model as a brain discharge abnormality detection result.

2. The method for detecting abnormal brain discharge according to claim 1, wherein: The collecting of the EEG signal data and key point spatial trajectory data of the detection object by the target device includes: collecting the EEG signal data of the detection object by an electroencephalograph; The method of collecting brain electrical signal data of the test subject by using an electroencephalograph includes: Fixing the detection electrode of the electroencephalograph at the target position of the detection object; The acquisition device interface of the electroencephalograph is connected to the detection electrode to acquire the brain electrical signal data through the acquisition device interface.

3. The method for detecting abnormal brain discharge according to claim 1, wherein: The collecting of the EEG signal data and key point spatial trajectory data of the detection object through the target device includes: collecting the key point spatial trajectory data of the face of the detection object through the facial video device; The collecting of the spatial trajectory data of the facial key points of the detection object by the facial video device includes: Capturing a facial video of a detection object based on the facial video device; wherein the facial video device is a facial motion capture helmet; Identify facial key points in the facial video, and obtain spatial coordinate data of the facial key points that change with time as the facial key points spatial trajectory data.

4. The method for detecting abnormal brain discharge according to claim 1, wherein: The collecting of the EEG signal data and key point spatial trajectory data of the detection object by the target device includes: collecting the key point spatial trajectory data of the detection object's body by the sensor device; The collecting of the spatial trajectory data of key points of the body of the detection object by the sensor device includes: Based on the sensing devices pre-set at the key points of the body of the detection object, the spatial coordinate data of the key points of the body of the detection object that changes with time is obtained as the spatial trajectory data of the key points of the body.

5. The method for detecting abnormal brain discharge according to claim 1, wherein: The collecting of the EEG signal data and key point spatial trajectory data of the detection object by the target device includes: collecting the key point spatial trajectory data of the body of the detection object by the human body video device; The method of collecting the key point spatial trajectory data of the body of the detection object by using the human body video equipment includes: Collecting a posture video of the detection object based on the human body video device; Identify key body points in the posture video, and obtain spatial coordinate data of the key body points that change with time as the spatial trajectory data of the key body points.

6. The method for detecting abnormal brain discharge according to claim 1, wherein: The method further includes: pre-training the abnormality prediction model; the pre-training of the abnormality prediction model includes: Obtaining training samples; wherein the training samples include historical detection data of the detection object and the annotation results corresponding to the historical detection data, and the historical detection data includes EEG signal data, key point spatial trajectory data and detection comprehensive information; inputting the detection data into an initial abnormality prediction model to calculate a brain discharge abnormality detection result; Calculating a loss function based on the labeling result and the abnormal brain discharge detection result; The model parameters of the initial anomaly prediction model are adjusted based on the loss function to obtain a trained anomaly prediction model.

7. The method for detecting abnormal brain discharge according to claim 6, wherein: The method further comprises: Obtain historical test data; Generate a status video of the detection object based on key point spatial trajectory data in the historical detection data; Determining whether the brain discharge of the detection subject is abnormal according to the status video is used as the labeling result.

8. A device for detecting abnormal brain discharge, characterized in that: include: A first acquisition module is used to acquire EEG signal data and key point spatial trajectory data of the detection object through a target device; the key points include facial key point spatial trajectory data and body key point spatial trajectory data; as well as The second acquisition module is used to collect comprehensive detection information; The comprehensive detection information includes basic information of the target device, basic information of the detection object, and basic information of the detection environment where the detection object is located; The prediction module is used to input the EEG signal data, the key point spatial trajectory data and the detection comprehensive information into a pre-trained abnormality prediction model, and obtain the abnormality prediction result output by the abnormality prediction model as the brain discharge abnormality detection result.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for detecting abnormal brain discharge according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the abnormal brain discharge detection method according to any one of claims 1 to 7.

Images