Method and device for detection of sleep apnea

Abstract

The invention relates to the training of an artificial neural network model to be used for the detection of sleep apnea, a prediction method, and a system to perform said prediction.

Description

[0001] METHOD AND DEVICE FOR DETECTION OF SLEEP APNEA

[0002] Technical Field

[0003] The invention relates to the training of an artificial neural network model to be used for the detection of sleep apnea, a prediction method, and a system to perform said prediction.

[0004] State of the Art

[0005] The term sleep apnea refers to a sleep disorder characterized by short-term or complete shortness of breath and interruption of airflow for at least 10 seconds during sleep. Due to sleep apnea, the level of oxygen saturation in the blood decreases and long-term airflow interruption results in death. For these reasons, devices that provide monitoring and detection are needed for the detection of sleep apnea.

[0006] Patent document TR 2021 / 003776 describes the sleep apnea warning system. Said invention is defined as a smart health monitoring system that allows people to wake up with an alertwarning system when they are out of breath and collects personalized and real-time data on the number of sleep apneas per day with healthcare technology.

[0007] TR 2017 / 06960 discloses a system for diagnosing and classifying sleep apnea using brain activity Electroencephalography (EEG) signals.

[0008] Patent document US20090062628A1 discloses a sleep apnea diagnostic device. The device mentioned here includes a three-dimensional gyroscope and a posture detector for chest movement and produces results by considering the data received from these two sensors.

[0009] Patent document US9186511B2 also discloses a sleep apnea treatment system. In this invention, a temperature sensor and gyroscope are used for the detection of sleep apnea.

[0010] The accuracy rates of these methods and devices are low or unknown. In addition, these devices can only be used in a hospital environment and the results obtained from these devices are not directly interpreted by the user and therefore a professional healthcare professional is needed.

[0011] All the problems mentioned above have made it necessary to make an innovation in the relevant field as a result.

[0012] Objects of the Invention

[0013] The main object of the invention is to provide a sleep apnea detection method and device that can provide high accuracy results.

[0014] The object of the invention is to enable the user to self-detect sleep apnea without the need for professional interpretation.

[0015] The object of the invention is to provide a patient-specific sleep apnea detection method.

[0016] Definitions of Figures Describing the Invention

[0017] The figures and related descriptions used to better explain the device developed by this invention are as follows.

[0018] Figure 1. An isometric image of the device of the invention.

[0019] Figure 2. A graph showing the comparison of temperature and acceleration curve data.

[0020] Figure 3a. A graph showing the acceleration of the time-dependent diaphragm on the x-axis for the detection of breathing profiles.

[0021] Figure 3b. A graph showing the acceleration of the time-dependent diaphragm on the y-axis for the detection of breathing profiles.

[0022] Figure 3c. A graph showing the acceleration of the time-dependent diaphragm on the z-axis for the detection of breathing profiles.

[0023] Figure 4. A representative image of the breathing data measured using an accelerometer to determine the delimiter boxes to be used in artificial neural network model training.

[0024] Figure 5a. A graph showing the precision performance of the artificial neural network model trained for the detection of sleep apnea. Figure 5b. A graph showing the recall performance of an artificial neural network model trained for the detection of sleep apnea.

[0025] Figure 5c. A graph showing the mAP@0.5 performance of the artificial neural network model trained for the detection of sleep apnea.

[0026] Figure 5d. A graph showing the object loss value performance of the artificial neural network model trained for the detection of sleep apnea.

[0027] Figure 6. The confusion matrix of the model.

[0028] Figure 7. A graph showing the detection of the time-dependent gravitational acceleration on the axis perpendicular to the diaphragm and the breathing epoch.

[0029] Figure 8. A graph showing the breathing profile predictions obtained as a result of the image processing algorithm with a prediction threshold of 0.6.

[0030] Definitions of Components / Pieces / Parts of the Invention

[0031] In order to better explain the device developed by this invention, the parts and pieces in the figures are numbered and corresponding numbers are given below.

[0032] 1. Accelerometer

[0033] 2. Fixing element

[0034] 3. Temperature sensor

[0035] 4. Control element

[0036] 5. Body

[0037] 6. Cable

[0038] P. Data processing unit

[0039] Detailed Description of the Invention

[0040] The subject matter of the invention relates to the training of an artificial neural network model to be used for the detection of sleep apnea, a prediction method, and a system to perform said prediction.

[0041] Referring to Figure 1, the present system comprises at least one sensor pair and a data processing unit (P) that can process the data received from said sensors. At least one of said sensor pairs is the accelerometer (1) and the other is the temperature sensor (3). Preferably, a triaxial accelerometer (1) was used in said system. Alternatively, it is also possible to use multiple accelerometers (1) to detect movement in different axes or regions.

[0042] The existing accelerometer (1) is arranged to be placed in the abdominal region of the user and to stay still there. As can be seen in Figure 1, said accelerometer (1) is connected to a fixing element (2). Preferably, the fixing element (2) has a truncated conical end and a cylindrical body and said accelerometer (1) is placed on the cylindrical body base. Alternatively, the fixing element (2) may also be an adhesive surface that can stick to the abdominal region or a belt-lock system that can be attached to the body.

[0043] The breathing epoch can be measured with an accelerometer (1) due to the movement of the diaphragm on an axis perpendicular to the abdomen during breathing. The maximum and minimum peak acceleration values of the diaphragm occur during the breathing and exhalation states. Therefore, the number of these peaks is used to analyze the respiratory frequency.

[0044] The temperature sensor (3) is configured to detect and / or measure temperature changes.

[0045] In the preferred embodiment, the temperature sensor (3) is configured to detect and / or measure both the temperature of the user's nasal base and the ambient temperature.

[0046] Although a temperature sensor (3) is preferably used, alternatively more than one individual temperature sensor (3) can be used to detect the ambient and nasal base temperature. The acceleration applied by the diaphragm to the abdomen during inspiration and exhalation is not only obtained as a result of one-way abdominal movement. Therefore, it is important to obtain data on all axes. As a result, acceleration measurements on the vertical axis are accepted as the main data in breath analysis since the diaphragm accelerates more steeply to the thorax. However, changes in body position are movements in other axes independent of the respiratory epoch, such as flexions, contractions and unexpected body movements, and these movements can also be detected by the device through an analysis. It is used to determine the breathing profiles by analyzing the result of the analysis of all the data obtained. In the application, a high-resolution (13-bit) triaxial accelerometer was used. The concave and convexity of the diaphragm during acceleration are related to the frequency of breathing. The time to reach the peak shows the depth of inhalation. In addition, shortness of breath, which is the biggest symptom of sleep apnea, can be determined in a certain period of time by the fact that it is seen as steady in the accelerometer (1) and does not reach the peaks or reach the minimum peak values.

[0047] The temperature sensor (3) must be fixed in a position close to the user's nasal base. Here, the temperature sensor (3) is preferably designed to be positioned between the upper lip of the user and the nostrils. Here, elements such as strap, thread, adhesive surface that will fix the temperature sensor (3) to the relevant position can be used. The temperature sensor (3) determines the hot and cold air currents during exhalation and inhalation and the relationship between upper respiratory and diaphragm movements can be matched. Preferably, an infrared temperature sensor (3) is used.

[0048] Preferably, an I2C output temperature sensor (3) with an accuracy of 0.5°C, a resolution of 0.02°C, and dimensions of 18 mm x 11 mm, capable of measuring in the temperature range of -20 - 150 °C, is used to measure breath temperature.

[0049] It was observed that the airflow caused by exhalation was much higher than the ambient temperature and the temperature values of the airflow caused by inhalation were measured lower than the ambient temperature. Exhaled breath temperature data provides critical data for snoring, mouth breathing, or shortness of breath, as well as control of the data of breathing profiles. With the graph of the change in temperature over time, data such as breathing length, depth, and frequency of breathing can be obtained. The measured exhaled breath temperature data proceeds in harmony with the diaphragm movements. The comparison of the exhaled breath temperature data with the breathing data and the coherence between them can be seen in the graph in Figure 2.

[0050] The accelerometer (1) and the temperature sensor (3) are connected to a control unit (4), preferably by means of cables (6). Preferably, the control unit is a processing unit, in particular a microprocessor. The Arduino microprocessor is used in one embodiment of the invention. Preferably, the control unit (4) is positioned in a case (5). The case (5) protects the control unit (4) and its components. The accelerometer (1) and the temperature sensor (3) transmit the data they collect to the control unit (4). Preferably, integrated circuits communication protocols (I2C) were used for data transfer. The unique I2C communication address of each sensor, which determines the sensor behavior and adjusts the measurements, is used.

[0051] The accelerometer (1) transmits the data it collects to a data processing unit (P) configured to execute the previously trained artificial neural network model, preferably based on deep learning. The data processing unit (P) can be provided on an external device outside the system, or the accelerometer (1) can be provided in an integrated way. Here, if the data processing unit (P) is located in an external device, a communication module (preferably wireless (Bluetooth, GPRS, Wi-Fi), alternatively wired) may be used to transmit the temperature and acceleration data.

[0052] Here, a data processing unit (P) can also be arranged to compare the data provided from the temperature sensor with the artificial neural network model output. The data processing unit (P) mentioned here may be the processing unit that executes the artificial neural network model, or a data processing unit (P) may also be used for this comparison process.

[0053] The accelerometer (1) and the temperature sensor (3) can send the data they collect to the control unit (4) or can be transmitted directly to a data processing unit (P). In this case, the communication module can also be used as described above. In addition, the data sent to the control unit (4) is transmitted to the data processing unit (P) with a communication module provided by the control unit (4). Furthermore, alternatively, the data processing unit (P) is provided as integrated with the accelerometer (1) and the temperature sensor (3), and the data is transmitted directly to the data processing unit (P).

[0054] In one embodiment, the results of the data processing unit (P) may be returned to the user or transmitted to the healthcare professional. Here, the results of the data processing unit (P) may be transmitted to the smartphone, tablet, computer of the user and / or healthcare professional or an alarm may be created for a negative result. In addition, the results of the data processing unit (P) can be sent back to the device containing the accelerometer (1) and the temperature sensor (3). Accordingly, the device may include a notification element and this notification element may be one of the obvious options such as a display or alert light. As previously described, the data processing unit (P) is configured to execute the pre-trained artificial neural network model, preferably based on deep learning.

[0055] It is necessary to prepare the appropriate dataset first for the training of said artificial neural network model. The above-described device or embodiments of the device may be used, or at least one accelerometer (1) placed in the abdominal region of the subjects who will only constitute the dataset may be used to prepare this dataset.

[0056] Data received from multiple users are collected continuously for a certain period of time. Here, it should not be understood from the "continuously" expression that the sensors also receive the data continuously. The sensors can receive the data at certain intervals. The "continuously" expression indicates that the time that constitutes the data is uninterrupted. The sensors may be configured to collect data, for example, every 100 milliseconds.

[0057] Here, users are asked to breathe for a certain period of time and hold their breath for a certain period of time. It is mandatory to have at least one breathing sequence and one breathing holding sequence. Multiple breathing sequences and breath holding sequences can also be used to prepare the dataset. Preferably, these data were obtained when subjects were in a horizontal position, i.e. in a supine position. In addition, or alternatively, data can be collected from users in sitting and standing positions. However, it has been observed that the supine position yields more accurate results.

[0058] In a preferred embodiment of the invention, the dataset can also be created by the artificial neural network model with the data obtained from the patient to whom sleep apnea tests will be carried out. The patient's breathing profiles can differ from the standard training model by parameters such as breathing frequency, stenosis, speed, exhalation force, and duration of breathlessness depending on the disease. The patient's breathing profiles can be recorded in the system by adding the data obtained from the patient as training data to the existing training model. In this way, the accuracy of the detection of breathing sequences and sleep apnea can be increased by using personalized artificial neural network training models.

[0059] The correct classification of the data obtained here is an important criterion for the success of the invention. For this reason, the numerical data collected from the users are visualized. A time -based graph is preferably created here. In a preferred embodiment, the obtained acceleration measurements are first smoothed by using spline curves.

[0060] Each of the breathing and breath holding sequences in the obtained data is marked and classified. Here, the positions of the breathing and breath holding sequences on the graph are preferably marked. A representative representation of the relevant breathing sequences is given on the graph in Figures 3a-3c.

[0061] Then, said data is divided into two as training and validation data and an artificial neural network model is trained in a predetermined number by using said training and validation data.

[0062] Preferably, said artificial neural network model is a model for object detection. The YOLO model was used in the application of the invention. However, models for the detection of different objects can also be used for said process. For example, CNNs, ResNet, VGGNet, MobileNet algorithms can also be selected as an artificial neural network model.

[0063] The present training method, in its most basic form, includes the steps of obtaining a dataset from multiple users containing at least one breathing sequence in which each user holds their breath for at least a predetermined period of time and inhales and exhales for a predetermined period of time and has diaphragm movement data taken from the accelerometer placed in the user abdomen, visualizing said dataset and classifying the inhalation, exhalation and breath holding sections on said image, dividing it into two as training and validation data according to a predetermined number with the obtained data and training a predetermined number of artificial neural network models by using said training and validation data.

[0064] Said classification is provided by rectangular delimiter boxes provided on the image in a preferred embodiment of the invention. As can be seen in Figure 4, the areas covering each inhalation, exhalation and breath holding sequence are individually surrounded by rectangular delimiter boxes and each delimiter box is marked as inhalation (inhaled), exhalation (exhaled) and breath holding (breathlessness) according to the sequence it covers. In the training steps, the image is divided into a grid of redundant blocks, as delimiter boxes are used, and each block contains probability values for the position and class of an object. Delimiter boxes consisting of combinations and intersections of these blocks are used to detect objects in the image. The grid box with the highest similarity was obtained by accepting the intersection of all the predicted boxes with an intersection on the conjunction. In this way, only the boxes with the highest detection probability are used for detection.

[0065] The trained artificial neural network model is used in the detection of sleep apnea with the data obtained from the user.

[0066] For this, the data from the patient to whom sleep apnea detection / test is carried out is fed as input to the artificial intelligence model trained as described, and accordingly, whether the patient has sleep apnea is determined, preferably in real time.

[0067] The system that the user themselves can use shown in Figure 1 is used to obtain data from the patient. This system basically includes an accelerometer (1) to be placed in the abdominal region to detect the diaphragm movements of the user, a data processing unit (P) that receives the data provided by the accelerometer (1) as input and will execute the trained artificial neural network model with said input. The data processing unit (P) may be provided externally or integrated with the accelerometer (1) as previously described.

[0068] In a preferred embodiment of the invention, there is also a temperature sensor (3). The temperature sensor is used to check / verify the prediction provided from the artificial neural network. Preferably, the temperature sensor (3) is arranged to detect the temperature data at the nasal base. As can be seen in Figure 2, the temperature at the nasal base and breathing are in a certain correlation. The increase in temperature indicates exhalation. Preferably, the processing unit (P) that executes the artificial neural network model or another processing unit (P) is configured to compare the artificial neural network model prediction with the data received from the temperature sensor, and optionally to create an alert in case of a discrepancy (discrepancy in inhalation, exhalation, or breathlessness) in this encounter and to create a response to this discrepancy.

[0069] The alert may be provided by a device that responds visually, audibly, or tactilely. The training method of the artificial neural network for predicting sleep apnea with a computer application can be provided as a program. The program comprises instructions for performing the steps of claim 10.

[0070] In addition, it can be used for the training of the artificial neural network for sleep apnea prediction in the computer-readable medium containing the instructions to perform the method steps according to claim 10.

[0071] In addition, a data processing unit (P) comprising suitable elements for performing the method steps according to claim 10 is also in compliance with the scope of the invention. Here, the data processing unit (P) comprises at least an electronic processor in the most general sense to process the data and inputs to receive the data to be provided from the accelerometer (1) and the temperature sensor (3).

[0072] The artificial neural network trained according to the processes given above is used in a computer applied prediction method for yield prediction.

[0073] Various tests were performed for the results of the relevant system. Here, patient data for sleep apnea detection were obtained and processed with the system given in claim 12.

[0074] For the aforementioned tests, 150 breathing measurements were taken from the accelerometer (1) and temperature sensors (3) in the supine position from 3 healthy subjects between the ages of 18-24 to be used in the training of the deep learning model of the invention. While the data of 3 healthy subjects were used to train the system, the system was tested with the data of 10 other healthy subjects. In the measurements, breathing and breathlessness (breath holding) were simulated in a controlled manner. During the measurements, the subject breathed through the nose 5 times in ~30 seconds, and then they were asked to hold their breath for ~10 seconds. This procedure was continued in one epoch during the measurement period.

[0075] As mentioned earlier, this data can be taken from the patient who will use the system and a user-specific training can be carried out.

[0076] The data collected from these measurements were used for the development of object detection algorithms as training, verification, and test datasets. The accelerometer (1) in the device was fixed to the diaphragm with medical tapes and was used to take acceleration values in 3 axes: parallel to the neck, parallel to the arms and perpendicular to the ribcage. The received acceleration data was transferred to the host computer in real time through the device and stored on this computer. These acceleration data were smoothed with spline curves to eliminate noise.

[0077] The obtained data were converted into time-dependent images (graphics).

[0078] Then, the dataset training and verification dataset was randomly divided into 120 and 30 images, respectively. For classification, the previously described delimiter boxes method was used, and the data were exported in a trainable .yaml file format. The classification success using the model trained on the breathing epoch in Figure 7 is shown in Figure 8. As shown in Figure 8, the delimiter boxes were able to successfully determine inhalation, exhalation and breath holding times with a prediction threshold of 0.6.

[0079] Here, YOLO architecture was used for the dataset consisting of breathing training and test images. The training process was carried out using an external CPU and running with the Python programming language. The whole process was operated with an external mainframe. When starting the training of the model, necessary parameters such as "image size", "batch size", "epoch", "YOLO model" were defined. These parameters were determined as 416 for image size, 16 for batch size, 230 for epoch and medium for YOLO model. The model was trained with 230 epochs to determine the training that worked and was compatible with maximum efficiency. In the training steps, the image is divided into a grid of redundant blocks, and each block contains probability values for the position and class of an object. Delimiter boxes consisting of combinations and intersections of these blocks are used to detect objects in the image. The grid box with the highest similarity was obtained by accepting the intersection of all the predicted boxes with an intersection on the conjunction. In this way, only the boxes with the highest detection probability are used for detection.

[0080] When the training process was completed, the performance of the best training output was evaluated and metrics determining the deep learning training result were obtained. The prediction results and confusion matrix of the training dataset were examined in order to determine the success of the developed algorithm. For this purpose, the progression of precision, sharpness, mAP@0.5 and object loss values for each epoch step were analyzed. As can be seen in Figure 5a-5c, the training weights corresponding to the 160th epoch step with the highest mAP@0.5 value and maximum adaptation were obtained with the .pt extension file. In the dataset trained in the 160th epoch step, precision, sharpness and mAP@0.5 values reached 0.9 and object loss was reduced to 0.1. This shows that the trained model is perfectly suited to detect breathing, exhalation, and breathlessness times from the diaphragm acceleration. In addition, the confusion matrix was calculated in the performance analysis of the trained model by comparing the predicted and actual conditions.

[0081] The corresponding matrix obtained is shown in Figure 6. The confusion matrix gives false positive (FP) and false negative (FN) results for each object detection class. As a result of the confusion matrix, all classes achieved prediction performance of up to 0.96 and have minimal FPs and FNs in the trained model. The fact that the FN value is less than 0.04 indicates that the model we detected gives very low false positive and negative values and the probability of misdiagnosis is minimized.

Claims

CLAIMS1. A computer applied artificial neural network training method for the detection of sleep apnea, characterized by:Receiving a dataset containing at least one sequence of at least one user holding their breath for at least a predetermined period of time and at least one sequence of at least one user inhaling and exhaling for a predetermined period, with data from an accelerometer (1) placed on the user's abdomen,Visualizing said dataset and classifying inhalation, exhalation and breath holding sections on said image,Dividing the obtained data into two as training and validation data according to a predetermined number and training an artificial neural network model in a predetermined number of epochs by using said training and validation data.

2. A method according to claim 1, characterized in that said user is the user for whom sleep apnea is to be detected.

3. A method according to claim 1, characterized in that said dataset is trained with data received from multiple users.

4. A method according to claim 1, characterized in that said dataset image is a time-sensor data graph.

5. A method according to claim 1 or 4, characterized in that each inhalation, exhalation and breath holding sequence in said dataset is classified according to its position on the image.

6. A method according to any one of preceding claims, characterized in that said classification is provided by forming rectangular delimiter boxes covering each inhalation, exhalation and breath holding sequence.

7. A method according to claim 6, characterized in that in the training steps, the rectangular delimiter boxes provided in the image and each box comprises probability values for the position and class of an object, and the delimiter boxes consisting of the boxes and their intersections are used to detect the objects in the image.

8. A method according to claim 7, characterized in that the delimiter box with the highest similarity in the training steps is obtained by accepting the intersection of all the predicted boxes with an intersection on the conjunction.

9. A method according to claim 1, characterized in that said artificial neural network model is YOLO.

10. A method according to claim 1, characterized in that said accelerometer data is smoothed by spline curves.

11. A method according to claim 1, characterized in that it can be used for breathing analysis by determining inhalation, exhalation, and breathlessness situations.

12. A computer applied prediction method for the detection of sleep apnea, characterized by: feeding diaphragm movement data from the accelerometer placed in the user abdomen is as input to an artificial neural network model trained by said method according to any one of claims 1-11.

13. A prediction method according to claim 11, characterized in that the temperature data at the user nasal base is detected and compared with the temperature data at the output of the artificial neural network model.

14. A data processing unit (P) comprising means for performing the method according to claim 12 or 13.

15. A program for performing a method according to claim 12 or 13 when executed by a data processing unit (P) .

16. A test system for the detection of sleep apnea, characterized in that it comprises the following:An accelerometer (1) to be placed in the abdominal region to detect the diaphragm movements of the user, anda data processing unit (P) for performing said method according to claim 12.

17. A test system according to claim 16, characterized in that it comprises a temperature sensor (3) to detect the temperature data at the nasal base of the user.

18. A test system according to claim 17, characterized in that said data processing unit (P) is configured to compare the output of artificial neural network model of the temperature sensor (3).

19. A test system according to claim 16, characterized in that it is wearable.

20. A test system according to claim 16 or 19, characterized in that said accelerometer (1) and data processing unit (P) are provided in an integrated manner.

21. A test system according to claim 16 or 19, characterized in that said accelerometer (1) and data processing unit (P) are external devices.

22. A test system according to claim 16, characterized in that said accelerometer (1) is triaxial.

23. A test system according to claim 16, characterized in that it is a device that uses a wireless communication module to transmit data to an external device.

24. A test system according to claim 21, characterized in that it comprises a wireless communication module to transmit the data of said accelerometer (1) and temperature sensor (3), wherein said data processing unit (P) is external.

25. A test system according to claim 16, characterized in that said accelerometer (1) and temperature sensor (3) are provided integrated or external, and the data processing unit (P) is an integrated or external device.

Images