Insomnia detection method, device and equipment based on infrared thermal imaging and storage medium

Abstract

The invention discloses an insomnia detection method, device and equipment and a computer readable storage medium, and the method comprises the steps: carrying out the image screening in a preset initial thermal imaging image, and obtaining a target thermal imaging image; wherein the thermal imaging image represents a temperature distribution image obtained by thermal imaging, and inputting the target thermal imaging image into an initial training model for training to obtain an insomnia analysis model; wherein the CA module is introduced into the initial training model, the CA module is inserted into the 3 * 3 convolution of the MV2 module, the thermal imaging image of the user is input into the insomnia analysis model for insomnia analysis, and an insomnia analysis result is obtained. According to the invention, the accuracy and stability of insomnia detection can be maintained under the condition of complex and diverse data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of insomnia analysis, and in particular to an insomnia detection method, device and equipment based on infrared thermography and a storage medium. BACKGROUND

[0002] Insomnia is a common neuropsychiatric disorder characterized by persistent difficulty falling asleep, sleep maintenance disturbances, and early morning awakening, accompanied by significant somatic symptoms such as decreased attention, emotional irritability, or lack of energy. Insomnia has become the second largest mental disorder and has caused social and economic burdens through increased medical costs and decreased productivity. Some studies have shown that long-term insomnia can cause severe headaches, mental confusion, fainting, and hallucinations, and lead to symptoms similar to depression. In addition, insomnia is a dangerous factor for hypertension, diabetes, and cerebrovascular disease.

[0003] Although polysomnography (PSG) is the gold standard for insomnia detection, there are still many limitations in clinical diagnosis. For example: the PSG data acquisition process is complex, requiring simultaneous processing of EEG, EMG, ECG and other data, and different medical institution equipment parameters will make data standardization difficult, thereby reducing the accuracy of insomnia detection.

[0004] Infrared thermography (IRT) is a non-invasive program technology based on infrared radiation, which captures the distribution of thermal energy emitted from the human body surface through an infrared camera, generating a visual image reflecting temperature differences. Its core principle is to detect local temperature differences caused by abnormal tissue metabolism or blood flow changes. Introducing infrared thermography data into the field of insomnia detection, using HOG features and SVM classifiers to locate the patient's head and face area, and using CNN to automatically extract insomnia features such as frontal temperature abnormalities and high heat in the eye socket, improves the accuracy of insomnia detection. However, when this method is applied to complex and diverse large-scale data, the detection results may not be stable. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes an insomnia detection method based on infrared thermography, which can maintain the accuracy and stability of insomnia detection in complex and diverse data.

[0006] The present application also proposes an insomnia detection device based on infrared thermography.

[0007] The present application also proposes an insomnia detection device based on infrared thermography.

[0008] The present application also proposes a computer-readable storage medium.

[0009] In a first aspect, one embodiment of the present application provides an insomnia detection method based on infrared thermal imaging, comprising:

[0010] Image screening is performed on the preset initial thermal imaging image to obtain a target thermal imaging image; wherein the thermal imaging image represents an image of temperature distribution obtained by thermal imaging;

[0011] The target thermal imaging image is input into an initial training model for training to obtain an insomnia analysis model; wherein the initial training model includes an MVA module and a RepViT module;

[0012] The thermal imaging image of the user is input into the insomnia analysis model for insomnia analysis to obtain an insomnia analysis result.

[0013] The insomnia detection method of the embodiment of the present application has at least the following beneficial effects: the thermal imaging image is used as the data source, avoiding the complexity of the polysomnogram which requires synchronous collection of multiple physiological signals, and abnormal data is excluded through the image screening link to obtain the target thermal imaging image, which can ensure the effectiveness of the image data. The initial training model is an untrained CaRe-MobileViT model, the insomnia analysis model is a trained CaRe-MobileViT model, the target thermal imaging image is input into the CaRe-MobileViT model for training to obtain the trained CaRe-MobileViT model, the CaRe-MobileViT model replaces the MV2 module with the MVA module to improve the feature expression capability of the model. In addition, the last layer MV2 module is replaced with the RepViT module, the thermal imaging image of the user is input into the CaRe-MobileViT model for insomnia analysis to obtain the insomnia analysis result, the expression capability of the model for multi-scale features is improved without significant increase in computational complexity and model parameter amount, and the accuracy and stability of insomnia detection are maintained in the case of complex and diverse data.

[0014] According to the insomnia detection method of some other embodiments of the present application, before the image screening in the preset initial thermal imaging image to obtain the target thermal imaging image, the method further comprises:

[0015] A first thermal imaging image obtained by shooting is acquired;

[0016] Incorrect images in the first thermal imaging image are removed to obtain the initial thermal imaging image.

[0017] According to the insomnia detection method of some other embodiments of the present application, the removing of the incorrect images in the first thermal imaging image to obtain the initial thermal imaging image comprises:

[0018] If the first thermal imaging image has data information recording error, the current first thermal imaging image is rejected.

[0019] If the first thermal imaging image has index missing, the current first thermal imaging image is rejected.

[0020] If the first thermal imaging image has shooting posture not meeting collection standard, the current first thermal imaging image is rejected.

[0021] If the first thermal imaging image has unclear or temperature interference, the current first thermal imaging image is rejected.

[0022] According to the insomnia detection method of some embodiments of the present application, the image screening in the preset initial thermal imaging image to obtain the target thermal imaging image comprises:

[0023] The initial thermal imaging image is subjected to image segmentation to obtain a second thermal imaging image.

[0024] The image with detection position error in the second thermal imaging image is rejected to obtain the target thermal imaging image.

[0025] According to the insomnia detection method of some embodiments of the present application, the rejection of the image with detection position error in the second thermal imaging image to obtain the target thermal imaging image comprises:

[0026] If the second thermal imaging image has detection position error, the current second thermal imaging image is rejected.

[0027] If the second thermal imaging image has incomplete detection position, the current second thermal imaging image is rejected.

[0028] If the second thermal imaging image has unobvious detection feature, the current second thermal imaging image is rejected.

[0029] According to the insomnia detection method of some embodiments of the present application, before the target thermal imaging image is input into the initial training model for training to obtain the insomnia analysis model, the method further comprises:

[0030] The target thermal imaging image is subjected to data set classification to obtain a training set, a verification set and a test set.

[0031] The training set, the verification set and the test set are respectively input into the initial training model for training.

[0032] In a second aspect, one embodiment of the present application provides an insomnia detection device based on infrared thermal imaging, comprising:

[0033] an image screening module configured to perform image screening on a preset initial thermal imaging image to obtain a target thermal imaging image; wherein the thermal imaging image represents an image of temperature distribution obtained by thermal imaging;

[0034] a model training module configured to input the target thermal imaging image into an initial training model to perform training and obtain an insomnia analysis model; wherein the initial training model comprises an MVA module and a RepViT module;

[0035] an insomnia analysis module configured to input a thermal imaging image of a user into the insomnia analysis model to perform insomnia analysis and obtain an insomnia analysis result.

[0036] In a third aspect, an embodiment of the present application provides an insomnia detection device based on infrared thermal imaging, comprising:

[0037] at least one processor, and

[0038] a memory in communication connection with the at least one processor; wherein

[0039] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the insomnia detection method based on infrared thermal imaging as described in the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer executable instructions for causing a computer to perform the insomnia detection method based on infrared thermal imaging as described in the first aspect.

[0041] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims thereof. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a specific embodiment flowchart of the insomnia detection method in the embodiments of the present application;

[0043] Figure 2 is Figure 1 is a specific embodiment flowchart of step 101 in the method;

[0044] Figure 3 is Figure 2 is a specific embodiment flowchart of step 202 in the method;

[0045] Figure 4is another specific embodiment flow diagram of the insomnia detection method in the embodiment of the present application;

[0046] Figure 5 is Figure 4 is a specific embodiment flow diagram of step 402 in the embodiment of the present application;

[0047] Figure 6 is another specific embodiment flow diagram of the insomnia detection method in the embodiment of the present application;

[0048] Figure 7 is a specific embodiment module block diagram of the insomnia detection device in the embodiment of the present application;

[0049] Figure 8 is a specific embodiment architecture diagram of the CaRe-MobileViT model in the embodiment of the present application;

[0050] Figure 9 is a specific embodiment architecture diagram of the MVA module structure in the embodiment of the present application;

[0051] Figure 10 is a specific embodiment architecture diagram of the RepViT structure in the embodiment of the present application.

[0052] Legend:

[0053] The image screening module 701, the model training module 702, and the insomnia analysis module 703. DETAILED DESCRIPTION

[0054] The concept and the technical effects of the present application will be described below in conjunction with the embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0055] In the description of the present application, if the orientation description such as “up”, “down”, “front”, “back”, “left”, “right” and the like is described, the orientation or position relationship shown in the drawings is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application. If a feature is referred to as “set”, “fixed”, “connected”, “installed” on another feature, it can be directly set, fixed, connected or installed on the other feature, or indirectly set, fixed, connected or installed on the other feature.

[0056] In the description of the embodiments of the application, if "several" is referred to, it means one or more, if "a plurality of" is referred to, it means two or more, if "greater than", "less than", "more than", "fewer than" are referred to, they should all be understood as not including the number itself, if "above", "below", "within" are referred to, they should all be understood as including the number itself. If "first", "second" are referred to, they should be understood as being used to distinguish technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the sequence of indicated technical features.

[0057] Infrared thermography (IRT) is a non-invasive procedure technology based on infrared radiation. It captures the distribution of thermal energy emitted by the human body surface through an infrared camera and generates a visual image reflecting temperature differences. The core principle is to detect local temperature differences caused by abnormal tissue metabolism or blood flow changes. Introducing infrared thermography data into the insomnia detection field, using HOG features and SVM classifiers to locate the patient's head and face area, and using CNN to automatically extract insomnia features such as frontal temperature abnormalities and high heat in the eye socket, improves the accuracy of insomnia detection. However, when this method is applied to complex and diverse large-scale data, the detection results may not be stable.

[0058] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes an insomnia detection method that can improve the stability of insomnia detection while maintaining the accuracy of insomnia detection.

[0059] Reference Figure 1 , Figure 1 A flowchart of an insomnia detection method in an embodiment of the present application is shown. In some embodiments, the insomnia detection method can include, but is not limited to, steps 101 to 103:

[0060] Step 101, image screening in a preset initial thermography image to obtain a target thermography image; wherein the thermography image represents an image of the temperature distribution obtained by thermography.

[0061] In step 101, image screening refers to the process of filtering low-quality or interference data through a preset rule, which can be implemented by image segmentation combined with position integrity detection, for example, after segmenting the head region by an edge detection algorithm, verifying whether the key regions such as the eye socket and forehead are complete. The temperature distribution in the thermography image is a relative temperature, and the thermography image can indirectly reflect the user's temperature.

[0062] The collection instrument of the thermal imaging image is a medical far-infrared thermal imager, the pixel of the detector is >=640x 480, the temperature measurement range is >=25℃-45℃, the temperature resolution is 0.01℃, the spatial resolution is 1.25mrad, the temperature width is set to 6.4, and the infrared thermal image system processing software is used for extraction, data processing and storage of the infrared thermal image. The temperature during detection is room temperature 24℃±2℃, the relative humidity is <90%, there is no direct sunlight and strong light interference, the indoor and outdoor ventilation is isolated, and the cooling or heating source of the detection room is away from the person to be detected. The axillary temperature of the patient is measured before detection, the person to be detected takes off all clothes and sits quietly in the detection environment for 10 minutes to fully dissipate heat and balance body temperature. The infrared skin temperature of the person to be detected is collected by a specially trained staff. The person to be detected stands at a distance of 1.8m from the infrared camera lens, and is photographed 5 times according to the head-up position, the frontal position, the back position and the bilateral position.

[0063] In step 102, the target thermal imaging image is input into an initial training model for training to obtain an insomnia analysis model; wherein the initial training model includes an MVA module and a RepViT module.

[0064] In step 102, the CA module refers to a channel attention mechanism module, which can generate channel weights through global average pooling and a fully connected layer. The CA is integrated into the MV2 module to obtain the MVA module, which enhances the spatial position perception and channel-position feature interaction to improve the feature extraction and fusion capability of the module. The RepViT module is a heterogeneous reparameterizable unit.

[0065] In step 103, the thermal imaging image of the user is input into the insomnia analysis model for insomnia analysis to obtain an insomnia analysis result.

[0066] The steps 101 to 103 shown in the embodiments of the present application use thermal imaging images as data sources, avoiding the complexity of synchronously collecting multiple physiological signals required by polysomnography, and excluding abnormal data through an image screening link to obtain a target thermal imaging image, which can ensure the effectiveness of the image data. The initial training model is a CaRe-MobileViT model that has not been trained, and the insomnia analysis model is a CaRe-MobileViT model that has been trained. The target thermal imaging image is input into the CaRe-MobileViT model for training to obtain a trained CaRe-MobileViT model. The CaRe-MobileViT model replaces the MV2 module with the MVA module to improve the feature expression capability of the model. In addition, the last layer of the MV2 module is replaced with the RepViT module, the thermal imaging image of the user is input into the CaRe-MobileViT model for insomnia analysis to obtain an insomnia analysis result, the expression capability of the model for multi-scale features is improved without significant increase in computational complexity and model parameter quantity, and the accuracy and stability of insomnia detection are maintained in the case of complex and diverse data.

[0067] Referring to Figure 8 , Figure 8 The architecture diagram of the CaRe-MobileViT model in the embodiment of the application is shown. In some embodiments, the CaRe-MobileViT model proposed in the present application is used for insomnia and non-insomnia patients, and the MV2 module is replaced with an MVA module on the structure of MobileViT V1, and the MV2 module of the last layer is replaced with RepViT, and the overall architecture is as shown in Figure 9 During training, MVA serves as a CNN module to down-sample the input features and extract local image features. The MVA module mainly consists of two 1x1 convolutions, a 3x3 deep convolution, and a CoordAtt attention, which are used to adjust the feature channel dimension, extract spatial features, and enhance the attention of the model to key areas. RepViT is responsible for processing high-level semantic information. During training, the input features are extracted by two parallel deep convolutions, then dynamically calibrated by an SE attention module, and finally changed in channel dimension by two 1x1 convolutions, and a skip connection is introduced to enhance the gradient flow and diversity.

[0068] Referring to Figure 2 , Figure 2 The flowchart of the insomnia detection method in the embodiment of the application is shown. In some embodiments, before the image screening in the preset initial thermal imaging image to obtain the target thermal imaging image, the insomnia detection method specifically further includes but is not limited to steps 201 to 202:

[0069] Step 201, acquiring a first thermal imaging image obtained by shooting.

[0070] In step 201, the first thermal imaging image refers to a set of original thermal distribution images directly collected by an infrared imaging device, which is used to construct an original data pool for subsequent quality screening.

[0071] Step 202, removing incorrect images in the first thermal imaging image to obtain an initial thermal imaging image.

[0072] In step 202, the incorrect image refers to thermal imaging data containing data record abnormality, index absence, posture deviation, or image quality defect. Specifically, data record error detection can be achieved by checking the matching of the timestamp in the device log and the image metadata, and index absence judgment can be achieved by checking the completeness of the body temperature parameter, environmental temperature, and other fields in the image auxiliary file, which is used to establish a multi-dimensional data quality evaluation standard.

[0073] The steps 201 to 202 shown in the embodiments of the present application are in the data collection stage, and a first thermal imaging image of the head region of a human body is generated. In the data cleaning process, the timestamp of each frame of image is verified by reading the log file generated by the device to determine whether it is consistent with the collection time recorded by the device, and then it is checked whether the index field is completely contained in the image attached file. The images with missing fields are rejected. Further, the coordinates of the key points of the human body are detected, and when the head tilt angle or the shoulder is not in a horizontal position, it is determined that the shooting posture does not meet the standard. Finally, the definition score is calculated, and when the score is lower than the threshold or the environmental heat source interference area exceeds 5% of the image area, the rejection operation is performed.

[0074] Referring to Figure 3 , Figure 3 A flowchart of the insomnia detection method in the embodiments of the present application is shown. In some embodiments, the images in the first thermal imaging image that are incorrect are rejected to obtain an initial thermal imaging image, which specifically includes but is not limited to steps 301 to 303.

[0075] Step 301, if the first thermal imaging image has a data information record error, the current first thermal imaging image is rejected.

[0076] In step 301, the data information record error refers to a situation where the metadata associated with the image, such as the collection time, device parameters or environmental temperature, has a logical contradiction or is beyond a reasonable range.

[0077] Step 302, if the first thermal imaging image has an index missing, the current first thermal imaging image is rejected.

[0078] In step 302, the index missing refers to a situation where the key physiological parameter annotation is missing, which can be achieved by metadata integrity checking.

[0079] Step 303, if the first thermal imaging image has a shooting posture that does not meet the collection standard, the current first thermal imaging image is rejected.

[0080] In step 303, the shooting posture that does not meet the collection standard refers to a situation where the spatial position of the human body relative to the thermal imaging device is beyond a preset angle range.

[0081] Step 304, if the first thermal imaging image has an unclear or temperature interference, the current first thermal imaging image is rejected.

[0082] In step 304, the unclear or temperature interference refers to a situation where there is motion blur or environmental heat source influence, which can be achieved by using an image quality evaluation algorithm.

[0083] The steps 301 to 304 shown in the embodiments of the present application realize data quality control by establishing a four-level screening mechanism. In the data integrity layer, a check algorithm is used to identify images with abnormal time stamps or parameter out-of-bounds. In the parameter specification layer, metadata integrity verification is performed. In the acquisition standardization layer, computer vision technology is used to detect body position deviation. In the image reliability layer, double detection is implemented in combination with thermal imaging characteristics.

[0084] Referring to Figure 4 , Figure 4 A flowchart of a sleeplessness detection method in the embodiments of the present application is shown. In some embodiments, image screening is performed on a preset initial thermal imaging image to obtain a target thermal imaging image, which specifically includes but is not limited to steps 401 to 402.

[0085] Step 401: Image segmentation is performed on the initial thermal imaging image to obtain a second thermal imaging image.

[0086] In step 401, image segmentation refers to the operation of separating the background region irrelevant to sleeplessness detection in the thermal imaging data, which can be specifically implemented by using a semantic segmentation model based on deep learning to identify the key physiological regions of the head and face and generate a mask.

[0087] Step 402: Images with incorrect detection positions in the second thermal imaging image are removed to obtain a target thermal imaging image.

[0088] In step 402, incorrect detection position refers to the case where the target region deviates from the preset physiological coordinate range, which can be specifically implemented by using a region matching algorithm in combination with a standard anatomical template to perform position verification, and by calculating the geometric feature difference between the target region in the image and the template to realize abnormal positioning.

[0089] The steps 401 to 402 shown in the embodiments of the present application remove the background noise in the original thermal imaging data through image segmentation operation to generate a second thermal imaging image containing only the physiological regions of the head and face. Subsequently, the segmented image is quality evaluated through three-dimensional screening standards of incorrect detection position, incomplete position, and unobvious feature. When the detection region deviates from the preset physiological coordinate range, the incorrect detection position screening is triggered; when the area of the target region is less than a preset threshold, the incomplete position screening is triggered; and when the temperature distribution gradient is lower than the feature recognition sensitivity, the unobvious feature screening is triggered. Through the multi-level screening mechanism, abnormal images with positioning deviation are excluded to ensure that the images input into the model have complete detection regions and obvious temperature features.

[0090] Referring to Figure 5 , Figure 5A flowchart of the insomnia detection method in the embodiment of the present application is shown. In some embodiments, the image with a detection position error in the second thermal imaging image is rejected to obtain a target thermal imaging image, which specifically includes but is not limited to steps 501 to 503:

[0091] In step 501, if the second thermal imaging image has a detection position error, the current second thermal imaging image is rejected.

[0092] In step 501, the detection position error refers to the target region deviating from the preset head or face anatomy position range, which can be specifically realized by comparing with a standard template using an image registration algorithm, and whether the key point coordinate offset exceeds a threshold value is determined.

[0093] In step 502, if the second thermal imaging image has a detection position error, the current second thermal imaging image is rejected.

[0094] In step 502, the detection position error refers to the target region having a partial loss or boundary truncation, which can be specifically realized by using an edge detection algorithm combined with a region integrity evaluation model, and whether the contour closure degree and area proportion meet the requirements is determined.

[0095] In step 503, if the second thermal imaging image has a detection feature that is not obvious, the current second thermal imaging image is rejected.

[0096] In step 503, the detection feature that is not obvious refers to the target region having a lack of significant gradient difference in temperature distribution, which can be specifically realized by calculating the local variance or entropy value after enhancing the contrast using histogram equalization, and whether the value is lower than a set threshold value is determined.

[0097] The steps 501 to 503 shown in the embodiments of the present application, in the image screening process, first, the images with positioning deviation caused by device calibration error or subject movement are excluded by coordinate offset detection, and the samples with accurate spatial position are retained. Then, the images passing the positioning detection are subjected to integrity evaluation, and the images with region loss caused by improper collection angle or shielding are rejected, so as to ensure that the data input into the model has complete face anatomy structure. Finally, the remaining images are subjected to feature saliency analysis, and the low-contrast images caused by environmental temperature interference or abnormal physiological metabolism are filtered, and the samples with clear temperature gradient distribution are retained. Through the three-level progressive screening mechanism, different dimensional interference factors are gradually excluded, and a high-precision and high-integrity training data set is formed.

[0098] Referring to Figure 9 , Figure 9An architecture diagram of the MVA module structure in the embodiments of the present application is shown. In some embodiments, CA (Coordinate Attention) is a spatial-channel collaborative attention module designed for lightweight convolutional neural networks, the core of which is to decouple two-dimensional global context encoding into mutually orthogonal one-dimensional feature aggregation. For input feature X, CA first uses global pooling kernels with sizes (H, 1) and (1, W) to encode each channel along the horizontal and vertical directions, respectively. Therefore, the output of the c-th channel with height h or width w can be represented as:

[0099]

[0100] In order to better utilize this pair of direction-aware attention maps, the authors designed an efficient and effective coordinated attention calculation process. First, the pair of feature maps is concatenated, and then a 1x1 convolution is used to change F1, and the calculation is as shown in the formula:

[0101] f = δ (F1([z h ,z w ])) (3)

[0102] The generated is the intermediate feature map of spatial information in the horizontal and vertical directions, where r represents the down-sampling ratio, which is used to control the size of the module. Secondly, f is divided into two separate tensors along the spatial dimension and Then, two 1x1 convolutions F h and F w are used to transform the feature maps f h and f w to the same number of channels as the input X, resulting in the following formula:

[0103] g h = σ (F h (f h )) (4)

[0104] g w = σ (F w (f w )) (5)

[0105] Finally, g h and g w are expanded as attention weights, and the final output of the CA module can be represented as:

[0106]

[0107] Referring to Figure 6 , Figure 6A flowchart of the insomnia detection method in the embodiment of the application is shown. In some embodiments, before the target thermal imaging image is input into the initial training model for training to obtain the insomnia analysis model, the insomnia detection method specifically further includes but is not limited to steps 601 to 602:

[0108] Step 601, classifying the target thermal imaging image into a data set to obtain a training set, a validation set and a test set.

[0109] In step 601, the number of insomnia samples and non-insomnia samples is balanced, and specifically, undersampling, oversampling or synthetic data technology can be used to achieve this, and by adjusting the sample distribution, class bias is eliminated.

[0110] Step 602, inputting the training set, the validation set and the test set into the initial training model for training.

[0111] In step 602, the training set, the validation set and the test set can be implemented in a ratio of 8:1:1, 7:2:1 or 6:2:2, and by fixing the segmentation method, the normativity of the training process is ensured. Data set classification refers to dividing data into non-overlapping subsets, and specifically, stratified sampling methods can be used to achieve this, and by maintaining the consistency of the distribution of samples of each class in the subset, information leakage is avoided.

[0112] The steps 601 to 602 shown in the embodiments of the application use a stratified division strategy to allocate the target thermal imaging image to the training set, the validation set and the test set, wherein the training set is used for model parameter updating, the validation set is used for hyperparameter tuning and early stopping control, and the test set is used for final performance evaluation. By dividing the independent data subsets, data reuse during the training process is prevented, which makes the model evaluation unreliable, and at the same time, the continuous monitoring of the validation set can dynamically adjust the model complexity, and the isolated evaluation of the test set can objectively reflect the generalization ability of the model.

[0113] For insomnia patients, the data of the head and face (data of the head under thermal imaging) is the key to diagnosis, and to eliminate the interference of non-target area thermal radiation, a conventional target detection process is used to realize the standardized extraction of the head and face region. In this study, an infrared thermal imaging special detection model developed by the research group is used, which uses the YOLOv8 architecture, and the 800 patient hand-raising position data accumulated in the previous project is labeled and trained using LabelMe.

[0114] In addition, further data cleaning is performed on the detection results, and data with incorrect detection positions and incomplete head and face temperature is proposed. Finally, the data set is divided into training set, validation set and test set according to 8:1:1. During training, the bilinear interpolation algorithm is first used to perform size standardization processing on the input image, and the original data is uniformly adjusted to 224*224 pixel resolution. According to the characteristics of infrared thermal imaging, random rotation and random horizontal rotation are added in the data enhancement stage, and there is a 50% probability of using before sending into the network for training. Data enhancement can effectively improve the generalization ability of the model and help improve the detection effect of the model.

[0115] Referring to Figure 10 , Figure 10 The architecture diagram of the RepViT structure in the embodiment of the application is shown. In some embodiments, the RepViT structure of the present application decouples the MobileNet-V3 basic module in structure and dynamically recombines parameters.

[0116] In order to decouple the two operations, the RepViT structure is reconstructed into a cascaded separation structure, as shown in the right side of the figure in Figure 10 This design performs token mixer (spatial dimension interaction) and channel mixer in stages, enhancing the fusion ability of multi-scale local features. This reconstruction strategy enables the token mixer and the channel mixer to form a parallel cooperative path during training, while maintaining the original calculation efficiency during the inference stage.

[0117] The end module of the neural network is responsible for the fine modeling of high-level semantic features. In this study, the last MV2 module of MobileViT V1 is replaced by a RepViT module, enabling the network to more effectively fuse multi-scale local features and enhance global correlation. The replacement result is shown in Figure 10 In addition, the feature expression ability of the model can be improved while maintaining the original calculation complexity of the model.

[0118] In some embodiments, Adam (Adaptive Moment Estimation)

[35] is an optimization algorithm widely used in deep learning, which combines the ideas of momentum and RMSProp, and uses the first and second moments of the gradient to adaptively adjust the learning rate of each parameter, quickly converging and stable performance during training. The specific calculation is shown in the formula:

[0119]

[0120] In the formula: g t represents the gradient at the current time step t; m t and v tare the first and second moment estimates of the gradient, respectively; β1 and β2 are the decay rates of the first and second moments, with default values of 0.9 and 0.999, respectively; and are the bias corrections of m t and v t to mitigate the estimation bias at the initial time; θ is the parameter to be solved.

[0121] The core of the Sharpness-Aware Minimization (SAM) optimization algorithm is to minimize the loss value and the maximum loss in its neighborhood at the same time, so as to guide the model to converge to a flat region. The implementation of the SAM optimizer in this study is based on the improvement of Lin et al., which calculates the perturbation direction through a sign function, and the calculation is represented as:

[0122]

[0123] where γ is a hyperparameter that controls the amplitude of the perturbation. Finally, the gradient update parameter is calculated at the perturbed point θ+∈ *

[0124]

[0125] To adapt to the task type, the loss function is modified to binary cross-entropy (BCE), which is mathematically represented as:

[0126]

[0127] where y i represents the true label of the sample, and p i is the model prediction probability. And the basic optimizer is replaced by Adam.

[0128] In addition, the present application embodiment also discloses an insomnia detection device based on infrared thermal imaging, please refer to Figure 7 , Figure 7 is an embodiment of the present application discloses the module block diagram of insomnia detection device. The insomnia detection device can realize the insomnia detection method described above, and the insomnia detection device comprises: an image screening module 701, a model training module 702 and an insomnia analysis module 703, and the image screening module 701, the model training module 702 and the insomnia analysis module 703 are all in communication connection.

[0129] ​The image screening module 701 performs image screening in the preset initial thermal imaging image to obtain a target thermal imaging image. The thermal imaging image represents an image of a temperature distribution obtained by thermal imaging. The model training module 702 inputs the target thermal imaging image into an initial training model to train an insomnia analysis model; wherein the initial training model includes an MVA module and a RepViT module. The insomnia analysis module 703 inputs the thermal imaging image of the user into the insomnia analysis model to perform insomnia analysis and obtain an insomnia analysis result.

[0130] The image screening module 701 uses a thermal imaging image as a data source, avoids the complexity of needing to synchronously collect multiple physiological signals by polysomnography, and obtains a target thermal imaging image by excluding abnormal data through an image screening link, which can ensure the effectiveness of the image data. The initial training model is an untrained CaRe-MobileViT model, and the insomnia analysis model is a trained CaRe-MobileViT model. The model training module 702 inputs the target thermal imaging image into the CaRe-MobileViT model to train the CaRe-MobileViT model, and the CaRe-MobileViT model replaces the MV2 module with an MVA module to improve the feature expression capability of the model. In addition, the MV2 module in the last layer is replaced with a RepViT module, the insomnia analysis module 703 inputs the thermal imaging image of the user into the CaRe-MobileViT model to perform insomnia analysis and obtain an insomnia analysis result, which improves the expression capability of the model for multi-scale features without significantly increasing the computational complexity and the number of model parameters, and realizes the accuracy and stability of insomnia detection in the case of complex and diverse data.

[0131] The operation process of the insomnia detection device of the embodiment is specifically described with reference to the insomnia detection method steps S101-S103, steps S201 and S202, steps S301-S304, steps S401 and S402, steps S501-S503, and steps S601 and S602 in the above description Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 , and will not be described here.

[0132] Another embodiment of the present application discloses an insomnia detection device based on infrared thermal imaging, comprising: at least one processor, and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method steps of the above embodiments Figure 1the control method steps S101-S103 in the method of detecting insomnia, Figure 2 the control method steps S201 and S202 in the method of detecting insomnia, Figure 3 the control method steps S301-S304 in the method of detecting insomnia, Figure 4 the control method steps S401 and S402 in the method of detecting insomnia, Figure 5 the control method steps S501-S503 in the method of detecting insomnia, and Figure 6 the control method steps S601 and S602 in the method of detecting insomnia.

[0133] Another embodiment of the present application discloses a computer readable storage medium, the storage medium comprising: the storage medium storing computer executable instructions, the computer executable instructions being used for causing a computer to execute the method of detecting insomnia, Figure 1 the control method steps S101-S103 in the method of detecting insomnia, Figure 2 the control method steps S201 and S202 in the method of detecting insomnia, Figure 3 the control method steps S301-S304 in the method of detecting insomnia, Figure 4 the control method steps S401 and S402 in the method of detecting insomnia, Figure 5 the control method steps S501-S503 in the method of detecting insomnia, and Figure 6 the control method steps S601 and S602 in the method of detecting insomnia.

[0134] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0135] As will be appreciated by one of ordinary skill in the art, all or some steps, systems of the above-disclosed methods can be implemented as software, firmware, hardware, or suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media.

[0136] The embodiments of the present application are described in detail above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, and various changes can be made within the scope of knowledge of those skilled in the art without departing from the gist of the present application. Furthermore, the embodiments of the present application and features in the embodiments can be combined with each other without conflict.

Claims

1. A method for detecting insomnia based on infrared thermal imaging, characterized in that, include: Image filtering is performed on a preset initial thermal imaging image to obtain a target thermal imaging image; wherein, the thermal imaging image represents an image of the temperature distribution obtained by thermal imaging; The target thermal imaging image is input into an initial training model for training to obtain an insomnia analysis model; wherein, the initial training model includes an MVA module and a RepViT module; The user's thermal imaging image is input into the insomnia analysis model to perform insomnia analysis and obtain the insomnia analysis results.

2. The insomnia detection method according to claim 1, characterized in that, Before performing image filtering on a preset initial thermal imaging image to obtain the target thermal imaging image, the method further includes: Acquire the first thermal imaging image obtained from the capture; The initial thermal imaging image is obtained by removing erroneous images from the first thermal imaging image.

3. The insomnia detection method according to claim 2, characterized in that, The process of removing erroneous images from the first thermal imaging image to obtain the initial thermal imaging image includes: If the first thermal imaging image contains data information recording errors, then the current first thermal imaging image is discarded; If the first thermal imaging image has missing indicators, then the current first thermal imaging image is discarded. If the first thermal imaging image has a shooting posture that does not meet the acquisition standard, then the current first thermal imaging image is discarded. If the first thermal imaging image is unclear or subject to temperature interference, then the current first thermal imaging image is discarded.

4. The insomnia detection method according to claim 1, characterized in that, The step of selecting images from a preset initial thermal imaging image to obtain a target thermal imaging image includes: The initial thermal imaging image is segmented to obtain a second thermal imaging image; Images with incorrect detection positions in the second thermal imaging image are removed to obtain the target thermal imaging image.

5. The insomnia detection method according to claim 4, characterized in that, The step of removing images with incorrectly detected positions in the second thermal imaging image to obtain the target thermal imaging image includes: If the second thermal imaging image has a detection position error, then the current second thermal imaging image is discarded; If the second thermal imaging image has incomplete detection positions, then the current second thermal imaging image is discarded; If the second thermal imaging image has no obvious detectable features, then the current second thermal imaging image is discarded.

6. The insomnia detection method according to claim 1, characterized in that, Before inputting the target thermal imaging image into the initial training model for training to obtain the insomnia analysis model, the method further includes: The target thermal imaging images are classified into training, validation and test sets. The training set, the validation set, and the test set are respectively input into the initial training model for training.

7. An insomnia detection device based on infrared thermal imaging, characterized in that, include: An image filtering module is used to filter images from a preset initial thermal imaging image to obtain a target thermal imaging image; wherein, the thermal imaging image represents an image of the temperature distribution obtained by thermal imaging; The model training module is used to input the target thermal imaging image into the initial training model for training to obtain the insomnia analysis model; wherein, the initial training model includes an MVA module and a RepViT module; The insomnia analysis module is used to input the user's thermal imaging image into the insomnia analysis model to perform insomnia analysis and obtain insomnia analysis results.

8. An insomnia detection device based on infrared thermal imaging, characterized in that, include: At least one processor, and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the insomnia detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the insomnia detection method as described in any one of claims 1 to 6.

Images