Sleep apnea syndrome disease degree grading method based on multi-modal feature fusion

Abstract

A sleep apnea syndrome disease degree grading method based on multi-modal feature fusion relates to the technical field of artificial intelligence and communication, and comprises the following steps: S1, acquiring a blood oxygen signal image; s2, obtaining first original probability distribution O1 by using a ResNet model; s3, using a DenseNet model to obtain second original probability distribution O2; s4, any real number vector is converted into probability distribution for realizing normalization through a softmax function; s5, dynamically optimizing the model fusion weight through a differential evolution algorithm to obtain an optimal weight; and S6, performing weighted average on the four-classification probabilities obtained by the two models by using an optimal weight, obtaining new four-classification probability distribution, taking a result with the maximum prediction probability, and converting the obtained result into a corresponding label to obtain a final classification result. Through combination of multiple model characteristic values and variation factor optimization model weights, prediction accuracy of sleep apnea syndrome disease degree grading is significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and communication technology, in particular to a sleep apnea syndrome disease degree grading method based on multi-modal feature fusion. BACKGROUND

[0002] Nowadays, sleep apnea syndrome (SAS) diagnosis relies on polysomnography (PSG) monitoring, which needs to analyze apnea hypopnea index (AHI), blood oxygen saturation and other parameters. The traditional method relies on professional physicians to make manual interpretation, which has the problems of low efficiency, strong subjectivity and high occupation of medical resources. How to quickly and low-cost sleep apnea syndrome diagnosis method has become an important topic in modern apnea diagnosis.

[0003] In recent years, artificial intelligence algorithms have developed rapidly. Machine learning, as the core field of artificial intelligence, enables computers to learn rules from data through algorithms, and has shown great potential in medical image analysis. Traditional machine learning relies on manual feature extraction, while deep learning realizes end-to-end automatic feature learning through multi-layer neural networks. In deep learning, convolutional neural networks (CNN) have become the cornerstone of medical image analysis due to their excellent spatial feature extraction capabilities. ResNet effectively solves the problem of gradient disappearance in deep networks through residual connections; DenseNet realizes feature reuse through dense cross-layer connections, significantly improving parameter efficiency. These two architectures perform well on general-purpose datasets such as ImageNet, but when directly applied to the medical field (such as SAS disease degree classification), they still face challenges such as small sample data and insufficient sensitivity to lesion areas. Therefore, current researches often use transfer learning combined with multi-model fusion strategy. Multi-feature value fusion analysis technology is a core method in the field of machine learning and pattern recognition, which can improve the robustness of model decision-making by fusing multi-dimensional features, and is an effective way to solve this problem. SUMMARY

[0004] In view of the deficiencies in the prior art, the purpose of the present application is to provide a sleep apnea syndrome disease degree grading method based on multi-modal feature fusion.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solutions: A sleep apnea syndrome disease degree grading method based on multi-modal feature fusion, comprising the following steps: S1 obtains blood oxygen signal values and pre-processes them to obtain corresponding blood oxygen signal images; S2 inputs the blood oxygen signal images into a pre-trained ResNet model to obtain a first original probability distribution O1; S3 inputs the blood oxygen signal image into the pre-trained DenseNet model to obtain a second original probability distribution O2; S4 converts an arbitrary real number vector into a normalized probability distribution through a softmax function; S5 dynamically optimizes the model fusion weight through a differential evolution algorithm to obtain an optimal weight; S6 performs weighted averaging on the four-class probabilities obtained by the two models using the optimal weight, and obtains a new four-class probability distribution, takes the result with the maximum prediction probability, and converts the obtained result into a corresponding label to obtain a final classification result.

[0006] In S1, the blood oxygen signal value needs to be subjected to linear difference and moving average filtering first, and then the obtained coordinate points are drawn into a blood oxygen image through a drawing function to obtain a blood oxygen signal image as an input.

[0007] Using filling missing data, t for the sampling point time, y for the blood oxygen signal amplitude.

[0008] The moving average filter used in the moving average filtering is , N is 5, x ( n ) is the data after completion and filtering.

[0009] In S4, the image x needs to be subjected to standardization transformation: , wherein , is the image data after standardization, and the image after standardization is subjected to forward propagation to obtain the model outputs and .

[0010] S5 includes the following sub-steps: Generating a plurality of initial weight individuals to constitute an initial weight combination; Setting a mutation factor varying with the number of iterations; In each iteration, a global optimal solution and two different weight individuals are selected at random to generate a mutation vector; Generating a test vector through a crossover operation and calculating the fitness of the test vector; Updating the weight individual according to the fitness and retaining the historical optimal solution; Iterating until a preset stop condition is met, and outputting the optimal weight.

[0011] A computer readable storage medium, having stored thereon a computer program, the program being executed by a processor to implement the above-mentioned sleep apnea syndrome disease degree grading method based on multi-modal feature fusion.

[0012] An electronic device comprising a memory, a processor, and a computer program stored on the memory, the processor implementing the above-mentioned sleep apnea syndrome disease degree grading method based on multi-modal feature fusion when executing the program.

[0013] The beneficial effects of the present application are: Through the joint and variation factor of multi-model characteristic value Optimizing the model weight significantly improves the prediction accuracy of sleep apnea syndrome disease degree grading.

[0014] Strong clinical applicability, supports the input of raw blood oxygen signals or derived graphs, and is compatible with data formats of different medical institutions. It can be integrated into portable devices to realize home screening and reduce the cost of PSG detection. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 : SAS disease degree grading flowchart. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0017] As shown in Figure 1 The present application discloses a sleep apnea syndrome disease degree grading method based on multi-modal feature fusion, which comprises the following steps: It comprises the following steps: S1 obtains blood oxygen signal values and pre-processes them to obtain corresponding blood oxygen signal images; S2 inputs the blood oxygen signal images into a pre-trained ResNet model to obtain a first original probability distribution; S3 inputs the blood oxygen signal images into a pre-trained DenseNet model to obtain a second original probability distribution; S4 converts an arbitrary real number vector into a normalized probability distribution through a softmax function; S5 dynamically optimizes the model fusion weight through a differential evolution algorithm to obtain an optimal weight; S6 uses optimal weights to weight average the four classification probabilities obtained by the two models, and obtains a new four classification probability distribution, takes the result with the maximum prediction probability, and converts the obtained result into the corresponding label to obtain the final classification result.

[0018] In order to realize the judgment of the degree of SAS, the data set obtained from the sleep health data set (SHHS) is preprocessed: the zero level artifact is eliminated and the smoothing processing is performed by the linear interpolation method and the moving average filter method. The linear interpolation method determines the value of an unknown time between two known times by using a straight line connecting the two points. For each sampling point, y is the blood oxygen signal amplitude, assuming that the coordinates are known, To obtain the value of at a certain time in the interval y , the following formula can be used: (1) The missing data is filled by linear interpolation. The obtained new data is put into the moving average filter for filtering. The moving average filter is a signal smoothing method, which regards the continuous sampling data as a queue with a fixed length of After a new measurement, the first data of the queue is removed, the remaining data is moved forward, and the new data is inserted into the tail of the queue. Then, the arithmetic average operation is performed on the queue, and the result is taken as the result of this measurement. The calculation formula of the moving average filter is as follows: (2) The data obtained after completion and filtering x ( n ), considering that the sleep respiratory signal pause time is greater than 10s, and the blood oxygen saturation signal belongs to stable physiological signal, the down-sampling frequency is set to 1Hz, so the moving average filter N is set to 5. The data after linear interpolation is filtered by the moving average filter, and then the two-dimensional image of the blood oxygen saturation is drawn by the drawing function.

[0019] The 1400 images obtained through data preprocessing are manually classified and randomly divided into a training set and a test set. About 20% of the images of each class are randomly selected as test set data. ImageNet is considered for transfer training, and the deep neural network model ResNet50 model pre-trained on the ImageNet dataset is used. The last fully connected layer is replaced with a new layer with an output dimension of four to adapt to the four-class task (non-disease, mild, moderate, severe). Similarly, load the structure model of DenseNet121, change the output layer to a 4-class fully connected layer, and set the initial learning rate to 0.001. Load the training set and test set data, and train the two models to obtain the trained optimal model, which is saved as a.pkl file.

[0020] To implement the SAS disease severity classification discrimination system for input blood oxygen signal images, the process shown in Figure 1 is required. After linear interpolation and moving average filtering of the blood oxygen data to be discriminated, the coordinate points obtained are plotted into a blood oxygen image by a drawing function. The obtained blood oxygen signal image is input and the pre-trained ResNet50 and DenseNet121 models are loaded, which meet the structure: (3) (4) wherein is the image input, is the ResNet feature extractor, which uses the ImageNet pre-trained ResNet50 model, including 5 convolution stages and a global pooling stage, with the output changed to a 4-dimensional feature vector; is the feature extractor of the DenseNet121 model, keeping the original weights unchanged and only changing the classification module to a four-part type; , is the ResNet50 classification layer parameter, fc represents the fully connected layer of ResNet50; represents the classification output of the ResNet architecture-based model for input x ; , is the DenseNet121 fully connected layer parameter, cls represents the fully connected layer of DenseNet121; represents the classification output of the DenseNet architecture-based model for input x .

[0021] Standardize the image x : (5) Take , Given image data that has undergone standard transformation, the outputs of each model are obtained through forward propagation of the transformed image: (6) (7) After modeling, the original probability distribution function is obtained. and The softmax function transforms any real vector into a normalized probability distribution. (8) The representation model represents the first... i The original output scores for each category, The representation model represents the first... r The original output scores for each category, K This represents the number of categories in the classification level.

[0022] (9) (10) Generate 15 initial weight combinations Each individual Satisfy uniform distribution (11) Normalize the weights: (12) , K This represents the total number of categories.

[0023] Set up the variable factor : (13) T This represents the total number of iterations. s This is the current iteration number. The optimal solution from the weight combination is selected as... And randomly select two different individuals , and with Different, use Generate vectors , (14) in s The progress of the labeling algorithm, i.e., the number of iterations.

[0024] Generate test vectors: (15) rand() generates a random number between [0, 1). For randomly selected dimensions, ensure that at least one dimension comes from ; j Number the model weights. j A value of 1 corresponds to a ResNet weight. j 2 corresponds to the DenseNet weights; As the optimal weight.

[0025] Calculate the fitness of the trial vector. (16) (17) For regularization terms; For the first i The correct prediction result for each sample; N The number of samples; The function finds the index of the maximum value. This is an indicator function that returns 1 if the prediction is correct, and 0 otherwise.

[0026] And update the weight combination (18) Preserve the historical best solution.

[0027] (19) The function finds the index of the minimum value. Two weights are then randomly selected. Continue iterating according to the above formulas (14)(15)(16)(18)(19) until the number of iterations reaches a certain threshold. s The iteration stops when the value reaches 100 or the improvement of the optimal solution is less than 0.1% for 5 consecutive iterations, and the optimal weight is output. (20) (twenty one) Using formula (21), the four-class probabilities obtained from the two models are weighted and averaged using the optimized model weights. After weighted averaging, a new four-class probability distribution is obtained. The result with the highest predicted probability is taken and converted into the corresponding label in [no disease, mild, moderate, severe] to obtain the final classification result.

[0028] In step S5, several initial weight combinations are generated, and these weight combinations are optimized to obtain the optimal weights for fusing the first normalized probability distribution and the second normalized probability distribution. The specific steps are as follows: Multiple initial weight individuals are generated to form an initial weight combination; Establish a variation factor that varies with the number of iterations; In each iteration, a globally optimal value and two randomly selected different weight individuals are chosen to generate a mutation vector; Experimental vectors are generated through crossover operations, and the fitness of the experimental vectors is calculated. The weights of individuals are updated based on fitness, while historical best solutions are retained; Iterate until the preset stopping condition is met, and output the optimal weight.

[0029] Specific examples are as follows Generate 15 initial weight combinations , For individuals, , K Define a variation factor to represent the total number of categories. , s For the current iteration number, randomly select three different individuals. , Randomly select one as the initial optimal solution. The other two are as Generate vectors ,pass Obtaining test vectors where rand() generates a random number between [0,1). Representing vectors The One element, Indicates the first s In the next iteration The first vector One element, For randomly selected dimensions, ensure that at least one dimension comes from Calculate the test vector The fitness of the weight combination is updated. ,Will Updated to The value of , and retain the historical best solution. , The function finds the index of the minimum value. Reselect. Through multiple iterations, at the number of iterations... s The iteration stops when the preset value is reached or when the improvement of the optimal solution is less than the set value for multiple consecutive times, and the optimal weight is output. .

[0030] The model was used to classify 140 images. According to the comparison of model performance in Table 1, the accuracy of the fused model is higher than that of the individual models, and the F1 score of the model is improved, reflecting the excellent performance of the model in terms of accuracy and coverage. The accuracy of each disease severity is maintained at a high level.

[0031] Table 1 shows the performance comparison between single and fusion models for SAS disease severity grading.

[0032] After establishing the model, we considered continuing to use this method to test its accuracy under different signal images and the model's generalizability. Respiratory signal data of different SAS severity were obtained from the Apnea-ECG Database dataset. After image preprocessing, the resulting respiratory signal images were used to train a single model using the same method. Then, the same fusion model method was used, and the fusion model results are shown in Table 2. The fusion model showed better accuracy and higher generalizability than the single model.

[0033] Table 2 shows the comparison of the effectiveness of the respiratory signal SAS disease severity fusion model.

[0034]

[0035] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for classifying the severity of sleep apnea syndrome based on multimodal feature fusion.

[0036] The readable storage medium may include a readable medium in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM). Program code is stored thereon, which is executed by a processor to cause the processor to perform the steps of the embodiments described in this specification.

[0037] The memory may also include programs / utilities having a set (at least one) of program modules, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0038] A bus can represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus structures.

[0039] The present invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory. When the processor executes the program, it implements the above-mentioned method for classifying the severity of sleep apnea syndrome based on multimodal feature fusion.

[0040] The electronic device can also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interfaces. Furthermore, the electronic device can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0041] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0042] The embodiments should not be regarded as limitations on the present invention, but any improvements made based on the spirit of the present invention should be within the protection scope of the present invention.

Claims

1. A method for grading the severity of sleep apnea syndrome based on multimodal feature fusion, characterized in that: It includes the following steps: S1 acquires blood oxygen signal values ​​and preprocesses them to obtain the corresponding blood oxygen signal image; S2 inputs the blood oxygen signal image into a pre-trained ResNet model to obtain the first original probability distribution O1; S3 inputs the blood oxygen signal image into the pre-trained DenseNet model to obtain the second original probability distribution O2; S4 uses the softmax function to convert any real vector into a normalized probability distribution. S5 dynamically optimizes the model fusion weights using the differential evolution algorithm to obtain the optimal weights; S6 uses the optimal weights to perform a weighted average of the four-class probabilities obtained from the two models, resulting in a new four-class probability distribution. The result with the highest predicted probability is selected, and the result is converted into the corresponding label to obtain the final classification result.

2. The method for grading the severity of sleep apnea syndrome based on multimodal feature fusion according to claim 1, characterized in that: The blood oxygen signal value in S1 needs to be filtered by linear interpolation and moving average first, and then the obtained coordinate points are plotted into a blood oxygen image through a plotting function to obtain the blood oxygen signal image as input.

3. The method for grading the severity of sleep apnea syndrome based on multimodal feature fusion according to claim 2, characterized in that: use Fill in the missing data. t For sampling point time, y This represents the amplitude of the blood oxygen signal.

4. The method for grading the severity of sleep apnea syndrome based on multimodal feature fusion according to claim 2, characterized in that: The moving average filter used in the moving average filtering is , N It is 5. x ( n () represents the data after completion and filtering.

5. The method for grading the severity of sleep apnea syndrome based on multimodal feature fusion according to claim 1, characterized in that: In S4, the image needs to be processed. x Perform standardization transformation: ,in , Given image data that has undergone standard transformation, the outputs of each model are obtained through forward propagation of the transformed image. and .

6. The method for grading the severity of sleep apnea syndrome based on multimodal feature fusion according to claim 1, characterized in that: S5 includes the following sub-steps: Multiple initial weight individuals are generated to form an initial weight combination; Establish a variation factor that varies with the number of iterations; In each iteration, a global optimal solution is selected and two different weight individuals are randomly selected to generate a mutation vector; Experimental vectors are generated through crossover operations, and the fitness of the experimental vectors is calculated. The weights of individuals are updated based on fitness, while historical best solutions are retained; Iterate until the preset stopping condition is met, and output the optimal weight.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method for classifying the severity of sleep apnea syndrome based on multimodal feature fusion as described in any one of claims 1-6.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the processor executes the program, it implements the method for classifying the severity of sleep apnea syndrome based on multimodal feature fusion as described in any one of claims 1-6.

Images