Method and system for analyzing cerebral vascular stenosis based on convolutional neural network
Through convolutional neural network analysis of transcranial Doppler images and physiological data, we can quickly and accurately judge the cerebrovascular stenosis, solve the problems of high examination costs and low recovery rates in the existing technology, and achieve safe and economical diagnosis and treatment Plan formulation.
Patent Information
- Application Number
- CN202410191877.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-21
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-02-21
AI Technical Summary
The prior art cannot quickly and accurately understand the cerebrovascular stenosis while ensuring patient safety and tolerance, resulting in high treatment costs and low recovery rates.
Using a method of cerebrovascular stenosis based on convolutional neural network, the first analysis model is trained to determine whether there is cerebrovascular stenosis through transcranial Doppler images and physiological data, and the second analysis model is trained to evaluate the stenosis degree by combining multimodal information. , providing fast and accurate diagnostic support.
The cost of examination is reduced and the recovery rate of patients is increased. Through the analysis model of convolutional neural network, doctors can quickly understand the patient's cerebrovascular stenosis, formulate reasonable treatment plans, and avoid unnecessary examination costs.
Smart Images

Figure CN118039128B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical information processing technology, and in particular to a method and system for analyzing cerebral vascular stenosis based on a convolutional neural network. Background Art
[0002] Cerebral vascular stenosis is one of the important causes of ischemic cerebrovascular disease. Cerebral vascular stenosis will reduce the blood flow through the cerebral blood vessels, and brain cells will die due to ischemia, thus leading to ischemic cerebrovascular disease. Clinically, different degrees of cerebral vascular stenosis require different treatments. If the cerebral vascular stenosis is mild, only medication is needed for the patient. If the cerebral vascular stenosis is severe, surgical intervention and vascular stent therapy are required for the patient. Therefore, timely and accurate information about the patient's cerebral vascular stenosis is a prerequisite for effective treatment of the patient.
[0003] At present, the examinations commonly used in clinical practice to understand the stenosis of patients' cerebral blood vessels include: Computed Tomography Angiography (CTA), Magnetic Resonance Angiography (MRA), Digital Subtraction Angiography (DSA), etc. However, CTA, DSA and other examinations are radioactive, and DSA is an invasive examination, which has a certain risk of injury to patients; and MRA currently has a long appointment time and a long examination time in most hospitals, which some patients cannot tolerate; at the same time, whether it is CTA, MRA, or MRA, their examination costs are relatively high. Transcranial Doppler (TCD) examination uses ultrasound examination, which is safe, radiation-free and inexpensive. However, TCD requires a high level of experience from the examining physician, and different hospitals may have different interpretations of the same TCD image, which affects the judgment of the patient's condition.
[0004] Therefore, there is an urgent need for a new method for analyzing cerebrovascular stenosis, which can assist doctors in understanding the patient's true cerebrovascular stenosis condition as soon as possible while ensuring the patient's safety, and formulate a reasonable treatment plan for the patient to reduce treatment costs and improve the patient's recovery rate. Summary of the invention
[0005] The present invention provides a method and system for analyzing cerebrovascular stenosis based on a convolutional neural network, which is used to solve the problem that the existing technology cannot assist doctors to understand the patient's true cerebrovascular stenosis condition as soon as possible while ensuring lower risk and higher tolerance for patients.
[0006] The present invention provides a method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network, comprising:
[0007] Obtaining transcranial Doppler images, physiological data, and cerebrovascular stenosis information of a target population, wherein the target population includes a healthy population and a cerebrovascular stenosis patient population, and the transcranial Doppler images of the target population are annotated with a label indicating whether cerebrovascular stenosis occurs;
[0008] Preprocess the transcranial Doppler images of the target population to obtain RGB images of the target population;
[0009] According to the RGB images of the target population, a first convolutional neural network is used to train a first analysis model for analyzing whether the subject has cerebral vascular stenosis;
[0010] According to the RGB images, physiological data and cerebrovascular stenosis information of the target population, a second convolutional neural network and a fully connected neural network are used, combined with a multimodal information fusion mechanism, to train a second analysis model for analyzing the cerebrovascular stenosis of the subjects.
[0011] According to the present invention, a method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is provided, and the physiological data include: age, blood pressure (including systolic pressure and diastolic pressure), gender, Peak value (systolic blood flow peak velocity), Mean value (average flow velocity), PI (pulsatility index), S / D (ratio of peak flow velocity to end-diastolic blood flow velocity).
[0012] According to the present invention, a method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is provided, wherein the first convolutional neural network includes, in order of connection, a first input layer, a first convolutional layer, a first residual block, a first fully connected layer, and a first output layer, wherein the number of neurons in the first output layer is 1, and the activation function is a Sigmoid function.
[0013] According to the present invention, a method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is provided, wherein a first convolutional neural network is used according to an RGB image of a target population to train a first analysis model, comprising:
[0014] Setting the training parameters and initial weights of the first analysis model, the training parameters include: the first epoch number (the number of training iterations), the first loss function, the first model optimization method, the first learning rate, the first momentum, and the adjustment strategy of the first learning rate;
[0015] For each epoch, the training step of the first analysis model is performed to obtain the first analysis model;
[0016] The training steps of the first analysis model include:
[0017] Perform forward calculation based on the RGB image to obtain the first model output of the training sample;
[0018] According to the label of whether cerebral vascular stenosis appears on the RGB image and the first model output of the training sample, a first model loss is obtained through a first loss function;
[0019] Perform back propagation calculation according to the first model optimization method to obtain a first gradient;
[0020] updating the training weights of the first analysis model according to the first gradient;
[0021] updating a first learning rate of the first analysis model according to an adjustment strategy for the first learning rate;
[0022] When the maximum number of first epochs is reached, the final training weights of the first analysis model are obtained, and the training of the first analysis model is terminated.
[0023] According to the present invention, a method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is provided, wherein the second convolutional neural network includes, in sequence, a second input layer, a second convolutional layer, a second residual block, a second fully connected layer, a second hidden layer, and a second output layer in a connection order to form a regression model, and the fully connected neural network includes, in sequence, a third input layer, a third hidden layer, a fourth hidden layer, and a third output layer in a connection order to form a four-layer fully connected model, wherein the second hidden layer of the second convolutional neural network is connected to the third input layer of the fully connected neural network, the number of neurons in the second output layer is 1, the activation function is a Sigmoid function, the activation functions of the third hidden layer and the fourth hidden layer are Relu functions, and the activation function of the third output layer is a Sigmoid function.
[0024] According to the present invention, a method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is provided, wherein the training process of the second analysis model includes a regression model training step and a four-layer fully connected model training step, wherein the second convolutional neural network is used according to the RGB image, physiological data and cerebrovascular stenosis information of the target population, and the second analysis model is trained in combination with a multimodal information fusion mechanism, including:
[0025] Set the training parameters and initial weights of the regression model. The training parameters include: the second epoch number, the second loss function, the second model optimization method, the second learning rate, the second momentum, and the second learning rate adjustment strategy.
[0026] For each epoch, the regression model training steps are executed until the maximum number of second epochs is reached, and the final training weights of the regression model are obtained;
[0027] Extracting physiological numerical features from physiological data, and using the output of the second output layer of the regression model as image features extracted from the RGB image, merging the image features and the physiological numerical features, and combining them with the cerebrovascular stenosis information as training data for a four-layer fully connected model, wherein the output of the second output layer of the regression model and the physiological numerical features are used as training inputs of the four-layer fully connected model, and the cerebrovascular stenosis information is used as training output of the four-layer fully connected model;
[0028] Set the training parameters and initial weights of the four-layer fully connected model. The training parameters include: the third epoch number, the third loss function, the third model optimization method, the third learning rate, the third momentum, and the third learning rate adjustment strategy.
[0029] For each epoch, the four-layer fully connected model training steps are performed until the maximum number of epochs is reached, and the final training weights of the regression model are obtained.
[0030] According to the present invention, a method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is provided, and the regression model training step includes:
[0031] Perform forward calculation based on the RGB image to obtain the second model output of the training sample;
[0032] According to the cerebral vascular stenosis information corresponding to the RGB image and the second model output of the training sample, a second model loss is obtained through a second loss function;
[0033] Perform back propagation calculation according to the second model optimization method to obtain a second gradient;
[0034] Update the weights of the regression model according to the second gradient;
[0035] Update the second learning rate of the regression model according to the adjustment strategy of the second learning rate;
[0036] When the maximum number of second epochs is reached, the final weight of the regression model is obtained, and the regression model training step ends;
[0037] The training steps of the four-layer fully connected model include:
[0038] Perform forward calculation based on the training data of the four-layer fully connected model to obtain the third model output of the training sample;
[0039] According to the cerebrovascular stenosis information of the training data of the four-layer fully connected model and the third model output of the training sample, the third model loss is obtained by using the third loss function;
[0040] Perform back propagation calculation according to the third model optimization method to obtain a third gradient;
[0041] Update the weights of the four-layer fully connected model according to the third gradient;
[0042] Update the third learning rate of the four-layer fully connected model according to the adjustment strategy of the third learning rate;
[0043] When the maximum number of the third epoch is reached, the final training weights of the four-layer fully connected model are obtained, and the four-layer fully connected model training step is ended.
[0044] The present invention provides a method for analyzing cerebral vascular stenosis based on a convolutional neural network, comprising:
[0045] Obtaining transcranial Doppler images and physiological data of a subject to be tested, wherein the subject to be tested is a healthy subject, a subject suspected of having cerebral vascular stenosis, or a subject having already had cerebral vascular stenosis;
[0046] Preprocessing the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested;
[0047] According to the RGB image, the first analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used to obtain an analysis result of whether the subject has cerebrovascular stenosis;
[0048] When the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used according to the RGB image and physiological data of the subject to obtain the degree of cerebrovascular stenosis of the subject.
[0049] The present invention also provides a cerebrovascular stenosis analysis system based on a convolutional neural network, comprising:
[0050] The data acquisition module is used to obtain transcranial Doppler images and physiological data of the subject to be tested, wherein the subject to be tested is a healthy person, a person suspected of having cerebral vascular stenosis, or a person who has already had cerebral vascular stenosis;
[0051] A preprocessing module is used to preprocess the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested;
[0052] A first analysis module is used to obtain an analysis result of whether the subject has cerebrovascular stenosis according to the RGB image by using a first analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network;
[0053] The second analysis module is used for: when the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used to obtain the degree of cerebrovascular stenosis of the subject according to the RGB image and physiological data of the subject.
[0054] The present invention also provides an electronic device, comprising a processor and a memory storing a computer program, wherein when the processor executes the computer program, it implements any of the above-mentioned methods for analyzing cerebrovascular stenosis based on a convolutional neural network and / or methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network.
[0055] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements any of the above-mentioned methods for analyzing cerebrovascular stenosis based on a convolutional neural network and / or methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network.
[0056] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute any of the above-mentioned methods for analyzing cerebrovascular stenosis based on convolutional neural networks and / or methods for constructing a cerebrovascular stenosis analysis model based on convolutional neural networks.
[0057] The present invention provides a method and system for analyzing cerebrovascular stenosis based on a convolutional neural network. The method and system combine transcranial Doppler images, physiological data and cerebrovascular stenosis information, use a convolutional neural network, and train to obtain a first analysis model for analyzing whether a subject has cerebrovascular stenosis and a second analysis model for analyzing the degree of cerebrovascular stenosis of the subject. Before using the first analysis model and the second analysis model, the doctor can first obtain the physiological data of the subject and let the subject undergo a transcranial Doppler TCD examination. TCD uses ultrasound for examination, which is safe and radiation-free, and has low cost. It can ensure the safety of the subject and reduce the examination cost at the same time. After obtaining the transcranial Doppler image of the subject, the doctor can first analyze whether the subject has cerebrovascular stenosis through the first analysis model. When it is indicated that the subject has not had cerebrovascular stenosis, the examination can be terminated to avoid incurring additional examination costs. When it is indicated that the subject has had cerebrovascular stenosis, the doctor can analyze the degree of cerebrovascular stenosis of the subject through the second analysis model. Based on this, the doctor can be assisted to understand the true cerebrovascular stenosis of the subject as soon as possible, and formulate a reasonable treatment plan for the true cerebrovascular stenosis patient, so as to reduce the treatment cost and improve the patient's cure rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0059] Figure 1 A schematic flow chart of a method for constructing a cerebral vascular stenosis analysis model based on a convolutional neural network provided by the present invention.
[0060] Figure 2 An example of a transcranial Doppler image.
[0061] Figure 3 An example of a cropped transcranial Doppler image.
[0062] Figure 4 This is an example of the structure of the residual neural network ResNet-18 used.
[0063] Figure 5 This is an example of the structure of the first analysis model.
[0064] Figure 6 This is an example of the structure of the regression model in the second analysis model.
[0065] Figure 7 This is an example of the structure of the four-layer fully connected model in the second analysis model.
[0066] Figure 8 A structural schematic diagram of a cerebral vascular stenosis analysis system based on a convolutional neural network provided by the present invention.
[0067] Fig. 9 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION
[0068] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0069] Combine the following Figure 1-Figure 9The present invention describes a method and system for analyzing cerebral vascular stenosis based on a convolutional neural network.
[0070] Figure 1 Schematic diagram of the process of constructing a cerebrovascular stenosis analysis model based on a convolutional neural network provided by the present invention. Figure 1 The present invention provides a method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network, which may include:
[0071] Step S110, obtaining transcranial Doppler images, physiological data, and cerebrovascular stenosis information of a target population, wherein the target population includes a healthy population and a cerebrovascular stenosis patient population, and the transcranial Doppler images of the target population are annotated with a label indicating whether cerebrovascular stenosis occurs;
[0072] Step S120, preprocessing the transcranial Doppler image of the target population to obtain an RGB image of the target population;
[0073] Step S130: according to the RGB image of the target population, a first residual neural network is used as a first convolutional neural network to train a first analysis model for analyzing whether the subject has cerebral vascular stenosis;
[0074] Step S140: Based on the RGB images, physiological data, and cerebrovascular stenosis information of the target population, a second residual neural network (the second residual neural network is used as a second convolutional neural network) and a fully connected neural network are used in combination with a multimodal information fusion mechanism to train a second analysis model for analyzing the cerebrovascular stenosis of the subject.
[0075] In one embodiment, the physiological data includes: age, blood pressure (systolic pressure and diastolic pressure), gender (0 and 1 can be used to represent different genders), Peak value (systolic blood flow peak velocity), Mean value (average flow velocity), PI (pulsatility index), S / D (ratio of peak flow velocity to end-diastolic blood flow velocity). The Peak value, Mean value, PI, and S / D can be extracted from the transcranial Doppler image.
[0076] In one embodiment, step S120 may be performed after extracting information from the transcranial Doppler image in step S110, by performing a Figure 2As shown in the figure, the ordinate represents blood flow velocity, the abscissa represents time, and each triangular peak represents a heart beat. The figure is just a black and white picture. The color from blue to red on the original Doppler image represents that the signal is getting stronger and stronger. The stronger the signal, the more red blood cells are at this speed. The data above the picture represents the parameters set during the Doppler examination, such as probe depth, signal frequency, signal strength, etc. The Peak at the bottom of the picture represents the peak blood flow velocity during systole, Mean represents the average blood flow velocity, PI represents the pulsatility index, and S / D represents the ratio of peak flow velocity to end-diastolic blood flow velocity) is cropped, and the transcranial Doppler image part with a black background (such as Figure 3 As shown in the figure, the cropped image is center-cropped to a uniform size (such as 224*224), and then the center-cropped image is converted into a three-channel RGB image, and finally the RGB image is normalized. In this way, the features of each individual in the target group include an RGB image and 8 physiological information features.
[0077] Furthermore, the first analysis model needs to solve a classification problem, and the categories are marked as 0 and 1 (1 represents the presence of cerebral vascular stenosis, and 0 represents the absence of cerebral vascular stenosis); the second analysis model solves a regression problem, and the stenosis degree (range (0, 1)) can be used as the real output. This embodiment constructs a data set for model training based on the characteristics and real labels (whether stenosis occurs and / or the degree of stenosis) of each individual in the target group.
[0078] The preprocessed data set is divided into a training set and a test set (for example, divided in a ratio of 9:1). Since the transcranial Doppler images of the same individual at different depths are similar, in order to make the training results more convincing, the training set and the test set are divided based on individuals (people), that is, all images of an individual belong to either the training set or the test set.
[0079] In order to streamline computing resources as much as possible, this embodiment improves the ResNet-18 residual neural network for the first analysis model and the second analysis model respectively, so as to obtain the first residual neural network and the second residual neural network as the model frameworks of the first analysis model and the second analysis model respectively. Figure 4The figure shows an example of the structure of the ResNet-18 residual neural network (FC: stands for fully connected layer, in the right box, o stands for the output size, k stands for the convolution kernel size, s stands for the convolution step size, p stands for the padding size, and the left box is for the consistency of residual block calculation, the convolution kernel size, number and step size used to reduce the dimension of the original output data). The original ResNet-18 residual neural network contains 18 layers, 17 convolutional layers and 1 fully connected layer, of which 16 of the 17 convolutional layers are in the residual block (that is, each residual block includes two convolutional layers); the overall structure of the network is: input image -> 1 convolutional layer -> 8 residual blocks -> 1 fully connected layer -> analysis results (category or regression value).
[0080] In one embodiment, Figure 5 The structure example of the first residual neural network is shown. The first residual neural network may include a first input layer, a first convolutional layer, a first residual block, a first fully connected layer, and a first output layer in the order of connection. Considering that whether cerebral vascular stenosis occurs is a binary classification problem, this embodiment changes the number of neurons in the first output layer to 1, and the activation function is a Sigmoid function, so that the output result of the first analysis model is 0 or 1. During training, the first residual neural network, i.e., the first analysis model, has an RGB image as input and an output as a classification result of whether cerebral vascular stenosis occurs.
[0081] In one embodiment, step S130 may include:
[0082] The training parameters and initial weights of the first residual neural network are set. The training parameters include: the first epoch number (training number), the first loss function, the first model optimization method, the first learning rate, the first momentum, and the adjustment strategy of the first learning rate. In this embodiment, the first epoch number (training number) is 20, the first loss function is the cross entropy loss function, the first model optimization method is SGD stochastic gradient descent, the first learning rate is 0.001, the first momentum is 0.9, and the adjustment strategy of the first learning rate is to change the learning rate to the original 0.1 every 6 epochs;
[0083] For each epoch, the training step of the first residual neural network is performed to obtain the first analysis model.
[0084] The training step of the first analysis model may include:
[0085] Perform forward calculation based on the RGB image to obtain the first model output of the training sample;
[0086] According to the label of whether cerebral vascular stenosis occurs in the RGB image and the first model output of the training sample, the cross entropy loss is obtained through the cross entropy loss function;
[0087] Perform back propagation calculation according to the first model optimization method to obtain a first gradient;
[0088] updating the training weights of the first analysis model according to the first gradient;
[0089] updating a first learning rate of the first analysis model according to an adjustment strategy for the first learning rate;
[0090] When the maximum number of first epochs is reached, the final training weights of the first analysis model are obtained, and the training of the first analysis model is terminated.
[0091] In one embodiment, the second residual neural network may include, in order of connection, a second input layer, a second convolutional layer, a second residual block, a second fully connected layer, a second hidden layer, and a second output layer to form a regression model. Figure 6 An example of the structure of the regression model part in the second residual neural network is shown. Considering the use of ResNet-18 residual neural network to construct the second analysis model, a multimodal information fusion mechanism is adopted, and the image features of the transcranial Doppler image and the physiological numerical features of the physiological data are considered at the same time. In order to extract image features, this embodiment adds a second hidden layer after the second fully connected layer in the original ResNet-18 residual neural network, and sets its number of neurons to 24, and then improves the number of neurons in the second output layer to 1, and the activation function is the Sigmoid function.
[0092] Furthermore, the second hidden layer output of the second residual neural network is used as a high-level image feature combined with the physiological data feature and input into a four-layer fully connected neural network. The four-layer fully connected neural network includes a third input layer, a third hidden layer, a fourth hidden layer, and a third output layer in a connection order to form a four-layer fully connected model. Figure 7 An example of the structure of the four-layer fully connected model part in the second residual neural network is shown. In this embodiment, the output of the second hidden layer of the regression model is used as the high-level numerical feature extracted from the RGB image, and is combined with 8 physiological features (i.e., the second hidden layer is connected to the third input layer) to build a simple four-layer fully connected neural network (e.g., input layer (32)-hidden layer (30)-hidden layer (30)-output layer (1)). The activation function of the third hidden layer and the fourth hidden layer is the Relu function, and the activation function of the third output layer is the Sigmoid function, so that the analysis result of the second analysis model is in the interval (0, 1). The second analysis model includes a regression model and a four-layer fully connected model. During training, the input of the second analysis model is the RGB image, physiological data and cerebrovascular stenosis information, and the output is cerebrovascular stenosis.
[0093] The training process of the second analysis model includes a regression model training step and a four-layer fully connected model training step. In one embodiment, step S140 may include:
[0094] Set the training parameters and initial weights of the regression model. The training parameters include: the second epoch number, the second loss function, the second model optimization method, the second learning rate, the second momentum, and the adjustment strategy of the second learning rate. In this embodiment, the second epoch number is 20, the second loss function is the RMSE loss function, the second model optimization method is the SGD stochastic gradient descent, the second learning rate is 0.001, the second momentum is 0.9, and the adjustment strategy of the second learning rate is to change the learning rate to the original 0.1 every 6 epochs;
[0095] For each epoch, the regression model training steps are executed until the maximum number of second epochs is reached, and the final training weights of the regression model are obtained;
[0096] Extracting physiological numerical features from the physiological data, and using the output of the second output layer of the regression model as image features extracted from the RGB image, and merging the image features and the physiological numerical features as training data for the four-layer fully connected model;
[0097] Set the training parameters and initial weights of the four-layer fully connected model. The training parameters include: the third epoch number, the third loss function, the third model optimization method, the third learning rate, the third momentum, and the third learning rate adjustment strategy. In this embodiment, the third epoch number is 20, the third loss function is the RMSE loss function, the third model optimization method is the SGD stochastic gradient descent, the third learning rate is 0.001, the third momentum is 0.9, and the third learning rate adjustment strategy is to change the learning rate to the original 0.1 every 6 epochs;
[0098] For each epoch, the four-layer fully connected model training steps are performed until the maximum number of epochs is reached, and the final training weights of the regression model are obtained.
[0099] Among them, the regression model training steps may include:
[0100] Perform forward calculation based on the RGB image to obtain the second model output of the training sample;
[0101] According to the cerebral vascular stenosis information corresponding to the RGB image and the second model output of the training sample, a second model loss is obtained by a second loss function, wherein the second loss function may be an RMSE loss function;
[0102] Perform back propagation calculation according to the second model optimization method to obtain a second gradient;
[0103] Update the weights of the regression model according to the second gradient;
[0104] Update the second learning rate of the regression model according to the adjustment strategy of the second learning rate;
[0105] When the maximum number of second epochs is reached, the final weight of the regression model is obtained, and the regression model training step ends.
[0106] And, the four-layer fully connected model training steps may include:
[0107] Perform forward calculation based on the training data of the four-layer fully connected model to obtain the third model output of the training sample;
[0108] According to the cerebrovascular stenosis information of the training data of the four-layer fully connected model and the third model output of the training sample, a third model loss is obtained by a third loss function, wherein the third loss function may be an RMSE loss function;
[0109] Perform back propagation calculation according to the third model optimization method to obtain a third gradient;
[0110] Update the weights of the four-layer fully connected model according to the third gradient;
[0111] Update the third learning rate of the four-layer fully connected model according to the adjustment strategy of the third learning rate;
[0112] When the maximum number of the third epoch is reached, the final training weights of the four-layer fully connected model are obtained, and the four-layer fully connected model training step is ended.
[0113] A specific embodiment for verifying the analysis effects of the first analysis model and the second analysis model is provided below.
[0114] Shenzhen People's Hospital provided a total of 17 patients with cerebrovascular stenosis and 12 healthy people with diagnosis and treatment data, including 356 transcranial Doppler images of patients with stenosis (i.e., patients with cerebrovascular stenosis) and 319 Doppler images of healthy people (i.e., patients without cerebrovascular stenosis). In the preprocessing stage of transcranial Doppler images, all transcranial Doppler images were converted into three-channel RGB images and saved in jpg format. The transcranial Doppler images were centrally cropped into blocks of 224*224.
[0115] In order to study the effect of the first analysis model, the following experiment is designed in this embodiment: the first residual neural network structure is adopted, and the loss function is changed to cross entropy for binary classification. A total of 20 independent experiments are carried out, and the training set of each experiment is (data of 15 randomly selected patients with cerebral vascular stenosis and 11 healthy people), and the test set is (data of 2 randomly selected patients with cerebral vascular stenosis and 1 healthy person). The gradient update method is SGD stochastic gradient descent, the learning rate is 0.001, and a momentum of 0.9 is added. And the learning rate adjustment strategy is added, which is changed to the previous 0.1 after every 7 epochs, and the number of epochs is 25. Table 1 gives the training set accuracy, test set accuracy and test set maximum accuracy after each epoch training.
[0116] Table 1 Accuracy of the first analysis model during training
[0117]
[0118] In order to study the effect of the second analysis model, the present embodiment designs the following experiment: using the second residual neural network structure (a middle layer of 24 neurons plus an output layer of 1 neuron after the fully connected layer) and a 4-layer fully connected network structure (input layer (32)-hidden layer (30)-hidden layer (30)-output layer (1)), the loss function is RMSE, the gradient update method is SGD stochastic gradient descent, and a momentum of 0.9 is added. A learning rate adjustment strategy is added, which is changed to the previous 0.1 after every 7 epochs. Since the regression result range is (0, 1), the Sigmoid activation function is added to the last output neuron. The sample of the data set after multimodal fusion is a 32-dimensional vector, and the true output is the stenosis (range is (0, 1)). K-fold cross validation is used, and the K value is 5. Table 2 gives the root mean square error values on the 5 validation sets after 25 epochs.
[0119] Table 2 Training results of the fully connected regression model
[0120]
[0121] It can be seen that the analysis accuracy and precision of the first analysis model and the second analysis model are both high. The present invention proposes a two-stage analysis method for the problem of cerebrovascular stenosis analysis, and comprehensively uses multi-source information of the target group for analysis. In the first stage, the first analysis model is trained to determine whether the subject has cerebrovascular stenosis. In the second stage, the multi-source information is combined to further analyze the stenosis of the subject who has cerebrovascular stenosis in the first stage analysis. This method can achieve a higher accuracy rate and provide doctors with more accurate analysis.
[0122] After the first analysis model and the second analysis model are trained, they can be used to analyze the cerebrovascular stenosis of the subject to be tested, so as to assist the doctor to accurately understand the patient's condition. Specific application methods may include:
[0123] Step S110', obtaining a transcranial Doppler image and physiological data of a subject, wherein the subject is a healthy person, a person suspected of having cerebral vascular stenosis, or a person who has already had cerebral vascular stenosis;
[0124] Step S120', preprocessing the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested;
[0125] Step S130', according to the RGB image, using the first analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network, to obtain an analysis result of whether the subject has cerebrovascular stenosis;
[0126] Step S140', when the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the cerebrovascular stenosis degree of the subject is obtained by using a second analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network according to the RGB image and physiological data of the subject.
[0127] The present invention provides a method for analyzing cerebrovascular stenosis based on a convolutional neural network. The method combines transcranial Doppler images and physiological data, adopts a residual neural network, and trains to obtain a first analysis model for analyzing whether a subject has cerebrovascular stenosis and a second analysis model for analyzing the degree of cerebrovascular stenosis of the subject. Before using the first analysis model and the second analysis model, the doctor can first obtain the physiological data of the subject and let the subject undergo a transcranial Doppler TCD examination. The TCD uses ultrasound for examination, which is safe and radiation-free, and has low cost. It can ensure the safety of the subject and reduce the examination cost at the same time. After obtaining the transcranial Doppler image of the subject, the doctor can first analyze whether the subject has cerebrovascular stenosis through the first analysis model. When it is indicated that the subject has not had cerebrovascular stenosis, the examination can be terminated to avoid incurring additional examination costs. When it is indicated that the subject has had cerebrovascular stenosis, the doctor can analyze the degree of cerebrovascular stenosis of the subject through the second analysis model. Based on this, the doctor can be assisted to understand the true cerebrovascular stenosis of the subject as soon as possible, and formulate a reasonable treatment plan for the true cerebrovascular stenosis patient, so as to reduce the treatment cost and improve the patient's cure rate.
[0128] The following is a description of the cerebrovascular stenosis analysis system based on a convolutional neural network provided by the present invention. The cerebrovascular stenosis analysis system based on a convolutional neural network described below and the cerebrovascular stenosis analysis method based on a convolutional neural network described above can be referenced to each other.
[0129] Reference Figure 8 The present invention provides a cerebrovascular stenosis analysis system based on a convolutional neural network, which may include:
[0130] The data acquisition module is used to obtain transcranial Doppler images and physiological data of the subject to be tested, wherein the subject to be tested is a healthy person, a person suspected of having cerebral vascular stenosis, or a person who has already had cerebral vascular stenosis;
[0131] A preprocessing module is used to preprocess the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested;
[0132] A first analysis module is used to obtain an analysis result of whether the subject has cerebrovascular stenosis according to the RGB image by using a first analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network;
[0133] The second analysis module is used for: when the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used to obtain the degree of cerebrovascular stenosis of the subject according to the RGB image and physiological data of the subject.
[0134] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to perform the following steps:
[0135] Obtaining transcranial Doppler images and physiological data of a subject to be tested, wherein the subject to be tested is a healthy subject, a subject suspected of having cerebral vascular stenosis, or a subject having already had cerebral vascular stenosis;
[0136] Preprocessing the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested;
[0137] According to the RGB image, the first analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used to obtain an analysis result of whether the subject has cerebrovascular stenosis;
[0138] When the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used according to the RGB image and physiological data of the subject to obtain the degree of cerebrovascular stenosis of the subject.
[0139] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0140] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can perform the following steps:
[0141] Obtaining transcranial Doppler images and physiological data of a subject to be tested, wherein the subject to be tested is a healthy subject, a subject suspected of having cerebral vascular stenosis, or a subject having already had cerebral vascular stenosis;
[0142] Preprocessing the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested;
[0143] According to the RGB image, the first analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used to obtain an analysis result of whether the subject has cerebrovascular stenosis;
[0144] When the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used according to the RGB image and physiological data of the subject to obtain the degree of cerebrovascular stenosis of the subject.
[0145] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented to perform the following steps when executed by a processor:
[0146] Obtaining transcranial Doppler images and physiological data of a subject to be tested, wherein the subject to be tested is a healthy subject, a subject suspected of having cerebral vascular stenosis, or a subject having already had cerebral vascular stenosis;
[0147] Preprocessing the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested;
[0148] According to the RGB image, the first analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used to obtain an analysis result of whether the subject has cerebrovascular stenosis;
[0149] When the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by any of the above-mentioned methods for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network is used according to the RGB image and physiological data of the subject to obtain the degree of cerebrovascular stenosis of the subject.
[0150] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network, characterized in that: include: Obtaining transcranial Doppler images, physiological data, and cerebrovascular stenosis information of the target population, wherein the target population includes healthy people and patients with cerebrovascular stenosis, and the transcranial Doppler images of the target population are annotated with labels indicating whether cerebrovascular stenosis occurs; Preprocess the transcranial Doppler images of the target population to obtain RGB images of the target population; According to the RGB images of the target population, a first convolutional neural network is used to train a first analysis model for analyzing whether the subject has cerebrovascular stenosis; According to the RGB images, physiological data, and cerebrovascular stenosis information of the target population, a second convolutional neural network and a fully connected neural network are used, combined with a multimodal information fusion mechanism, to train a second analysis model for analyzing the cerebrovascular stenosis of the subjects. The first convolutional neural network includes, in order of connection, a first input layer, a first convolutional layer, a first residual block, a first fully connected layer, and a first output layer, wherein the number of neurons in the first output layer is 1, and the activation function is a Sigmoid function; The first analysis model is obtained by training a first convolutional neural network based on the RGB image of the target population, including: Setting training parameters and initial weights of the first analysis model, the training parameters including: first epoch number, first loss function, first model optimization method, first learning rate, first momentum, and first learning rate adjustment strategy; For each epoch, the training step of the first analysis model is performed to obtain the first analysis model; The training steps of the first analytical model include: Perform forward calculation based on the RGB image to obtain the first model output of the training sample; Obtaining a first model loss through a first loss function according to a label of whether cerebral vascular stenosis occurs in the RGB image and a first model output of the training sample; Perform back propagation calculation according to the first model optimization method to obtain a first gradient; updating the training weights of the first analysis model according to the first gradient; updating a first learning rate of the first analysis model according to an adjustment strategy for the first learning rate; When the maximum number of first epochs is reached, the final training weight of the first analysis model is obtained, and the training of the first analysis model is terminated; The second convolutional neural network includes, in order of connection, a second input layer, a second convolutional layer, a second residual block, a second fully connected layer, a second hidden layer, and a second output layer, forming a regression model. The fully connected neural network includes, in order of connection, a third input layer, a third hidden layer, a fourth hidden layer, and a third output layer, forming a four-layer fully connected model, wherein the second hidden layer of the second convolutional neural network is connected to the third input layer of the fully connected neural network, the number of neurons in the second hidden layer is 24, the number of neurons in the second output layer is 1, the activation function is a Sigmoid function, the activation functions of the third hidden layer and the fourth hidden layer are Relu functions, and the activation function of the third output layer is a Sigmoid function; The training process of the second analysis model includes a regression model training step and a four-layer fully connected model training step, wherein the second analysis model is trained based on the RGB images, physiological data, and cerebrovascular stenosis information of the target population, using a second convolutional neural network and a fully connected neural network, combined with a multimodal information fusion mechanism, including: Set the training parameters and initial weights of the regression model. The training parameters include: the second epoch number, the second loss function, the second model optimization method, the second learning rate, the second momentum, and the second learning rate adjustment strategy. For each epoch, the regression model training steps are executed until the maximum number of second epochs is reached, and the final training weights of the regression model are obtained; Extracting physiological numerical features from the physiological data, and using the output of the second output layer of the regression model as image features extracted from the RGB image, and merging the image features and the physiological numerical features as training data for the four-layer fully connected model; Set the training parameters and initial weights of the four-layer fully connected model. Training parameters: the third epoch number, the third loss function, the third model optimization method, the third learning rate, the third momentum, and the third learning rate adjustment strategy. For each epoch, the four-layer fully connected model training steps are executed until the maximum number of epochs is reached, and the final training weights of the regression model are obtained; The regression model training steps include: Perform forward calculation based on the RGB image to obtain the second model output of the training sample; According to the cerebral vascular stenosis information corresponding to the RGB image and the second model output of the training sample, a second model loss is obtained through a second loss function; Perform back propagation calculation according to the second model optimization method to obtain a second gradient; Update the weights of the regression model according to the second gradient; Update the second learning rate of the regression model according to the adjustment strategy of the second learning rate; When the maximum number of second epochs is reached, the final weight of the regression model is obtained, and the regression model training step ends; And / or, the four-layer fully connected model training steps include: Perform forward calculation based on the training data of the four-layer fully connected model to obtain the third model output of the training sample; According to the cerebrovascular stenosis information of the training data of the four-layer fully connected model and the third model output of the training sample, a third model loss is obtained through a third loss function; Perform back propagation calculation according to the third model optimization method to obtain a third gradient; Update the weights of the four-layer fully connected model according to the third gradient; Update the third learning rate of the four-layer fully connected model according to the adjustment strategy of the third learning rate; When the maximum number of the third epoch is reached, the final training weights of the four-layer fully connected model are obtained, and the four-layer fully connected model training step is ended.
2. A cerebrovascular stenosis analysis system based on convolutional neural network, characterized in that: include: The data acquisition module is used to obtain transcranial Doppler images and physiological data of the subject to be tested, wherein the subject to be tested is a healthy person, a person suspected of having cerebral vascular stenosis, or a person who has already had cerebral vascular stenosis; A preprocessing module is used to preprocess the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested; A first analysis module is used to: obtain an analysis result of whether the subject has cerebrovascular stenosis according to the RGB image by using the first analysis model constructed by the method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network according to claim 1; The second analysis module is used for: when the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by the method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network according to claim 1 is used to obtain the cerebrovascular stenosis degree of the subject according to the RGB image and physiological data of the subject.
3. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor performs the following steps: Obtaining transcranial Doppler images and physiological data of a subject to be tested, wherein the subject to be tested is a healthy subject, a subject suspected of having cerebral vascular stenosis, or a subject having already had cerebral vascular stenosis; Preprocessing the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested; According to the RGB image, a first analysis model constructed by the method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network according to claim 1 is used to obtain an analysis result of whether the subject has cerebrovascular stenosis; When the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by the method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network according to claim 1 is used to obtain the degree of cerebrovascular stenosis of the subject based on the RGB image and physiological data of the subject.
4. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor Follow these steps: Obtaining transcranial Doppler images and physiological data of a subject to be tested, wherein the subject to be tested is a healthy subject, a subject suspected of having cerebral vascular stenosis, or a subject having already had cerebral vascular stenosis; Preprocessing the transcranial Doppler image of the subject to be tested to obtain an RGB image of the subject to be tested; According to the RGB image, a first analysis model constructed by the method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network according to claim 1 is used to obtain an analysis result of whether the subject has cerebrovascular stenosis; When the analysis result of the first analysis model indicates that the subject has cerebrovascular stenosis, the second analysis model constructed by the method for constructing a cerebrovascular stenosis analysis model based on a convolutional neural network according to claim 1 is used to obtain the degree of cerebrovascular stenosis of the subject based on the RGB image and physiological data of the subject.
