Method and system for predicting serious postpartum hemorrhage

Abstract

The embodiment of the invention provides a method and a system for predicting serious postpartum hemorrhage. The method comprises the following steps of: firstly, acquiring multi-modal medical image data, and preprocessing and aligning the multi-modal medical image data; performing segmentation processing on the image through a sliding window strategy, and extracting structural features, texture features, hemodynamic features and blood vessel distribution features from a segmentation result; then the features serve as nodes to construct a graph structure, the relation between the nodes is coded through a graph convolutional network, and the relevance of the features on the topology level is calculated; and finally, fusing the features according to the relevance, inputting a prediction model, and outputting a prediction result. According to the method, through multi-feature fusion and multi-modal data integration, the prediction capability and the prediction accuracy are improved, the robustness of the model in different data environments is enhanced, the bleeding risk can be recognized in time in the middle and later periods of pregnancy of the pregnant woman, early intervention and treatment suggestions are provided, and the risk of bleeding is reduced. The occurrence rate and fatality of serious postpartum hemorrhage are obviously reduced.

Description

[Technical field]

[0001] The invention belongs to the technical field of image processing, and in particular relates to a prediction method and system for severe postpartum hemorrhage. [Background technology]

[0002] Severe postpartum hemorrhage (SPPH) is one of the major complications that occur after childbirth and is one of the leading causes of maternal mortality worldwide. Traditionally, the prediction of SPPH mainly relies on clinical risk assessment, which includes factors such as the mother's medical history, labor process and mode of delivery. However, the assessment of these factors is subjective, and due to large individual differences, the accuracy and sensitivity are relatively limited. In addition, since the occurrence of severe postpartum hemorrhage is often accompanied by suddenness and unpredictability, it is difficult to effectively screen all high-risk mothers based on existing risk factors alone. If a prediction model based on big data and artificial intelligence technology is established by integrating the clinical information, biochemical indicators, and imaging data of pregnant women, the prediction accuracy of SPPH can be effectively improved. This will help to improve the early identification of high-risk mothers, reduce the incidence of severe postpartum hemorrhage and its serious complications, and thus improve the overall health of pregnant women. [Summary of the invention]

[0003] In view of this, an embodiment of the present invention provides a method and system for predicting severe postpartum hemorrhage.

[0004] In a first aspect, an embodiment of the present invention provides a method for predicting severe postpartum hemorrhage, the method comprising:

[0005] S1. Preprocessing and aligning multimodal medical image data after acquiring the multimodal medical image data, wherein the multimodal medical image data includes MRI image data and ultrasound Doppler image data of pregnant women in the middle and late stages of pregnancy;

[0006] S2, segmenting the image using a sliding window strategy, and extracting structural features, texture features, hemodynamic features, and vascular distribution features from the segmentation results;

[0007] S3, constructing a graph structure using features as nodes, using graph convolutional networks to encode the relationships between nodes, and calculating the correlation between structural features, texture features, hemodynamic features, and vascular distribution features at the topological level;

[0008] S4. After integrating the structural features, texture features, hemodynamic features and vascular distribution features according to their correlation, the prediction results are output after input into the prediction model.

[0009] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S1 includes:

[0010] S11, after denoising and suppressing artifacts on multimodal medical imaging data, normalize the resolution and grayscale;

[0011] S12, constructing a preliminary deep learning model for contour detection, locating the uterus and placenta regions to generate regions of interest;

[0012] S13. Select the uterine horn, placental edge, and blood vessel entrance of the MRI image area of ​​interest and the ultrasound Doppler image area of ​​interest respectively. If the thin plate spline deformation alignment can be performed, the two modes are aligned in a unified coordinate system; if the thin plate spline deformation alignment cannot be performed, the two modes are aligned in an adversarial network.

[0013] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S12 specifically includes:

[0014] S121. Build a preliminary deep learning model:

[0015]

[0016] Where X represents the normalized multimodal medical imaging data, Θ represents all trainable parameters of the preliminary deep learning model, and W out and b out represents the weight and bias of the output layer, F e (1) and Represents the feature map of the penultimate layer of the encoder and decoder, CC is the abbreviation of Concat, which means the concatenation of feature maps, US is the abbreviation of UpSample, which means the upsampling operation, σ is the activation function, and represents the weights of the two convolution kernels of the last decoding layer, and Represents the bias of the two convolution kernels of the last decoding layer;

[0017] S122. Construct the loss function of the preliminary learning model:

[0018] Among them, L Dice is the Dice loss, L CE is the cross entropy loss, p i is the predicted value, g i is the predicted value, ε is the true value, y i is the true label, is the prediction probability, N is the number of training samples, α and β are weight parameters;

[0019] S123, minimizing the loss function L(Θ) by gradient descent method based on multiple training samples, and substituting the trainable parameter Θ at this time into the preliminary deep learning model;

[0020] S124. Input the multimodal medical imaging data to be processed into the trained preliminary deep learning model to generate a predicted probability map. Convert the probability map into a binary mask and smooth it. Use the Canny edge detection algorithm to extract the edge contour of the mask. After cropping, generate a region of interest based on the uterus and placenta area.

[0021] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S13 specifically includes:

[0022] S131, respectively select the uterine horn, placental edge and blood vessel entrance of the MRI image area of ​​interest and the ultrasound Doppler image area of ​​interest, extract the precise coordinates, and form an ultrasound image landmark point set and MRI image landmark point set

[0023] S132. Construct thin plate spline mapping function Where A is the translation vector and B is the affine transformation matrix and ω i is the non-rigid deformation weight and U(r)=r 2 log r 2 is the radial basis function, ||·|| is the Euclidean distance; by minimizing the deformation energy, a linear equation system is established to solve A, B and ω i Solve to get the mapping function f(x);

[0024] S133, through the formula Calculate the error MSE value of the landmark points after registration. If the MSE value is within the preset error range, it is determined that the MRI image region of interest and the ultrasound Doppler image region of interest can be deformed and registered with thin plate splines. i The coordinate system of the MRI image is transformed by thin plate spline mapping function for alignment;

[0025] S134. If the MSE value is not within the preset error range, it is determined that the MRI image region of interest and the ultrasound Doppler image region of interest cannot be deformably registered using thin plate splines, and an adversarial network is used for registration and alignment.

[0026] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S134 specifically includes:

[0027] Construct a generator G, a generator F, and a discriminator D MRI and the discriminator D US The adversarial network model, in which the generator G is used to transform the ultrasound image X US Converted to MRI style image G(X US ), the generator F is used to transform the MRI image X MRI Converted to ultrasound style image F(X MRI ), discriminator D MRI Used to determine whether the image is a real MRI image, the discriminator D US Used to determine whether the image is a real ultrasound image;

[0028] Construct a total loss function that includes adversarial loss and cycle consistency loss:

[0029] L total (G,F,D MRI ,D US )=L GAN (G,D MRI ,X US ,X MRI )+L GAN (F,D US ,X MRI ,X US )+λL cycle (G,F);

[0030] Among them, L GAN (G,D MRI ,X US ,X MRI ) represents the generator G and the discriminator D MRI The adversarial loss, L GAN (F,D US ,X MRI ,X US ) represents generators F and D US The adversarial loss, L cycle (G, F) represents the cycle consistency loss, λ is the weight parameter, is the expected operator; it is expressed in the following formula:

[0031]

[0032] Initialize the training parameters, and iterate the training by continuously updating the discriminator and generator until the total loss function reaches the preset number of training rounds to obtain the final adversarial network model;

[0033] Use the generator G to transform the ultrasound image X US Converted to MRI style image G(X US ), and compare it with the MRI image X MRIto align.

[0034] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S2 specifically includes:

[0035] S21, preset window size and sliding step size, window area W (x,y) =I[:,y:y+H,x:x+W], where x and y are the horizontal and vertical coordinates of the upper left corner of the window, H and W are the height and width of the input image, and I is the input image;

[0036] S22, W (x,y) Input into the trained segmentation model f to generate the corresponding segmentation mask M (x,y) =f(W (x,y) ; Θ), splice the segmentation masks of all windows back to the original image size, where K is the number of categories, Θ is the trainable parameter of the segmentation model;

[0037] S23, extracting the boundary of the target area by an edge detection algorithm, calculating the number of boundary pixels, thereby obtaining the perimeter of the target area, and obtaining the area, shape factor and aspect ratio of the target area according to the perimeter, wherein the area, perimeter, shape factor and aspect ratio constitute structural features;

[0038] S24. Calculate the contrast, correlation, uniformity and angular second-order moment of the target area based on the gray level co-occurrence matrix GLCM(i,j), where GLCM(i,j) represents the gray level at a given direction θ and distance d, and the number of times a pixel with gray level i and a pixel with gray level j co-occur under a specific spatial relationship; where

[0039] The contrast ratio is given by the formula Perform calculations;

[0040] The correlation is given by the formula Perform calculations,

[0041] Uniformity is determined by the formula Perform calculations;

[0042] The angular second moment is given by the formula Perform calculations;

[0043] S25. The extracted hemodynamic characteristics include blood flow velocity, blood volume and pulse wave propagation velocity; the vascular distribution characteristics include vascular density, number of vascular branches and vascular curvature.

[0044] According to the above aspects and any possible implementation, an implementation is further provided, wherein S3 specifically includes:

[0045] S31, constructing a graph structure G = (V, E), where the nodes in the node set V represent features, and the edges in the edge set E represent the associations between the nodes;

[0046] S32, update the feature vector of each node through the graph convolution network on the graph structure G = (V, E), where the node feature H of the lth layer (l+1) It is expressed as; in, for The degree matrix of is the adjacency matrix with self-loops added, W (l) is the learnable weight matrix of the lth layer, σ is the activation function;

[0047] S33, after the encoding output of the graph convolutional network, the topological correlation between nodes is calculated based on the embedded vector, and the correlation is measured using cosine similarity is represented by; where z i For node v i The corresponding embedding vector, z i =f(h i ,{h j |j∈N(i)}), h i For node v i The corresponding eigenvector, N(i) node v i The corresponding neighbor node set.

[0048] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S4 specifically includes:

[0049] S41. Generate high-order feature representation based on the correlation between features, through the formula Assign dynamic weights to features of different categories and obtain a comprehensive feature vector f fused ; Among them, α k is the dynamic weight, through the formula Calculate, e k =W T tanh(Wf k +b), W is the weight matrix, b is the bias vector, f k is the feature vector of the kth category, K is the total number of feature categories;

[0050] S42, perform nonlinear dimensionality reduction on the comprehensive feature vector through the trained encoder to generate a low-dimensional feature representation z=f(x)=σ(W e x+b e ), where W e is the weight matrix of the encoder, σ is the activation function, b e is the bias vector of the encoder;

[0051] The encoder is trained as follows:

[0052] Construct an encoder with an input layer size of N, a hidden layer size of M, and an output layer size of N; initialize the weight matrix W of the encoder and decoder e and W d , and the bias vectors b of the encoder and decoder e and b d ; Input f fused The low-dimensional feature z is obtained through the encoder; the low-dimensional feature z is reconstructed through the decoder Calculate the reconstruction error loss function and calculate the loss function relative to W e , W d 、b e and b d The gradient of is calculated and the parameters are updated; the iterative calculation is performed until the loss function converges, and an encoder that can be used independently is obtained.

[0053] S43. Input the low-dimensional features into the trained integrated prediction learning model to obtain the prediction results.

[0054] According to the above aspects and any possible implementation, an implementation is further provided, wherein the integrated prediction learning model structure includes:

[0055] Input layer, used to input low-dimensional features z;

[0056] M independent prediction models predict the low-dimensional feature z respectively and generate their own prediction results Independent prediction models include at least: fully connected neural networks, such as support vector machines, random forests, such as gradient boosting machines;

[0057] Ensemble layer, used to determine the weights ω based on the validation set performance of each model m , weighted integration of the prediction results of all independent models

[0058] Output layer, used to output the final prediction results

[0059] In a second aspect, an embodiment of the present invention provides a prediction system for severe postpartum hemorrhage, the system comprising:

[0060] A preprocessing module, used for preprocessing and aligning multimodal medical image data after acquiring the multimodal medical image data, wherein the multimodal medical image data includes MRI image data and ultrasound Doppler image data of pregnant women in the middle and late stages of pregnancy;

[0061] The segmentation and extraction module is used to segment the image through a sliding window strategy and extract structural features, texture features, hemodynamic features and vascular distribution features from the segmentation results;

[0062] The computational encoding module is used to construct a graph structure using features as nodes, encode the relationship between nodes using a graph convolutional network, and calculate the correlation between structural features, texture features, hemodynamic features, and vascular distribution features at the topological level;

[0063] The fusion prediction module is used to fuse the structural features, texture features, hemodynamic features and vascular distribution features according to their correlation, input them into the prediction model, and then output the prediction results.

[0064] One of the above technical solutions has the following beneficial effects:

[0065] The method of the embodiment of the present invention proposes a prediction method and system for severe postpartum hemorrhage. The method first obtains multimodal medical imaging data and then pre-processes and aligns it. The multimodal medical imaging data includes MRI image data and ultrasonic Doppler image data of pregnant women in the middle and late stages of pregnancy; then the image is segmented by a sliding window strategy, and structural features, texture features, hemodynamic features and vascular distribution features are extracted from the segmentation results; then the features are used as nodes to construct a graph structure, and the relationship between the nodes is encoded using a graph convolutional network, and the correlation of the structural features, texture features, hemodynamic features and vascular distribution features at the topological level is calculated; finally, the structural features, texture features, hemodynamic features and vascular distribution features are fused according to the correlation, and the prediction results are output after input into the prediction model. The method of the present invention improves the prediction ability and accuracy of the prediction through multi-feature fusion and multimodal data integration, and the application of the graph convolutional network improves the model's understanding and processing capabilities of complex data relationships. The method of the present invention has a flexible architecture, can adjust the feature extraction method, the number of graph neural network layers and the type of prediction model according to specific application requirements, and has good scalability and adaptability. The present invention uses a sliding window strategy to segment the image and extract multiple features, thereby enhancing the robustness of the model in different data environments and reducing the prediction error caused by data noise or missing data; calculating the correlation of features at the topological level helps to identify key features and improve the stability and reliability of the model. The method of the present invention will not only be limited to MRI and ultrasonic Doppler image data in the future. The system will be compatible with more types of medical imaging data and adapt to the development and application needs of future medical imaging technology. Through an accurate prediction model, the present invention can timely identify the risk of bleeding in pregnant women in the middle and late stages of pregnancy, provide early intervention and treatment recommendations, and significantly reduce the incidence and lethality of severe postpartum hemorrhage; the integrated prediction system can serve as an auxiliary decision-making tool for clinicians to improve the quality and efficiency of medical services.

Brief Description of the Drawings

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 A schematic flow chart of a method for predicting severe postpartum hemorrhage provided by an embodiment of the present invention;

[0068] Figure 2 A schematic block diagram of a severe postpartum hemorrhage prediction system provided by an embodiment of the present invention;

[0069] Figure 3 A schematic diagram of the hardware structure of a severe postpartum hemorrhage prediction system provided by an embodiment of the present invention. [Specific implementation method]

[0070] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Please refer to Figure 1 , which is a flow chart of a method for predicting severe postpartum hemorrhage provided by an embodiment of the present invention. As shown in the figure, the method comprises the following steps:

[0072] S1. Preprocessing and aligning multimodal medical image data after acquiring the multimodal medical image data, wherein the multimodal medical image data includes MRI image data and ultrasound Doppler image data of pregnant women in the middle and late stages of pregnancy;

[0073] S2, segmenting the image using a sliding window strategy, and extracting structural features, texture features, hemodynamic features, and vascular distribution features from the segmentation results;

[0074] S3, constructing a graph structure using features as nodes, using graph convolutional networks to encode the relationships between nodes, and calculating the correlation between structural features, texture features, hemodynamic features, and vascular distribution features at the topological level;

[0075] S4. After integrating the structural features, texture features, hemodynamic features and vascular distribution features according to their correlation, the prediction results are output after input into the prediction model.

[0076] Specifically, the S1 includes:

[0077] S11, after denoising and suppressing artifacts on multimodal medical imaging data, normalize the resolution and grayscale;

[0078] S12, constructing a preliminary deep learning model for contour detection, locating the uterus and placenta regions to generate regions of interest;

[0079] S13. Select the uterine horn, placental edge, and blood vessel entrance of the MRI image area of ​​interest and the ultrasound Doppler image area of ​​interest respectively. If the thin plate spline deformation alignment can be performed, the two modes are aligned in a unified coordinate system; if the thin plate spline deformation alignment cannot be performed, the two modes are aligned in an adversarial network.

[0080] Specifically, the S12 specifically includes:

[0081] S121. Build a preliminary deep learning model:

[0082]

[0083] Where X represents the normalized multimodal medical imaging data, Θ represents all trainable parameters of the preliminary deep learning model, and W out and b out represents the weight and bias of the output layer, F e (1) and Represents the feature map of the penultimate layer of the encoder and decoder, CC is the abbreviation of Concat, which means the concatenation of feature maps, US is the abbreviation of UpSample, which means the upsampling operation, σ is the activation function, and represents the weights of the two convolution kernels of the last decoding layer, and Represents the bias of the two convolution kernels of the last decoding layer;

[0084] S122. Construct the loss function of the preliminary learning model:

[0085] Among them, L Dice is the Dice loss, L CE is the cross entropy loss, p i is the predicted value, g i is the predicted value, ε is the true value, y i is the true label, is the prediction probability, N is the number of training samples, α and β are weight parameters;

[0086] S123, minimizing the loss function L(Θ) by gradient descent method based on multiple training samples, and substituting the trainable parameter Θ at this time into the preliminary deep learning model;

[0087] S124. Input the multimodal medical imaging data to be processed into the trained preliminary deep learning model to generate a predicted probability map. Convert the probability map into a binary mask and smooth it. Use the Canny edge detection algorithm to extract the edge contour of the mask. After cropping, generate a region of interest based on the uterus and placenta area.

[0088] Specifically, the S13 includes:

[0089] S131, respectively select the uterine horn, placental edge and blood vessel entrance of the MRI image area of ​​interest and the ultrasound Doppler image area of ​​interest, extract the precise coordinates, and form an ultrasound image landmark point set and MRI image landmark point set

[0090] S132. Construct thin plate spline mapping function Where A is the translation vector and B is the affine transformation matrix and ω i is the non-rigid deformation weight and U(r)=r 2 log r 2 is the radial basis function, ||·|| is the Euclidean distance; by minimizing the deformation energy, a linear equation system is established to solve A, B and ω i Solve to get the mapping function f(x);

[0091] S133, through the formula Calculate the error MSE value of the landmark points after registration. If the MSE value is within the preset error range, it is determined that the MRI image region of interest and the ultrasound Doppler image region of interest can be deformed and registered with thin plate splines. i The coordinate system of the MRI image is transformed by thin plate spline mapping function for alignment;

[0092] S134. If the MSE value is not within the preset error range, it is determined that the MRI image region of interest and the ultrasound Doppler image region of interest cannot be deformably registered using thin plate splines, and an adversarial network is used for registration and alignment.

[0093] Specifically, the S134 includes:

[0094] Construct a generator G, a generator F, and a discriminator D MRI and the discriminator DUS The adversarial network model, in which the generator G is used to transform the ultrasound image X US Converted to MRI style image G(X US ), the generator F is used to transform the MRI image X MRI Converted to ultrasound style image F(X MRI ), discriminator D MRI Used to determine whether the image is a real MRI image, the discriminator D US Used to determine whether the image is a real ultrasound image;

[0095] Construct a total loss function that includes adversarial loss and cycle consistency loss:

[0096] L total (G,F,D MRI ,D US )=L GAN (G,D MRI ,X US ,X MRI )+L GAN (F,D US ,X MRI ,X US )+λL cycle (G,F);

[0097] Among them, L GAN (G,D MRI ,X US ,X MRI ) represents the generator G and the discriminator D MRI The adversarial loss, L GAN (F,D US ,X MRI ,X US ) represents generators F and D US The adversarial loss, L cycle (G, F) represents the cycle consistency loss, λ is the weight parameter, is the expected operator; it is expressed in the following formula:

[0098]

[0099] Initialize the training parameters, and iterate the training by continuously updating the discriminator and generator until the total loss function reaches the preset number of training rounds to obtain the final adversarial network model;

[0100] Use the generator G to transform the ultrasound image X US Converted to MRI style image G(X US ), and compare it with the MRI image X MRI to align.

[0101] Specifically, the S2 specifically includes:

[0102] S21, preset window size and sliding step size, window area W (x,y) =I[:,y:y+H,x:x+W], where x and y are the horizontal and vertical coordinates of the upper left corner of the window, H and W are the height and width of the input image, and I is the input image;

[0103] S22, W (x,y) Input into the trained segmentation model f to generate the corresponding segmentation mask M (x,y) =f(W (x,y) ; Θ), splice the segmentation masks of all windows back to the original image size, where K is the number of categories, Θ is the trainable parameter of the segmentation model;

[0104] S23, extracting the boundary of the target area by an edge detection algorithm, calculating the number of boundary pixels, thereby obtaining the perimeter of the target area, and obtaining the area, shape factor and aspect ratio of the target area according to the perimeter, wherein the area, perimeter, shape factor and aspect ratio constitute structural features;

[0105] S24. Calculate the contrast, correlation, uniformity and angular second-order moment of the target area based on the gray level co-occurrence matrix GLCM(i,j), where GLCM(i,j) represents the gray level at a given direction θ and distance d, and the number of times a pixel with gray level i and a pixel with gray level j co-occur under a specific spatial relationship; where

[0106] The contrast ratio is given by the formula Perform calculations;

[0107] The correlation is given by the formula Perform calculations,

[0108] Uniformity is determined by the formula Perform calculations;

[0109] The angular second moment is given by the formula Perform calculations;

[0110] S25. The extracted hemodynamic characteristics include blood flow velocity, blood volume and pulse wave propagation velocity; the vascular distribution characteristics include vascular density, number of vascular branches and vascular curvature.

[0111] Specifically, the S3 specifically includes:

[0112] S31, constructing a graph structure G = (V, E), where the nodes in the node set V represent features, and the edges in the edge set E represent the associations between the nodes;

[0113] S32, update the feature vector of each node through the graph convolution network on the graph structure G = (V, E), where the node feature H of the lth layer (l+1) It is expressed as; in, for The degree matrix of is the adjacency matrix with self-loops added, W (l) is the learnable weight matrix of the lth layer, σ is the activation function;

[0114] S33, after the encoding output of the graph convolutional network, the topological correlation between nodes is calculated based on the embedded vector, and the correlation is measured using cosine similarity is represented by; where z i For node v i The corresponding embedding vector, z i =f(h i ,{h j |j∈N(i)}), h i For node v i The corresponding eigenvector, N(i) node v i The corresponding neighbor node set.

[0115] Specifically, S4 includes:

[0116] S41. Generate high-order feature representation based on the correlation between features, through the formula Assign dynamic weights to features of different categories and obtain a comprehensive feature vector f fused ; Among them, α k is the dynamic weight, through the formula Calculate, e k =W T tanh(Wf k +b), W is the weight matrix, b is the bias vector, f k is the feature vector of the kth category, K is the total number of feature categories;

[0117] S42, perform nonlinear dimensionality reduction on the comprehensive feature vector through the trained encoder to generate a low-dimensional feature representation z=f(x)=σ(W e x+b e ), where W e is the weight matrix of the encoder, σ is the activation function, b e is the bias vector of the encoder;

[0118] The encoder is trained as follows:

[0119] Construct an encoder with an input layer size of N, a hidden layer size of M, and an output layer size of N; initialize the weight matrix W of the encoder and decodere and W d , and the bias vectors b of the encoder and decoder e and b d ; Input f fused The low-dimensional feature z is obtained through the encoder; the low-dimensional feature z is reconstructed through the decoder Calculate the reconstruction error loss function and calculate the loss function relative to W e , W d , b e and b d The gradient of is calculated and the parameters are updated; the iterative calculation is performed until the loss function converges, and an encoder that can be used independently is obtained.

[0120] S43. Input the low-dimensional features into the trained integrated prediction learning model to obtain the prediction results.

[0121] Specifically, the integrated prediction learning model structure includes:

[0122] Input layer, used to input low-dimensional features z;

[0123] M independent prediction models predict the low-dimensional feature z respectively and generate their own prediction results Independent prediction models include at least: fully connected neural networks, such as support vector machines, random forests, such as gradient boosting machines;

[0124] Ensemble layer, used to determine the weights ω based on the validation set performance of each model m , weighted integration of the prediction results of all independent models

[0125] Output layer, used to output the final prediction results

[0126] Through the above steps, the present invention achieves the following technical effects:

[0127] The method of the embodiment of the present invention proposes a prediction method and system for severe postpartum hemorrhage. The method first obtains multimodal medical imaging data and then preprocesses and aligns it. The multimodal medical imaging data includes MRI image data and ultrasonic Doppler image data of pregnant women in the middle and late stages of pregnancy; then the image is segmented by a sliding window strategy, and structural features, texture features, hemodynamic features and vascular distribution features are extracted from the segmentation results; then the features are used as nodes to construct a graph structure, and the relationship between the nodes is encoded using a graph convolutional network, and the correlation between the structural features, texture features, hemodynamic features and vascular distribution features at the topological level is calculated; finally, the structural features, texture features, hemodynamic features and vascular distribution features are fused according to the correlation, and the prediction results are output after inputting into the prediction model.

[0128] The method of the present invention improves the prediction ability and accuracy of prediction through multi-feature fusion and multi-modal data integration, and the application of graph convolutional network improves the model's understanding and processing ability of complex data relationships. The method of the present invention has a flexible architecture, can adjust the feature extraction method, the number of graph neural network layers and the type of prediction model according to specific application requirements, and has good scalability and adaptability. The present invention uses a sliding window strategy to segment the image, extract multiple features, enhance the robustness of the model in different data environments, and reduce the prediction error caused by data noise or missing; calculate the correlation of features at the topological level, help identify key features, and improve the stability and reliability of the model. The method of the present invention will not only be limited to MRI and ultrasonic Doppler image data in the future, and the system will be compatible with more types of medical imaging data to meet the development and application needs of medical imaging technology in the future. Through an accurate prediction model, the present invention can timely identify the risk of bleeding in pregnant women in the middle and late stages of pregnancy, provide early intervention and treatment recommendations, and significantly reduce the incidence and lethality of severe postpartum hemorrhage; the integrated prediction system can be used as an auxiliary decision-making tool for clinicians to improve the quality and efficiency of medical services.

[0129] The embodiments of the present invention further provide device embodiments for implementing the steps and methods in the above method embodiments.

[0130] Please refer to Figure 2 , which is Figure 2 A schematic block diagram of a prediction system for severe postpartum hemorrhage provided by an embodiment of the present invention. The system comprises:

[0131] A preprocessing module 210, for preprocessing and aligning multimodal medical image data after acquisition, wherein the multimodal medical image data includes MRI image data and ultrasound Doppler image data of pregnant women in the middle and late stages of pregnancy;

[0132] The segmentation and extraction module 220 is used to perform segmentation processing on the image by using a sliding window strategy, and extract structural features, texture features, hemodynamic features and vascular distribution features from the segmentation results;

[0133] A computational coding module 230 is used to construct a graph structure using features as nodes, encode the relationship between nodes using a graph convolutional network, and calculate the correlation between structural features, texture features, hemodynamic features, and vascular distribution features at the topological level;

[0134] The fusion prediction module 240 is used to fuse the structural features, texture features, hemodynamic features and vascular distribution features according to the correlation, input them into the prediction model, and then output the prediction result.

[0135] Since each unit module in this embodiment can execute Figure 1 For the method shown in the embodiment, the part not described in detail in this embodiment can be referred to Figure 1 Related instructions.

[0136] Please refer to Figure 3 , which is a hardware structure diagram of a prediction system for severe postpartum hemorrhage provided by an embodiment of the present invention. The data prediction device includes at least one processor and a memory, and the at least one processor is coupled to the memory and is used to read and execute instructions in the memory to execute the data prediction method provided by the embodiment of the present invention.

[0137] In a third aspect, an embodiment of the present invention provides a computer-readable medium having a program code stored therein, and when the program code is run on a computer, the computer executes the method for predicting severe postpartum hemorrhage provided by an embodiment of the present invention.

[0138] At the hardware level, the device may include a processor, and optionally an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. Of course, the device may also include hardware required for other services.

[0139] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0140] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0141] The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0142] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0143] For the convenience of description, the above device is described as various units or modules according to their functions. Of course, when implementing the present invention, the functions of each unit or module can be implemented in the same or multiple software and / or hardware.

[0144] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0148] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0149] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0150] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0151] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0152] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0153] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0154] Each embodiment of the present invention is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0155] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for predicting severe postpartum hemorrhage, characterized in that: The method comprises: S1. Preprocessing and aligning multimodal medical image data after acquiring the multimodal medical image data, wherein the multimodal medical image data includes MRI image data and ultrasound Doppler image data of pregnant women in the middle and late stages of pregnancy; S2, segmenting the image using a sliding window strategy, and extracting structural features, texture features, hemodynamic features, and vascular distribution features from the segmentation results; S3, constructing a graph structure using features as nodes, using graph convolutional networks to encode the relationships between nodes, and calculating the correlation between structural features, texture features, hemodynamic features, and vascular distribution features at the topological level; S4. After integrating the structural features, texture features, hemodynamic features and vascular distribution features according to their correlation, the prediction results are output after input into the prediction model.

2. The method for predicting severe postpartum hemorrhage according to claim 1, characterized in that: The S1 includes: S11, after denoising and suppressing artifacts on multimodal medical imaging data, normalize the resolution and grayscale; S12, constructing a preliminary deep learning model for contour detection, locating the uterus and placenta regions to generate regions of interest; S13. Select the uterine horn, placental edge, and blood vessel entrance of the MRI image area of ​​interest and the ultrasound Doppler image area of ​​interest respectively. If the thin plate spline deformation alignment can be performed, the two modes are aligned in a unified coordinate system; if the thin plate spline deformation alignment cannot be performed, the two modes are aligned in an adversarial network.

3. The method for predicting severe postpartum hemorrhage according to claim 2, characterized in that: The S12 specifically includes: S121. Build a preliminary deep learning model: Where X represents the normalized multimodal medical imaging data, Θ represents all trainable parameters of the preliminary deep learning model, and W out and b out represents the weights and biases of the output layer, and Represents the feature map of the penultimate layer of the encoder and decoder, CC is the abbreviation of Concat, which means the concatenation of feature maps, US is the abbreviation of UpSample, which means the upsampling operation, σ is the activation function, and represents the weights of the two convolution kernels of the last decoding layer, and Represents the bias of the two convolution kernels of the last decoding layer; S122. Construct the loss function of the preliminary learning model: Among them, L Dice is the Dice loss, L CE is the cross entropy loss, p i is the predicted value, g i is the predicted value, ε is the true value, y i is the true label, is the prediction probability, N is the number of training samples, α and β are weight parameters; S123, minimizing the loss function L(Θ) by gradient descent method based on multiple training samples, and substituting the trainable parameter Θ at this time into the preliminary deep learning model; S124. Input the multimodal medical imaging data to be processed into the trained preliminary deep learning model to generate a predicted probability map. Convert the probability map into a binary mask and smooth it. Use the Canny edge detection algorithm to extract the edge contour of the mask. After cropping, generate a region of interest based on the uterus and placenta area.

4. The method for predicting severe postpartum hemorrhage according to claim 3, characterized in that: The S13 specifically includes: S131, respectively select the uterine horn, placental edge and blood vessel entrance of the MRI image area of ​​interest and the ultrasound Doppler image area of ​​interest, extract the precise coordinates, and form an ultrasound image landmark point set and MRI image landmark point set S132. Construct thin plate spline mapping function Where A is the translation vector and B is the affine transformation matrix and ω i is the non-rigid deformation weight and U(r)=r 2 logr 2 is the radial basis function, ||·|| is the Euclidean distance; by minimizing the deformation energy, a linear equation system is established to solve A, B and ω i Solve to get the mapping function f(x); S133, through the formula Calculate the error MSE value of the landmark points after registration. If the MSE value is within the preset error range, it is determined that the MRI image region of interest and the ultrasound Doppler image region of interest can be deformed and registered with thin plate splines. i The coordinate system of the MRI image is transformed by thin plate spline mapping function for alignment; S134. If the MSE value is not within the preset error range, it is determined that the MRI image region of interest and the ultrasound Doppler image region of interest cannot be deformably registered using thin plate splines, and an adversarial network is used for registration and alignment.

5. The method for predicting severe postpartum hemorrhage according to claim 4, characterized in that: The S134 specifically includes: Construct a generator G, a generator F, and a discriminator D MRI and the discriminator D US The adversarial network model, in which the generator G is used to transform the ultrasound image X US Converted to MRI style image G(X US ), the generator F is used to transform the MRI image X MRI Converted to ultrasound style image F(X MRI ), discriminator D MRI Used to determine whether the image is a real MRI image, the discriminator D US Used to determine whether the image is a real ultrasound image; Construct a total loss function that includes adversarial loss and cycle consistency loss: L total (G,F,D MRI ,D US )=L GAN (G,D MRI ,X US ,X MRI )+L GAN (F,D US ,X MRI ,X US )+λL cycle (G,F); Among them, L GAN (G,D MRI ,X US ,X MRI ) represents the generator G and the discriminator D MRI The adversarial loss, L GAN (F,D US ,X MRI ,X US ) represents generators F and D US The adversarial loss, L cycle (G, F) represents the cycle consistency loss, λ is the weight parameter, is the expected operator; it is expressed in the following formula: Initialize the training parameters, and iterate the training by continuously updating the discriminator and generator until the total loss function reaches the preset number of training rounds to obtain the final adversarial network model; Use the generator G to transform the ultrasound image X US Converted to MRI style image G(X US ), and compare it with the MRI image X MRI to align.

6. The method for predicting severe postpartum hemorrhage according to claim 5, characterized in that: The S2 specifically includes: S21, preset window size and sliding step size, window area W (x,y) =I[:,y:y+H,x:x+W], where x and y are the horizontal and vertical coordinates of the upper left corner of the window, H and W are the height and width of the input image, and I is the input image; S22, W (x,y) Input into the trained segmentation model f to generate the corresponding segmentation mask M (x,y) =f(W (x,y) ; Θ), the segmentation masks of all windows are spliced ​​back to the original image size, where K is the number of categories, Θ is the trainable parameter of the segmentation model; S23, extracting the boundary of the target area by an edge detection algorithm, calculating the number of boundary pixels, thereby obtaining the perimeter of the target area, and obtaining the area, shape factor and aspect ratio of the target area according to the perimeter, wherein the area, perimeter, shape factor and aspect ratio constitute structural features; S24. Calculate the contrast, correlation, uniformity and angular second-order moment of the target area based on the gray level co-occurrence matrix GLCM(i,j), where GLCM(i,j) represents the gray level at a given direction θ and distance d, and the number of times a pixel with gray level i and a pixel with gray level j co-occur under a specific spatial relationship; where The contrast ratio is given by the formula Perform calculations; The correlation is given by the formula Perform calculations, Uniformity is determined by the formula Perform calculations; The angular second moment is given by the formula Perform calculations; S25. The extracted hemodynamic characteristics include blood flow velocity, blood volume and pulse wave propagation velocity; the vascular distribution characteristics include vascular density, number of vascular branches and vascular curvature.

7. The method for predicting severe postpartum hemorrhage according to claim 6, characterized in that: The S3 specifically includes: S31, constructing a graph structure G = (V, E), where the nodes in the node set V represent features, and the edges in the edge set E represent the associations between the nodes; S32, update the feature vector of each node through the graph convolution network on the graph structure G = (V, E), where the node feature H of the lth layer (l+1) It is expressed as; in, for The degree matrix of is the adjacency matrix with self-loops added, W (l) is the learnable weight matrix of the lth layer, σ is the activation function; S33, after the encoding output of the graph convolutional network, the topological correlation between nodes is calculated based on the embedded vector, and the correlation is measured using cosine similarity is represented by; where z i For node v i The corresponding embedding vector, z i =f(h i ,{h j |j∈N(i)}), h i For node v i The corresponding eigenvector, N(i) node v i The corresponding neighbor node set.

8. The method for predicting severe postpartum hemorrhage according to claim 7, characterized in that: The S4 specifically includes: S41. Generate high-order feature representation based on the correlation between features, through the formula Assign dynamic weights to features of different categories and obtain a comprehensive feature vector f fused ; Among them, α k is the dynamic weight, through the formula Calculate, e k =W T tanh(Wf k +b), W is the weight matrix, b is the bias vector, f k is the feature vector of the kth category, K is the total number of feature categories; S42, perform nonlinear dimensionality reduction on the comprehensive feature vector through the trained encoder to generate a low-dimensional feature representation z=f(x)=σ(W e x+b e ), where W e is the weight matrix of the encoder, σ is the activation function, b e is the bias vector of the encoder; The encoder is trained as follows: Construct an encoder with an input layer size of N, a hidden layer size of M, and an output layer size of N; initialize the weight matrix W of the encoder and decoder e and W d , and the bias vectors b of the encoder and decoder e and b d ; Input f fused The low-dimensional feature z is obtained through the encoder; the low-dimensional feature z is reconstructed through the decoder Calculate the reconstruction error loss function and calculate the loss function relative to W e , W d 、b e and b d The gradient of is calculated and the parameters are updated; the iterative calculation is performed until the loss function converges, and an encoder that can be used independently is obtained. S43. Input the low-dimensional features into the trained integrated prediction learning model to obtain the prediction results.

9. The method for predicting severe postpartum hemorrhage according to claim 8, characterized in that: The integrated prediction learning model structure includes: Input layer, used to input low-dimensional features z; M independent prediction models predict the low-dimensional feature z respectively and generate their own prediction results Independent prediction models include at least: fully connected neural networks, such as support vector machines, random forests, such as gradient boosting machines; Ensemble layer, used to determine the weights ω based on the validation set performance of each model m , weighted integration of the prediction results of all independent models Output layer, used to output the final prediction results 10. A prediction system for severe postpartum hemorrhage using the method according to any one of claims 1 to 9, characterized in that: The system comprises: A preprocessing module, used for preprocessing and aligning multimodal medical image data after acquiring the multimodal medical image data, wherein the multimodal medical image data includes MRI image data and ultrasound Doppler image data of pregnant women in the middle and late stages of pregnancy; The segmentation and extraction module is used to segment the image through a sliding window strategy and extract structural features, texture features, hemodynamic features and vascular distribution features from the segmentation results; The computational encoding module is used to construct a graph structure using features as nodes, encode the relationship between nodes using a graph convolutional network, and calculate the correlation between structural features, texture features, hemodynamic features, and vascular distribution features at the topological level; The fusion prediction module is used to fuse the structural features, texture features, hemodynamic features and vascular distribution features according to their correlation, input them into the prediction model, and then output the prediction results.

Images