Spontaneous premature delivery prediction system and method based on B ultrasonic image

Abstract

The invention relates to a spontaneous premature delivery prediction system and method based on a B ultrasonic image, the system comprises a model training unit and a spontaneous premature delivery prediction unit, the model training unit is used for training models required by spontaneous premature delivery prediction, and the models comprise an image segmentation model and a multiple regression prediction model; the spontaneous premature delivery prediction unit is used for performing cervical segmentation and image feature extraction on a clinical image by using a trained image segmentation model, summarizing image features and table data, performing overall risk prediction by using a trained multiple regression prediction model and outputting a prediction result of the overall risk; the prediction result is used for reflecting the probability of premature delivery of the pregnant woman, and auxiliary information is provided for a doctor.

Description

Technical Field

[0001] The present invention relates to the technical field of computer-aided diagnosis, and in particular to a spontaneous premature birth prediction system and method based on B-ultrasound images. Background Art

[0002] Premature birth refers to delivery after 28 weeks of pregnancy but before 37 weeks. Premature birth can be divided into iatrogenic and spontaneous premature birth. The former refers to the early termination of pregnancy due to pregnancy complications or comorbidities for the safety of mother and child; the latter refers to premature birth caused by non-iatrogenic reasons.

[0003] Spontaneous premature birth seriously endangers the health of offspring. More than one-third of neonatal deaths are related to premature birth. Premature births are also at a greatly increased risk of developing sepsis, patent ductus arteriosus, mental retardation, and visual and hearing disorders. The risk of chronic non-communicable diseases is also significantly higher than that of the general population. The prediction of spontaneous premature birth can guide clinical precision intervention, thereby prolonging pregnancy, promoting fetal development and maturation, and reducing the harm of premature birth to the health of offspring.

[0004] Cervical ultrasound imaging is an important test for predicting spontaneous preterm birth, among which cervical length is the most frequently used feature. The hospital will monitor the cervical length of pregnant women at different stages of pregnancy. When the cervical length shortens, the probability of premature birth increases, and the hospital will conduct clinical intervention. In addition, the doctor will observe other characteristics, such as cervical shape, cervical angle, etc., for comprehensive consideration. Cervical ultrasound examination is an imaging examination that relies on manual measurement by doctors. Taking cervical length as an example, the measurement requires the doctor to identify the position of the internal and external cervical os on the ultrasound image, and then use the caliper that comes with the ultrasound system to measure the length. The doctor's experience and subjective factors will affect the accuracy of the results.

[0005] AI-assisted diagnosis is not limited by the doctor's experience. The application of AI in medical imaging can enable some community hospitals or hospitals in remote areas to approach the technical level that was previously only available in top hospitals. In this way, more people can get better medical care. In addition, patients who need to be observed for danger can also be tested more frequently near their residence to detect risks early. AI-assisted diagnosis has been widely studied in radiology imaging (CT, MRI, X-ray, etc.), but it is still in its infancy in predicting premature birth using B-ultrasound images.

[0006] The application of artificial intelligence in ultrasound imaging faces some challenges and difficulties. First, ultrasound images usually have low contrast and noise, which increases the complexity of image analysis and pattern recognition. Second, ultrasound images are diverse, and image features vary greatly under different devices and scanning conditions, which leads to challenges in model generalization ability.

[0007] In addition to ultrasound examination, other detection indicators are also correlated with preterm birth, such as basic information like patient age and medical history, as well as laboratory test indicators such as blood tests and urine tests. These tabular data, together with ultrasound examination, serve as the clinical reference basis. Summary of the Invention

[0008] The present invention aims to provide a system and method for predicting spontaneous preterm birth based on B-ultrasound images. The technical problems to be solved at least include how to obtain effective information from cervical B-ultrasound images through cervical region segmentation and image feature extraction using artificial intelligence, and combine the tabular data to provide auxiliary diagnosis for doctors.

[0009] To achieve the above object, the present invention provides a system for predicting spontaneous preterm birth based on B-ultrasound images, including a model training unit and a spontaneous preterm birth prediction unit. The model training unit is used to train the models required for predicting spontaneous preterm birth, including an image segmentation model and a multiple regression prediction model. The spontaneous preterm birth prediction unit is used to perform cervical segmentation and image feature extraction on clinical images using the trained image segmentation model, summarize the image features and tabular data, and use the trained multiple regression prediction model to perform overall risk prediction and output the prediction result of the overall risk. The prediction result is used to reflect the probability of a pregnant woman having a preterm birth and provide auxiliary information for doctors.

[0010] Preferably, the input of the model training unit is an image data set and a tabular data set, and the output is an image segmentation model and a multiple regression prediction model.

[0011] Preferably, the image data set includes cervical B-ultrasound images and annotations of the cervical region range.

[0012] Preferably, the model training unit includes an image data loading module, an image segmentation training module, an image feature extraction module, a table data loading module, a regression model training module, and a model export module. The image data loading module is used to load cervical B-ultrasound images and image segmentation annotations in the image dataset. The image segmentation training module uses the open-source model Efficient-UNet based on TensorFlow and performs cervical B-ultrasound image segmentation training according to the image segmentation annotations. The training process has a total of 1000 rounds, the learning rate is 0.0001, the optimizer is adaptive moment estimation, and the loss is the intersection over union. After the training is completed, an image segmentation model is obtained. The image feature extraction module is used to extract 6 image features of the cervical B-ultrasound images in the image dataset. The table data loading module is used to load 10 table features in the table dataset. The regression model training module is used to combine the table features in the table dataset and the image features extracted from the cervical B-ultrasound images in the image dataset as independent variables, and use whether preterm birth occurs as the dependent variable to train a multiple regression prediction model. The model export module is used to export the trained image segmentation model and multiple regression prediction model.

[0013] Preferably, the 6 image features extracted by the image feature extraction module include the area of the cervical region, the longest diameter of the cervical region, the shortest diameter of the cervical region, the degree of cervical depression, the area of the largest black region in the cervix, and the proportion of the area of the largest black region in the cervix.

[0014] Preferably, the area of the cervical region is used to represent the overall situation of the cervical region. The calculation method of the area of the cervical region is as follows: after calculating the area of the cervical region on the calculation image, it is scaled according to the scale of the image, and the true cervical area A is obtained according to the following calculation formula: where M is the polygon forming the cervical region; s is the scale of the B-ultrasound image; area is the function for calculating the polygon area.

[0015] Preferably, the degree of cervical depression is represented by the difference I between the area of the cervical region and the area of the convex hull of the cervical region. The difference I between the area of the cervical region and the area of the convex hull of the cervical region is obtained according to the following calculation formula: H = hull(M) where hull is the function for finding the convex hull of the polygon; area is the function for calculating the polygon area; M is the polygon forming the cervical region; s is the scale of the B-ultrasound image.

[0016] Preferably, the longest diameter and the shortest diameter of the cervical region are used to represent the stretching condition of the cervical region, and are obtained according to the following calculation formula: Where M is the polygon forming the cervical region; m1 and m2 are the antipodal points within the polygon M forming the cervical region; pairs is the set composed of the antipodal points m1 and m2; max_diameter is the maximum distance between the antipodal points m1 and m2; min_diameter is the minimum distance between the antipodal points m1 and m2; D1 is the longest diameter of the cervical region; D2 is the shortest diameter of the cervical region; s is the scale of the B-ultrasound image.

[0017] Preferably, the area of the largest black region in the cervix and the area ratio of the largest black region in the cervix are obtained according to the following calculation formula: Where blacks represents all the black regions within the cervical region with a gray level greater than a predetermined threshold, M' represents the gray level, threshold represents the predetermined threshold, findCounters is a function for finding continuous regions, blacks i represents the i-th black region within the cervical region with a gray level greater than the predetermined threshold, B represents the area of the largest black region in the cervix, s represents the scale of the B-ultrasound image, Q represents the area ratio of the largest black region in the cervix; A represents the actual cervical area.

[0018] Preferably, the 10 table features loaded by the table data loading module include chorionicity, assisted reproductive technology, fasting blood glucose, blood urea nitrogen, history of uterine fibroids, history of adenomyosis, cervical incompetence, inflammation, anemia, and thrombin time.

[0019] Preferably, the inflammation includes neutrophil percentage and white blood cell level.

[0020] Preferably, the anemia includes hemoglobin and hematocrit.

[0021] Preferably, the input of the spontaneous preterm birth prediction unit is an image segmentation model, a multiple regression prediction model, clinical images, and table data, and the output is a prediction result of the overall risk.

[0022] Preferably, the spontaneous preterm birth prediction unit includes an image loading module, a model loading module, a cervical region segmentation module, an image feature extraction module, a tabular data loading module, and an overall risk prediction module. The image loading module is used to load the cervical B-ultrasound image in the clinical image. The model loading module is used to load the trained image segmentation model and the multiple regression prediction model. The cervical region segmentation module is used to segment the cervical region of the cervical B-ultrasound image in the clinical image loaded by the image loading module by using the trained image segmentation model. The image feature extraction module is used to extract image features on the basis of the segmentation result to obtain image features. The tabular data loading module is used to load tabular data and extract the required tabular features from the tabular data. The overall risk prediction module is used to summarize the tabular features and the image features and input them into the multiple regression prediction model. The multiple regression prediction model performs overall risk prediction, and finally outputs the prediction result of the overall risk. The prediction result is used to reflect the probability of a pregnant woman having a preterm birth and provides auxiliary information for doctors.

[0023] The present invention also provides a prediction method for a spontaneous preterm birth prediction system based on B-ultrasound images, including the following steps: The first step, cervical segmentation model training: Load the cervical B-ultrasound images and image segmentation annotations in the image dataset, and use the open-source model Efficient-UNet based on TensorFlow to perform cervical B-ultrasound image segmentation training according to the image segmentation annotations. The training process has a total of 1000 rounds, the learning rate is 0.0001, the optimizer is adaptive moment estimation, and the loss is the intersection over union. After training is completed, an image segmentation model is obtained. The second step, B-ultrasound image feature extraction: Extract 6 image features of the cervical B-ultrasound images in the image dataset. The 6 image features include the cervical region area, the longest diameter of the cervical region, the shortest diameter of the cervical region, the degree of cervical depression, the area of the largest black region in the cervix, and the proportion of the area of the largest black region in the cervix. The cervical region area is used to represent the overall situation of the cervical region. The calculation method of the cervical region area is as follows: after calculating the area of the cervical region on the calculation image, scale it according to the scale of the image, and obtain the real cervical area A according to the following calculation formula: where M is the polygon that makes up the cervical region; s is the scale of the B-ultrasound image; area is the function for calculating the polygon area; The degree of cervical depression is represented by the difference I between the area of the cervical region and the area of the convex hull of the cervical region. The difference I between the area of the cervical region and the area of the convex hull of the cervical region is obtained according to the following calculation formula: H = hull(M) where hull is a function for finding the convex hull of a polygon; area is a function for calculating the area of a polygon; M is the polygon that makes up the cervical region; s is the scale of the B-ultrasound image; The longest diameter and the shortest diameter of the cervical region are used to represent the stretching condition of the cervical region, and are obtained according to the following calculation formula: where M is the polygon that makes up the cervical region; m1 and m2 are antipodal points within the polygon M that makes up the cervical region; pairs is the set composed of the antipodal points m1 and m2; max_diameter is the maximum distance between the antipodal points m1 and m2; min_diameter is the minimum distance between the antipodal points m1 and m2; D1 is the longest diameter of the cervical region; D2 is the shortest diameter of the cervical region; s is the scale of the B-ultrasound image; The area of the largest black region in the cervix and the proportion of the area of the largest black region in the cervix are obtained according to the following calculation formula: where blacks represents all black regions within the cervical region with a gray level greater than a predetermined threshold, M’ represents the gray level, threshold represents the predetermined threshold, findCounters is a function for finding continuous regions, blacks i represents the i-th black region within the cervical region with a gray level greater than the predetermined threshold, B represents the area of the largest black region in the cervix, s represents the scale of the B-ultrasound image, Q represents the proportion of the area of the largest black region in the cervix; A represents the actual cervical area; Step 3: Load tabular data: Load 10 table features in the table dataset; the 10 table features include chorionicity, assisted reproductive technology, fasting blood glucose, blood urea nitrogen, history of uterine fibroids, history of adenomyosis, cervical incompetence, inflammation, anemia, and thrombin time; Step 4: Multiple regression model training: The table features in the table data set and the image features extracted from the cervical B-ultrasound images in the image data set are combined as independent variables, and whether premature birth is used as the dependent variable to train a multivariate regression prediction model; the trained image segmentation model and multivariate regression prediction model are exported; Step 5: Prediction of spontaneous preterm birth: Load the cervical B-ultrasound image in the clinical image, load the trained image segmentation model and the multivariate regression prediction model; use the trained image segmentation model to segment the cervical area of ​​the cervical B-ultrasound image in the clinical image; extract image features based on the segmentation result to obtain image features; load table data, and extract the required table features from the table data; summarize the table features and the image features and input them into the multivariate regression prediction model, use the multivariate regression prediction model to predict the overall risk, and finally output the prediction result of the overall risk; the prediction result is used to reflect the probability of premature birth in pregnant women and provide auxiliary information for doctors.

[0024] Compared with the prior art, the present invention has the following beneficial effects: The spontaneous premature birth prediction system and method based on B-ultrasound images of the present invention utilize a new spontaneous premature birth prediction algorithm to obtain effective information from cervical B-ultrasound images through cervical region segmentation and image feature extraction, thereby providing auxiliary diagnosis for doctors.

[0025] The present invention proposes several new cervical B-ultrasound image features that can be used to predict spontaneous premature birth, and verifies the effectiveness of these features.

[0026] Experimental results show that the AUC of the method of the present invention in predicting spontaneous preterm birth is 0.77, which is close to the level of doctors in top hospitals. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the specific implementation methods of the present application, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0028] Figure 1 It is a structural schematic diagram of the spontaneous premature birth prediction system based on B-ultrasound images described in the present invention.

[0029] Figure 2 It is a flow chart of the method for predicting spontaneous premature birth based on B-ultrasound images described in the present invention. DETAILED DESCRIPTION

[0030] The present invention is described in more detail hereinafter to facilitate understanding of the present invention.

[0031] The present invention designs and implements a method for predicting spontaneous preterm birth based on B-ultrasound images. This method first trains a cervical segmentation model and a multiple regression prediction model, and then predicts the preterm birth risk through cervical segmentation, image feature extraction and combined with tabular data.

[0032] A. Model training 1) Training of the cervical segmentation model A total of 902 cervical B-ultrasound images and image segmentation annotations were collected in this application. Among them, 672 were used as the training set and 230 were used as the test set. The image annotation was performed using the open-source tool CVAT. This study has obtained the approval of the Medical Science Research Ethics Committee of Peking University Third Hospital (No. 2022-337-02).

[0033] This application uses the open-source model Efficient-UNet based on TensorFlow for cervical segmentation. Among them, the training process has a total of 1000 rounds, the learning rate is 0.0001, the optimizer is adopted as adaptive moment estimation, and the loss is the intersection over union. For PP-LiteSeg, FCN, DeepLabv3 and Efficient-Unet, their average intersection over union is 74.0%, 82.0%, 84.9% and 87.3% respectively. This application selects the best-performing Efficient-Unet, that is, the model with the highest segmentation effect, as the model to be implemented in the final method.

[0034] 2) Training of the multiple regression model This application uses a total of 6 image features and 10 tabular features for multiple regression with whether preterm birth occurs. The image features include the area of the cervical region, the longest diameter of the cervical region, the shortest diameter of the cervical region, the degree of cervical depression, the area of the largest black region in the cervix, and the proportion of the area of the largest black region in the cervix. The tabular features include chorionicity, assisted reproductive technology, fasting blood glucose, blood urea nitrogen, history of uterine fibroids, history of adenomyosis, cervical incompetence, inflammation (neutrophil percentage / white blood cell level), anemia (hemoglobin / hematocrit), and thrombin time.

[0035] B. Overall process of predicting spontaneous preterm birth The algorithm for predicting spontaneous preterm birth is carried out by imitating the methods used in clinical diagnosis. During clinical diagnosis, doctors obtain cervical B-ultrasound images, locate the position of the cervix in the cervical B-ultrasound images, then observe the characteristics of the cervical part based on experience, and use the tools built into the B-ultrasound machine for manual measurement. In addition to B-ultrasound examination, the patient's medical history is also inquired and various laboratory tests are conducted, and these test indicators are entered into the patient's medical record. Doctors will retrieve the indicators related to preterm birth prediction from the medical record for reference, and after summarizing the characteristics in the B-ultrasound image and the tabular data, the overall preterm birth risk is obtained.

[0036] The algorithm for predicting spontaneous preterm birth uses a deep neural network for cervical positioning, image processing for feature extraction, and summarizes the extracted features with common tabular data for overall risk prediction. For the cervical positioning part, the optimal image segmentation neural network is used to infer the position of the cervix from the image. For the feature extraction part, an automated extraction algorithm is designed based on the collected doctor's experience and knowledge. For the summarization of image features and tabular data, a multiple regression model is established for multiple indicators. This application uses multiple tabular data indicators related to preterm birth prediction, and these indicators are obtained through inquiring medical history, blood tests, and urine tests.

[0037] In current clinical practice, doctors use manually measured cervical length and tabular data for prediction. Therefore, this application uses the cervical length to represent the result of manual measurement, and the ultrasonic features extracted by the model represent the result of automatic diagnosis. The AUC values predicted by using only the cervical length, only the features extracted from the ultrasonic image, using the cervical length and tabular data, and using the ultrasonic features and tabular data are 0.704, 0.626, 0.767, and 0.769 respectively. The automatic prediction method of this application can achieve an effect equivalent to that of manual measurement by top doctors.

[0038] C. Extraction of B-ultrasound image features After segmenting the cervical region using a neural network, this application uses image processing methods to extract several features related to preterm birth from it. Specifically, the features used include three parts: the overall situation of the cervical region, the shape of the cervix, and the black area inside the cervix, for a total of six features, which are: cervical area (A), depression degree of the cervical region (I), longest diameter of the cervical region (D1), shortest diameter of the cervical region (D2), largest black area area inside the cervix (B), and proportion of the largest black area inside the cervix (Q).

[0039] This application verifies the contributions of different image features to the overall results. On the one hand, this application calculates the AUC values for predicting preterm birth using each image feature separately in a regression model, and confirms the correlation between individual features and the time of preterm birth. On the other hand, this application determines the impact of excluding each feature by calculating the AUC value after excluding one feature. This application also evaluates the effects of using these six features as a combined set and completely excluding them. It is observed that when all image features are excluded and only table data is relied on, the AUC drops to 0.721. This result is significantly lower than the predictive value obtained using ultrasound image information, highlighting the importance of ultrasound imaging in the diagnosis of spontaneous preterm birth.

[0040] Denote the polygon forming the cervical region as M, and the scale of the B-ultrasound image as s. The calculation methods of each feature are introduced below.

[0041] 1) Overall situation of the cervix As pregnancy progresses, the structure of the cervix changes. First, the overall situation of the cervical region is represented by the area of the cervix. After calculating the area of the cervical region on the image and then scaling it according to the scale of the image, the true cervical area (A) can be obtained.

[0042] where area is a function for calculating the area of a polygon.

[0043] 2) Shape of the cervix The cervix is a narrow passage between the uterus and the vagina. As pregnancy progresses, the cervix may gradually shorten in preparation for childbirth. However, compared with the situation of normal-term delivery, those who experience spontaneous preterm birth tend to show this process earlier, and at this time, the shape of the cervix changes. Compared with those who give birth at normal time, the cervix of those who have preterm birth shortens, resulting in a depression in the cervical region, and the shape of the cervix changes from a convex polygon to a concave polygon, and the parts on both sides of the cervical os look like they extend outwards. This application uses the ratio of the depression degree to the diameter of the cervical region to depict the shape of the cervix.

[0044] First, obtain the convex hull of the cervix. Let the convex hull of the cervix be H. Here, the convex hull is a general mathematical concept. The convex hull of a polygon is the smallest convex polygon that contains this polygon. For example, the convex hull of a convex polygon is itself, but the convex hull of a concave polygon is not itself. H here is the abbreviation of Hull, and H is a polygon.

[0045] H = hull(M) where hull is a function for finding the convex hull of a polygon. As a traditional mathematical concept, finding the convex hull of a polygon is a mature method in the prior art.

[0046] The degree of cervical concavity is represented by the difference (I) between the area of the cervical region and the area of the convex hull of the cervical region. On the B-ultrasound image, the convex hull of the cervical region is a two-dimensional polygon, and the area of the convex hull is the area of a polygon.

[0047] If two parallel lines can be formed from two vertices of the cervical region that pass through these two vertices respectively, such that the entire cervical region lies between these two parallel lines, then the line connecting these two vertices can form a diameter of the cervical region, and these two vertices are called antipodal points. Find all the antipodal points and the diameters they form, and use the longest diameter (D1) and the shortest diameter (D2) among them to represent the stretching condition of the cervical region.

[0048] where pairs is the set composed of antipodal points, and Distance is the function to calculate the distance between two points. 3) The black region inside the cervix Due to the principle of B-ultrasound, different tissues will present colors with different grayscales. There will be a part in the internal region of the cervix that is different from the surrounding tissues and presents a different black color from the surrounding. This black region will change with different stages of labor. The characteristics of the black region are expressed by the area (B) and the proportion (Q) of the largest black region inside the cervix. To obtain the segmentation of the black region, first find the places inside the cervical region where the gray value is greater than a certain value, and then find the continuous regions formed by these pixels. Finally, find the largest one among these regions.

[0049] where blacks represents all the black regions inside the cervical region where the gray value is greater than the predetermined threshold, M’ represents the gray value, threshold represents the predetermined threshold, findCounters is the function to find continuous regions, and blacks i represents the i-th black region inside the cervical region where the gray value is greater than the predetermined threshold, B represents the area of the largest black region inside the cervix, s represents the scale of the B-ultrasound image, Q represents the proportion of the area of the largest black region inside the cervix; A represents the actual area of the cervix.

[0050] Based on the above content, such as Figure 1and Figure 2 As shown in and

[0051] , the present invention provides a system for predicting spontaneous preterm birth based on B-ultrasound images, including a model training unit and a spontaneous preterm birth prediction unit. The model training unit is used to train the models required for predicting spontaneous preterm birth, including an image segmentation model and a multiple regression prediction model. The spontaneous preterm birth prediction unit is used to perform cervical segmentation and image feature extraction on clinical images by using the trained image segmentation model, summarize the image features and tabular data, and use the trained multiple regression prediction model to perform overall risk prediction and output the prediction result of the overall risk. The prediction result is used to reflect the probability of a pregnant woman having a preterm birth and provide auxiliary information for doctors.

[0051] Preferably, the input of the model training unit is an image data set and a tabular data set, and the output is an image segmentation model and a multiple regression prediction model.

[0052] Preferably, the image data set includes cervical B-ultrasound images and annotations of the cervical region range.

[0053] Preferably, the model training unit includes an image data loading module, an image segmentation training module, an image feature extraction module, a tabular data loading module, a regression model training module, and a model export module. The image data loading module is used to load the cervical B-ultrasound images and image segmentation annotations in the image data set. The image segmentation training module uses the open-source model Efficient-UNet based on TensorFlow and performs cervical B-ultrasound image segmentation training according to the image segmentation annotations. The training process has 1000 rounds, the learning rate is 0.0001, the optimizer is adaptive moment estimation, and the loss is intersection over union. After training is completed, an image segmentation model is obtained. The image feature extraction module is used to extract 6 image features of the cervical B-ultrasound images in the image data set. The tabular data loading module is used to load 10 tabular features in the tabular data set. The regression model training module is used to combine the tabular features in the tabular data set and the image features extracted from the cervical B-ultrasound images in the image data set as independent variables, and use whether to have a preterm birth as the dependent variable to train a multiple regression prediction model. The model export module is used to export the trained image segmentation model and multiple regression prediction model.

[0054] Preferably, the 6 image features extracted by the image feature extraction module include the area of the cervical region, the longest diameter of the cervical region, the shortest diameter of the cervical region, the degree of cervical depression, the area of the largest black region in the cervix, and the proportion of the area of the largest black region in the cervix.

[0055] Preferably, the area of the cervical region is used to represent the overall condition of the cervical region; the calculation method of the area of the cervical region is as follows: after calculating the area of the cervical region on the calculation image, it is scaled according to the scale of the image, and the true cervical area A is obtained according to the following calculation formula: where M is the polygon constituting the cervical region; s is the scale of the B-ultrasound image; area is the function for calculating the area of the polygon.

[0056] Preferably, the degree of cervical depression is represented by the difference I between the area of the cervical region and the area of the convex hull of the cervical region, and the difference I between the area of the cervical region and the area of the convex hull of the cervical region is obtained according to the following calculation formula: H = hull(M) where hull is the function for finding the convex hull of the polygon; area is the function for calculating the area of the polygon; M is the polygon constituting the cervical region; s is the scale of the B-ultrasound image.

[0057] Preferably, the longest diameter and the shortest diameter of the cervical region are used to represent the stretching condition of the cervical region, and are obtained according to the following calculation formula: where M is the polygon constituting the cervical region; m1 and m2 are the antipodal points within the polygon M constituting the cervical region; pairs is the set composed of the antipodal points m1 and m2; max_diameter is the maximum distance between the antipodal points m1 and m2; min_diameter is the minimum distance between the antipodal points m1 and m2; D1 is the longest diameter of the cervical region; D2 is the shortest diameter of the cervical region; s is the scale of the B-ultrasound image.

[0058] Preferably, the area of the largest black region in the cervix and the proportion of the area of the largest black region in the cervix are obtained according to the following calculation formula: where blacks represents all the black regions inside the cervical region with gray level greater than a predetermined threshold, M' represents the gray level, threshold represents the predetermined threshold, findCounters is a function to find continuous regions, and blacks i represents the i-th black region inside the cervical region with gray level greater than the predetermined threshold, B represents the area of the largest black region inside the cervix, s represents the scale of the B-ultrasound image, Q represents the area ratio of the largest black region inside the cervix; A represents the true cervical area.

[0059] Preferably, the 10 table features loaded by the table data loading module include chorionicity, assisted reproductive technology, fasting blood glucose, blood urea nitrogen, history of uterine fibroids, history of adenomyosis, cervical incompetence, inflammation (neutrophil percentage / white blood cell level), anemia (hemoglobin / hematocrit), and prothrombin time.

[0060] Preferably, the input of the spontaneous preterm birth prediction unit is an image segmentation model, a multiple regression prediction model, clinical images, and table data, and the output is the prediction result of the overall risk.

[0061] Preferably, the spontaneous preterm birth prediction unit includes an image loading module, a model loading module, a cervical region segmentation module, an image feature extraction module, a table data loading module, and an overall risk prediction module. The image loading module is used to load the cervical B-ultrasound image in the clinical image. The model loading module is used to load the trained image segmentation model and multiple regression prediction model. The cervical region segmentation module is used to segment the cervical region of the cervical B-ultrasound image in the clinical image loaded by the image loading module by using the trained image segmentation model. The image feature extraction module is used to extract image features based on the segmentation result to obtain image features. The table data loading module is used to load table data and extract the required table features from the table data. The overall risk prediction module is used to summarize the table features and the image features and input them into the multiple regression prediction model. The multiple regression prediction model is used to predict the overall risk, and finally output the prediction result of the overall risk. The prediction result is used to reflect the probability of a pregnant woman having a preterm birth and provide auxiliary information for doctors.

[0062] The method for predicting spontaneous preterm birth based on B-ultrasound images according to the present invention first trains a cervical segmentation model and a multiple regression prediction model, and then predicts the preterm birth risk through cervical segmentation, image feature extraction, and combination with table data.

[0063] The cervical segmentation model is an image segmentation model. In the present invention, the open-source model Efficient-UNet based on TensorFlow is used for cervical segmentation. Given a B-ultrasound image, the input of the cervical segmentation model is the pixel values of the B-ultrasound image, and the output is the coordinate positions of the polygon in the cervical region. Multiple regression prediction is a traditional machine learning method. The specific method is as follows: Given a series of variables x1, x2... xn and a prediction target y, find a set of parameters to fit x and y such that 。

[0064] During the model training process, the image dataset includes cervical B-ultrasound images and annotations of the cervical region range. An artificial intelligence method is used to train a cervical segmentation model. In addition, several features will be extracted based on the cervical region segmentation. Using these features and the features in the tabular data as independent variables, and whether preterm birth has occurred as the dependent variable, a multiple regression prediction model is trained.

[0065] For the prediction of spontaneous preterm birth, first obtain the image, then use the image segmentation model to identify the cervical region, and then perform image feature extraction. These features mathematically model the experience of clinicians and can reflect the magnitude of the risk of spontaneous preterm birth. Finally, the extracted features are aggregated with the tabular data, and the multiple regression prediction model is used for overall risk prediction.

[0066] The process of the method for predicting spontaneous preterm birth based on B-ultrasound images according to the present invention is as follows: First, train the models required for predicting spontaneous preterm birth, including the image segmentation model and the multiple regression prediction model. Then, perform the prediction of spontaneous preterm birth, perform cervical segmentation and image feature extraction on the clinical images, and aggregate the image features and tabular data to predict the overall risk.

[0067] Figure 2 specifically shows the process of the method for predicting spontaneous preterm birth based on B-ultrasound images. The whole process is divided into two parts. The first part is model training, and the second part is the prediction of spontaneous preterm birth.

[0068] For the first part (i.e., model training), first load the images, and train the image segmentation model according to the cervical region segmentation annotations in the dataset to obtain the cervical segmentation model. Then load the tabular data, combine the tabular data and the features extracted from the images as independent variables, and whether preterm birth occurs as the dependent variable, and train the multiple regression prediction model. Finally, export the model.

[0069] For the second part (i.e., the prediction of spontaneous preterm birth), first, the models are loaded to obtain the cervical segmentation model and the multiple regression prediction model. Then, the cervical B-ultrasound images are acquired, and the cervical region is segmented using the cervical segmentation model. Based on the segmentation results, image features are extracted to obtain the image features. Finally, the required tabular features are extracted from the tabular data, aggregated with the image features, and the multiple regression prediction model is used for overall risk prediction. Finally, the overall risk is output. This prediction result reflects the probability of a pregnant woman having a preterm birth, providing auxiliary information for doctors.

[0070] Generally speaking, the advantages of the present invention are as follows: 1. The present invention proposes a new algorithm for predicting spontaneous preterm birth.

[0071] 2. The present invention proposes new features for predicting spontaneous preterm birth.

[0072] 3. The present invention has an accuracy rate close to that of top doctors in the prediction of spontaneous preterm birth.

[0073] Based on the present invention, further improvement directions include: 1. For the task of predicting spontaneous preterm birth, more image features can be added. In addition to the cervical B-ultrasound image features extracted in the present invention, more features can be extracted, such as cervical length, cervical angle, etc. These features can be added to the method proposed in the present invention for the prediction of spontaneous preterm birth.

[0074] 2. For the task of predicting spontaneous preterm birth, other cervical segmentation models can also be replaced. For example, the latest image segmentation model can be used, or a dedicated segmentation model for medical images can be designed.

[0075] 3. The method of first segmenting a certain region and then extracting image features proposed in the present invention can be applied to other types of B-ultrasound images, such as abdominal B-ultrasound images. The information of multiple B-ultrasound images can also be combined for the prediction of spontaneous preterm birth.

[0076] Therefore, the key content and core technology to be protected by the present invention is the process of predicting spontaneous preterm birth. The process of predicting spontaneous preterm birth can introduce more types of images, replace the segmentation model, or introduce more features.

[0077] The preferred embodiments of the present invention have been described above, but they are not intended to limit the present invention. Those skilled in the art can make improvements and changes to the embodiments disclosed herein without departing from the scope and spirit of the present invention.

Claims

1. A spontaneous preterm birth prediction system based on B-ultrasound images, characterized in that, The described spontaneous preterm birth prediction system based on B-ultrasound images includes a model training unit and a spontaneous preterm birth prediction unit. The model training unit is used to train the models required for spontaneous preterm birth prediction, including an image segmentation model and a multiple regression prediction model. The spontaneous preterm birth prediction unit is used to perform cervical segmentation and image feature extraction on clinical images using the trained image segmentation model, summarize the image features and tabular data, and use the trained multiple regression prediction model to perform overall risk prediction and output the prediction results of the overall risk. The prediction results are used to reflect the probability of a pregnant woman having a preterm birth and provide auxiliary information for doctors.

2. The B-ultrasound image-based spontaneous preterm birth prediction system according to claim 1, wherein The input of the model training unit is an image data set and a tabular data set, and the output is an image segmentation model and a multiple regression prediction model.

3. The spontaneous preterm birth prediction system based on B-ultrasound images according to claim 2, wherein The image data set includes cervical B-ultrasound images and annotations of the cervical region range.

4. The B-ultrasound image-based spontaneous preterm birth prediction system according to claim 1, wherein The model training unit includes an image data loading module, an image segmentation training module, an image feature extraction module, a tabular data loading module, a regression model training module, and a model export module. The image data loading module is used to load the cervical B-ultrasound images and image segmentation annotations in the image data set. The image segmentation training module uses the open-source model Efficient-UNet based on TensorFlow and performs cervical B-ultrasound image segmentation training according to the image segmentation annotations. The training process has a total of 1000 rounds, the learning rate is 0.0001, the optimizer is the adaptive moment estimation, and the loss is the intersection over union. After training is completed, an image segmentation model is obtained. The image feature extraction module is used to extract 6 image features of the cervical B-ultrasound images in the image data set. The tabular data loading module is used to load 10 tabular features in the tabular data set. The regression model training module is used to combine the tabular features in the tabular data set and the image features extracted from the cervical B-ultrasound images in the image data set as independent variables, and use whether to have a preterm birth as the dependent variable to train a multiple regression prediction model. The model export module is used to export the trained image segmentation model and multiple regression prediction model.

5. The system for predicting spontaneous preterm birth based on B-ultrasound images according to claim 4, characterized in that, The 6 image features extracted by the image feature extraction module include the cervical region area, the longest diameter of the cervical region, the shortest diameter of the cervical region, the degree of cervical depression, the area of the largest black region in the cervix, and the proportion of the area of the largest black region in the cervix.

6. The B-ultrasound image-based spontaneous preterm birth prediction system according to claim 4, wherein The 10 tabular features loaded by the tabular data loading module include chorionicity, assisted reproductive technology, fasting blood glucose, blood urea nitrogen, history of uterine fibroids, history of adenomyosis, cervical incompetence, inflammation, anemia, and thrombin time.

7. The B-ultrasound image-based spontaneous preterm birth prediction system according to claim 6, wherein, The inflammation includes the percentage of neutrophils and the white blood cell level.

8. The B-ultrasound image-based spontaneous preterm birth prediction system according to claim 6, wherein, The anemia includes hemoglobin and hematocrit.

9. The spontaneous preterm birth prediction system based on B-ultrasound images according to claim 1, wherein The input of the spontaneous preterm birth prediction unit is an image segmentation model, a multiple regression prediction model, clinical images, and tabular data, and the output is the prediction result of the overall risk.

10. The spontaneous preterm birth prediction system based on B-ultrasound images according to claim 1, characterized in that, The described spontaneous preterm birth prediction unit includes an image loading module, a model loading module, a cervical region segmentation module, an image feature extraction module, a tabular data loading module, and an overall risk prediction module. The image loading module is used to load the cervical B-ultrasound image in the clinical image. The model loading module is used to load the trained image segmentation model and the multiple regression prediction model. The cervical region segmentation module is used to perform cervical region segmentation on the cervical B-ultrasound image in the clinical image loaded by the image loading module using the trained image segmentation model. The image feature extraction module is used to extract image features based on the segmentation result to obtain image features. The tabular data loading module is used to load tabular data and extract the required tabular features from the tabular data. The overall risk prediction module is used to summarize the tabular features and the image features and input them into the multiple regression prediction model. The multiple regression prediction model performs overall risk prediction and finally outputs the prediction result of the overall risk. The prediction result is used to reflect the probability of a pregnant woman having a preterm birth and provide auxiliary information for doctors.

Images