Deep learning-based ultrasonic follicle image instance segmentation method and system

Abstract

The invention discloses an ultrasonic follicle image instance segmentation method and system based on deep learning. The method comprises the following steps: collecting 3D follicle ultrasonic image data of a patient; slicing the 3D follicle ultrasonic image data, converting the 3D follicle ultrasonic image data into 2D follicle ultrasonic image data, and taking the 2D follicle ultrasonic image data as the input of the model; determining a possible follicle region in the input 2D follicle ultrasound image data, and generating a bounding box around a target by using a convolutional neural network; through operation on a feature map generated by the feature pyramid network, predicting a bounding box and a category probability of the target follicles; a multi-instance learning framework is adopted as a center method of weak supervision instance segmentation, a part containing the target follicles in a bounding box is regarded as a positive packet by using bounding box compactness prior, a part not containing the target follicles is regarded as a negative packet, and the target follicles are segmented through positive and negative packet information learning; and calculating the maximum diameter of each follicle based on the segmented images of all follicles in the data segmentation output result. According to the invention, the working efficiency of medical staff is improved, and the medical experience of a patient is optimized at the same time.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence image segmentation technology, and in particular discloses a method and system for instance segmentation of ultrasonic follicle images based on deep learning. Background Art

[0002] With the development of artificial intelligence and deep learning technology, image segmentation technology is also developing rapidly. In the field of general images, there are some instance segmentation solutions with good results. However, in the field of medical images, the performance of general image instance segmentation models still needs to be improved. Due to the small amount of data, high difficulty of professional knowledge, lack of high-quality annotation and other problems, instance segmentation solutions in the field of medical images cannot directly use general image instance segmentation models.

[0003] In the field of Assisted Reproductive Technology (ART), the ovulation stage is a crucial part. The purpose of this stage is to stimulate a woman's ovaries to produce multiple mature eggs, thereby increasing the chances of successful pregnancy. To ensure that the follicles are developing properly, doctors will regularly use ultrasound scans to monitor the size and number of follicles.

[0004] In the existing technology, the manual ultrasound image analysis method used in the clinic has the problems of high error and low efficiency. At present, most clinics rely on manual measurement of the maximum diameter of follicles when evaluating follicular development. This method is not only time-consuming, but also easy to introduce manual errors due to the subjectivity of the operator. At the same time, due to the reliance on the operator's experience, the measurement efficiency is low. These errors may affect the accurate assessment of follicular development, and thus affect the success rate of the entire assisted reproductive process.

[0005] In order to improve the accuracy and efficiency of follicle size assessment, it is particularly necessary to introduce artificial intelligence (AI) and deep learning technology. Through these advanced technologies, an automated system can be developed to process ultrasound images and automatically identify and segment the follicle area. Such a system can reduce the interference of human factors and provide more accurate measurement of the maximum diameter of the follicle. Summary of the invention

[0006] The present invention provides an instance segmentation method and system for ultrasonic follicle images based on deep learning, aiming to solve at least one of the above-mentioned defects.

[0007] One aspect of the present invention relates to an instance segmentation method of ultrasound follicle images based on deep learning, comprising the following steps:

[0008] Data collection: Collect 3D follicle ultrasound image data of patients;

[0009] Data processing: Slice the 3D follicle ultrasound image data, slice it along the z-axis direction with each pixel as the unit, and convert the 3D follicle ultrasound image data into 2D follicle ultrasound image data as the input of the model;

[0010] Data detection: Determine the possible follicle region in the input 2D follicle ultrasound image data and generate a bounding box around the target using a convolutional neural network; predict the bounding box and class probability of the target follicle by operating on the feature map generated by the feature pyramid network;

[0011] Data segmentation: The multi-instance learning framework is used as the central method of weakly supervised instance segmentation. The part of the bounding box containing the target follicle is regarded as a positive bag, and the part not containing the target follicle is regarded as a negative bag by using the bounding box compactness prior. The target follicle is segmented by learning the positive and negative bag information.

[0012] Result output: Based on the segmented images of all follicles in the data segmentation output results, the maximum diameter of each follicle is calculated.

[0013] Furthermore, in the step of data collection, an overall framework of weakly supervised follicle segmentation is adopted, and the weakly supervised follicle segmentation includes an encoder with a backbone, and a decoder with a detection branch and a segmentation branch.

[0014] Furthermore, in the step of data detection, the feature pyramid network uses the target follicle bounding box loss function to improve the detection accuracy. The target follicle bounding box loss function is expressed as:

[0015]

[0016] in, represents the overall loss of the target follicle bounding box, is the bounding box regression loss, which uses the full intersection-over-union loss and distribution focus loss to measure the degree of bounding box regression; is the classification loss, using binary cross entropy loss to classify whether the target is a follicle.

[0017] Furthermore, in the data segmentation step, b i is the set of mask probabilities of pixel instances belonging to bag i, and the multi-instance learning loss function is defined as:

[0018]

[0019] in, is the multi-instance learning loss function, y i is the judgment parameter. When package i is a positive package, y i =1; when packet i is a negative packet, y i =0, b iis the set of mask probabilities of pixel instances belonging to bag i.

[0020] Furthermore, in the data segmentation step, the detection and segmentation losses are combined to perform multi-task learning, so that the feature pyramid network learns features that are beneficial to both detection and segmentation tasks. The multi-task loss is defined as:

[0021]

[0022] in, is the multi-task loss, represents the overall loss of the target follicle bounding box, and α is a hyperparameter used to balance the contribution of MIL loss relative to detection loss; Loss function for multi-instance learning.

[0023] Another aspect of the present invention relates to an instance segmentation system for ultrasound follicle images based on deep learning, comprising:

[0024] A data collection module, used for collecting 3D follicle ultrasound image data of patients;

[0025] A data processing module is used to slice the 3D follicle ultrasound image data, slice each pixel along the z-axis direction, and convert the 3D follicle ultrasound image data into 2D follicle ultrasound image data as the input of the model;

[0026] The data detection module is used to determine the possible follicle area in the input 2D follicle ultrasound image data and generate a bounding box around the target using a convolutional neural network; by operating on the feature map generated by the feature pyramid network, the bounding box and class probability of the target follicle are predicted;

[0027] The data segmentation module is used to adopt the multi-instance learning framework as the central method of weakly supervised instance segmentation, and use the bounding box compactness prior to regard the part of the bounding box containing the target follicle as a positive bag, and the part not containing the target follicle as a negative bag, and learn to segment the target follicle through the positive and negative bag information;

[0028] The result output module is used to calculate the maximum diameter of each follicle based on the segmented images of all follicles in the data segmentation output results.

[0029] Furthermore, in the data collection module, an overall framework of weakly supervised follicle segmentation is adopted, and the weakly supervised follicle segmentation includes an encoder with a backbone, and a decoder with a detection branch and a segmentation branch.

[0030] Furthermore, in the data detection module, the feature pyramid network uses the target follicle bounding box loss function to improve the detection accuracy. The target follicle bounding box loss function is expressed as:

[0031]

[0032] in, represents the overall loss of the target follicle bounding box, is the bounding box regression loss, which uses the full intersection-over-union loss and distribution focus loss to measure the degree of bounding box regression; is the classification loss, using binary cross entropy loss to classify whether the target is a follicle.

[0033] Furthermore, in the data segmentation module, b i is the set of mask probabilities of pixel instances belonging to bag i, and the multi-instance learning loss function is defined as:

[0034]

[0035] in, is the multi-instance learning loss function, y i is the judgment parameter. When package i is a positive package, y i =1; when packet i is a negative packet, y i =0, b i is the set of mask probabilities of pixel instances belonging to bag i.

[0036] Furthermore, in the data segmentation module, the detection and segmentation losses are combined to perform multi-task learning, so that the feature pyramid network learns features that are beneficial to both detection and segmentation tasks. The multi-task loss is defined as:

[0037]

[0038] in, is the multi-task loss, represents the overall loss of the target follicle bounding box, and α is a hyperparameter used to balance the contribution of MIL loss relative to detection loss; Loss function for multi-instance learning.

[0039] The beneficial effects achieved by the present invention are:

[0040] The present invention provides an instance segmentation method and system of ultrasonic follicle images based on deep learning, which includes data collection: collecting 3D follicle ultrasonic image data of patients; data processing: slicing the 3D follicle ultrasonic image data, slicing each pixel in the z-axis direction, converting the 3D follicle ultrasonic image data into 2D follicle ultrasonic image data as the input of the model; data detection: determining the possible follicle area in the input 2D follicle ultrasonic image data, and generating a bounding box around the target by using a convolutional neural network; predicting the bounding box and category probability of the target follicle by operating on the feature map generated by the feature pyramid network; data segmentation: using a multi-instance learning framework as the central method of weakly supervised instance segmentation, using the bounding box compactness prior to regard the part containing the target follicle in the bounding box as a positive bag, and the part not containing the target follicle as a negative bag, and learning to segment the target follicle through the positive and negative bag information; result output: based on the segmented images of all follicles in the data segmentation output result, calculating the maximum diameter of each follicle. The deep learning-based instance segmentation method and system of ultrasonic follicle images provided by the present invention are based on a deep learning algorithm and adopt weakly supervised deep learning ultrasonic follicle image instance segmentation, aiming to solve the problem of extremely limited pixel-level annotation in the field of assisted reproduction; the AI ​​algorithm is used to assist in the measurement of the maximum diameter of the target follicle, which effectively reduces the errors caused by current clinical manual measurements and improves the accurate assessment of follicular development; the AI ​​algorithm is used to assist in the measurement of the maximum diameter of the target follicle, which can perform follicle measurement in batches and efficiently, thereby improving the work efficiency of medical staff and optimizing the patient's medical experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of an example segmentation method of ultrasonic follicle images based on deep learning of the present invention;

[0042] Figure 2 Schematic diagram of modeling the model of the present invention. DETAILED DESCRIPTION

[0043] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0044] like Figure 1 and Figure 2 As shown, the first embodiment of the present invention proposes an instance segmentation method of ultrasonic follicle images based on deep learning, comprising the following steps:

[0045] Step S100, data collection: collecting the patient's 3D follicle ultrasound image data.

[0046] Collect patient follicle ultrasound image data. The patient follicle ultrasound image data needed to be used mainly comes from transvaginal puncture ultrasound image data. The data sources include USOVA3D and proprietary data sets. USOVA3D is a public 3D ovarian ultrasound image dataset that contains 17 training samples and 19 test samples. The proprietary data set comes from patient follicle ultrasound images collected during clinical examinations at the hospital's assisted reproductive center. All privacy information, including patient personal information, is desensitized, and only follicle image information is retained.

[0047] Step S200, data processing: slicing the 3D follicle ultrasound image data, slicing each pixel along the z-axis direction, converting the 3D follicle ultrasound image data into 2D follicle ultrasound image data as the input of the model.

[0048] The 3D follicle ultrasound image data is sliced. Since the original data format is 3D image data, and the model used in this scheme requires the input data to be 2D images, the original 3D image data will be sliced. In order to retain the most effective information of the follicles for model training, each pixel along the z-axis direction is sliced ​​as a unit and used as the input of the model.

[0049] Step S300, data detection: determine the possible follicle area in the input 2D follicle ultrasound image data, and use a convolutional neural network to generate a bounding box around the target; predict the bounding box and category probability of the target follicle by operating on the feature map generated by the feature pyramid network.

[0050] Identify possible follicle regions and generate bounding boxes around the target using a convolutional neural network (CNN) foundation. This branch operates on feature maps generated by FPN (Feature Pyramid Networks) (specifically P3, P4, and P5 levels) to predict bounding boxes and class probabilities for target follicles.

[0051] Step S400, data segmentation: adopt the multi-instance learning framework as the central method of weakly supervised instance segmentation, use the bounding box compactness prior to regard the part of the bounding box containing the target follicle as a positive bag, and the part not containing the target follicle as a negative bag, and learn to segment the target follicle through the positive and negative bag information.

[0052] The Multi-Instance Learning (MIL) framework is adopted as the central method for weakly supervised instance segmentation, and the coarse annotations are processed using the bounding box compactness prior (BBTP). The compactness prior generates positive and negative bags based on the scan lines within each bounding box. The positive bag consists of pixels that intersect the bounding box and therefore contains at least one object pixel, while the negative bag consists of pixels that do not intersect any bounding box, indicating that no object is present.

[0053] Step S500, result output: based on the segmented images of all follicles in the data segmentation output result, the maximum diameter of each follicle is calculated.

[0054] Based on the output result of the segmentation module, that is, based on the segmented image of all follicles in the image, the maximum diameter of each follicle is calculated and provided to the clinical expert.

[0055] In the field of medical imaging, although image segmentation technology has made great progress, it still cannot fully meet clinical needs. Starting from clinical needs, this solution proposes a method for automatically segmenting follicles in ultrasound images. With only a small number of pixel-level annotation masks (≤10 cases), the segmentation result and the corresponding maximum diameter can be obtained by inputting an original image. This is the first application of image instance segmentation based on deep learning in the field of assisted reproduction. This method provides an effective solution for follicle segmentation in ultrasound images, significantly improving segmentation accuracy while reducing annotation requirements, reducing interference from human factors in the clinical environment, avoiding manual errors caused by experience differences, and optimizing the quality of assisted reproductive services and patient experience while improving the work efficiency of clinical medical staff.

[0056] Further, see Figure 1 and Figure 2 In the example segmentation method of ultrasound follicle images based on deep learning provided in this embodiment, in step S100, an overall framework of weakly supervised follicle segmentation is adopted, and the weakly supervised follicle segmentation includes an encoder with a backbone, and a decoder with a detection branch and a segmentation branch.

[0057] Weakly Supervised Follicle Segmentation (WSFS) is designed for the segmentation of follicles in ultrasound images. Weakly supervised learning techniques are used to address the challenge of limited pixel-level annotations, thereby reducing the need for manual annotation. The overall framework of WSFS, such as Figure 2 As shown, multiple components are integrated to achieve high-precision segmentation, consisting of an encoder and a decoder with a detection branch and a segmentation branch.

[0058] Backbone: WSFS uses CSPDarknet as the backbone / encoder for feature extraction, which is processed by a series of convolutional and deconvolutional layers, supplemented by residual connections and bottleneck structures to reduce the size of the network and improve performance. The C2f module is used as a basic building block to improve computational efficiency. The backbone network adds a feature pyramid network (FPN) module, which facilitates multi-scale feature representation, which is crucial for capturing follicles of different sizes and shapes.

[0059] Preferably, see Figure 1 and Figure 2 In step S300, the feature pyramid network uses the target follicle bounding box loss function to improve the detection accuracy. The target follicle bounding box loss function is expressed as:

[0060]

[0061] In formula (1), represents the overall loss of the target follicle bounding box, is the bounding box regression loss, which uses the complete intersection over union loss (CIoU) loss and distribution focus loss to measure the degree of bounding box regression. Combining the anchor-free method and improving the generalization ability, it allows the network to quickly focus on the distribution of the target location and its surrounding area. is the classification loss, using Binary CrossEntropy Loss (BCE) to classify whether the target is a follicle. After prediction, Non-Maximum Suppression (NMS) is applied to eliminate redundant bounding boxes and produce a set of optimized detection results.

[0062] Further, see Figure 1 and Figure 2 In the example segmentation method of ultrasound follicle images based on deep learning provided in this embodiment, in step S400, record b i is the set of mask probabilities of pixel instances belonging to bag i, and the multi-instance learning loss function is defined as:

[0063]

[0064] In formula (2), is the multi-instance learning loss function, y i is the judgment parameter. When package i is a positive package, y i =1; when packet i is a negative packet, y i =0, b i is the set of mask probabilities of pixel instances belonging to bag i.

[0065] Preferably, see Figure 1 and Figure 2 In the example segmentation method of ultrasound follicle images based on deep learning provided in this embodiment, in step S400, detection and segmentation losses are combined to perform multi-task learning, so that the feature pyramid network learns features that are beneficial to both detection and segmentation tasks. The multi-task loss is defined as:

[0066]

[0067] In formula (3), is the multi-task loss, represents the overall loss of the target follicle bounding box, and α is a hyperparameter used to balance the contribution of MIL loss relative to detection loss. Due to the high noise and scarcity of follicle ultrasound images, α is empirically set to 30. Loss function for multi-instance learning.

[0068] The instance segmentation method of ultrasound follicle images based on deep learning provided in this embodiment constructs an instance segmentation model of ultrasound follicle images based on a weakly supervised deep learning method, which can automatically segment all target follicles in ultrasound images. By inputting an ultrasound follicle image, all individual follicles in the image are segmented and their maximum diameters are calculated. In actual application scenarios, there is no need for medical experts to manually mark and measure, which improves the accuracy of follicle development assessment and clinical work efficiency.

[0069] Please see Figure 1 and Figure 2 The present invention also relates to an instance segmentation system of ultrasound follicle images based on deep learning, including a data collection module, a data processing module, a data detection module, a data segmentation module and a result output module, wherein the data collection module is used to collect 3D follicle ultrasound image data of patients; the data processing module is used to slice the 3D follicle ultrasound image data, slice each pixel in the z-axis direction, and convert the 3D follicle ultrasound image data into 2D follicle ultrasound image data as the input of the model. The data detection module is used to determine the possible follicle area in the input 2D follicle ultrasound image data, and generate a bounding box around the target using a convolutional neural network; by operating on the feature map generated by the feature pyramid network, the bounding box and category probability of the target follicle are predicted. The data segmentation module is used to use a multi-instance learning framework as the central method of weakly supervised instance segmentation, and use the bounding box tightness prior to regard the part of the bounding box containing the target follicle as a positive bag, and the part not containing the target follicle as a negative bag, and learn to segment the target follicle through the positive and negative bag information. The result output module is used to calculate the maximum diameter of each follicle based on the segmented images of all follicles in the data segmentation output results.

[0070] The data collection module collects the patient's follicle ultrasound image data. The patient's follicle ultrasound image data that needs to be used is mainly transvaginal puncture ultrasound image data. The data sources include USOVA3D and proprietary data sets. USOVA3D is a public 3D ovarian ultrasound image dataset that contains 17 training samples and 19 test samples. The proprietary data set comes from the patient's follicle ultrasound images collected during the clinical examination of the hospital's assisted reproductive center. All privacy information including patient personal information is desensitized, and only the follicle image information is retained.

[0071] The data processing module slices the 3D follicle ultrasound image data. Since the original data format is 3D image data, and the model used in this scheme requires the input data to be 2D images, the original 3D image data will be sliced. In order to retain the most effective information of follicles for model training, each pixel along the z-axis direction is used as a unit slice as the input of the model.

[0072] The data detection module determines the possible follicle area and generates a bounding box around the target using a convolutional neural network (CNN) foundation. This branch operates on the feature map generated by FPN (Feature Pyramid Networks) (specifically P3, P4, and P5 levels) to predict the bounding box and class probability of the target follicle.

[0073] The data segmentation module adopts the Multi-Instance Learning (MIL) framework as the central method for weakly supervised instance segmentation, and uses the bounding box tightness prior (BBTP) to process coarse annotations. The tightness prior generates positive and negative bags based on the scan lines within each bounding box. The positive bag consists of pixels that intersect the bounding box and therefore contains at least one object pixel, while the negative bag consists of pixels that do not intersect any bounding box, indicating that no object exists.

[0074] The result output module is based on the output result of the segmentation module, that is, according to the segmented image of all follicles in the image, calculates the maximum diameter of each follicle and provides it to the clinical expert.

[0075] Further, see Figure 1 and Figure 2 In the example segmentation system of ultrasound follicle images based on deep learning provided in this embodiment, an overall framework of weakly supervised follicle segmentation is adopted in the data collection module. The weakly supervised follicle segmentation includes an encoder with a backbone and a decoder with a detection branch and a segmentation branch.

[0076] Weakly Supervised Follicle Segmentation (WSFS) is designed for the segmentation of follicles in ultrasound images. Weakly supervised learning techniques are used to address the challenge of limited pixel-level annotations, thereby reducing the need for manual annotation. The overall framework of WSFS, such as Figure 2 As shown, multiple components are integrated to achieve high-precision segmentation, consisting of an encoder and a decoder with a detection branch and a segmentation branch.

[0077] Backbone: WSFS uses CSPDarknet as the backbone / encoder for feature extraction, which is processed by a series of convolutional and deconvolutional layers, supplemented by residual connections and bottleneck structures to reduce the size of the network and improve performance. The C2f module is used as a basic building block to improve computational efficiency. The backbone network adds a feature pyramid network (FPN) module, which facilitates multi-scale feature representation, which is crucial for capturing follicles of different sizes and shapes.

[0078] Further, see Figure 1 and Figure 2 In the example segmentation system of ultrasound follicle images based on deep learning provided in this embodiment, in the data detection module, the feature pyramid network uses the target follicle bounding box loss function to improve the detection accuracy. The target follicle bounding box loss function is expressed as:

[0079]

[0080] In formula (4), represents the overall loss of the target follicle bounding box, is the bounding box regression loss, which uses the complete intersection over union loss (CIoU) loss and distribution focus loss to measure the degree of bounding box regression. Combining the anchor-free method and improving the generalization ability, it allows the network to quickly focus on the distribution of the target location and its surrounding area. is the classification loss, using Binary CrossEntropy Loss (BCE) to classify whether the target is a follicle. After prediction, Non-Maximum Suppression (NMS) is applied to eliminate redundant bounding boxes and produce a set of optimized detection results.

[0081] Further, see Figure 1 and Figure 2 In the example segmentation system of ultrasound follicle images based on deep learning provided in this embodiment, in the data segmentation module, record b iis the set of mask probabilities of pixel instances belonging to bag i, and the multi-instance learning loss function is defined as:

[0082]

[0083]

[0084] In formula (5), is the multi-instance learning loss function, y i is the judgment parameter. When package i is a positive package, y i =1; when packet i is a negative packet, y i =0, b i is the set of mask probabilities of pixel instances belonging to bag i.

[0085] Preferably, see Figure 1 and Figure 2 In the example segmentation system of ultrasound follicle images based on deep learning provided in this embodiment, in the data segmentation module, detection and segmentation losses are combined to perform multi-task learning, so that the feature pyramid network learns features that are beneficial to both detection and segmentation tasks. The multi-task loss is defined as:

[0086]

[0087] In formula (6), is the multi-task loss, represents the overall loss of the target follicle bounding box, and α is a hyperparameter used to balance the contribution of MIL loss relative to detection loss. Due to the high noise and scarcity of follicle ultrasound images, α is empirically set to 30. Loss function for multi-instance learning.

[0088] Compared with the prior art, the instance segmentation method and system of ultrasound follicle images based on deep learning provided in this embodiment include data collection: collecting 3D follicle ultrasound image data of patients; data processing: slicing the 3D follicle ultrasound image data, slicing each pixel along the z-axis direction, and converting the 3D follicle ultrasound image data into 2D follicle ultrasound image data as the input of the model; data detection: determining the possible follicle area in the input 2D follicle ultrasound image data, and generating a bounding box around the target using a convolutional neural network; predicting the bounding box and category probability of the target follicle by operating on the feature map generated by the feature pyramid network; data segmentation: using a multi-instance learning framework as the central method of weakly supervised instance segmentation, using the bounding box tightness prior to regard the part of the bounding box containing the target follicle as a positive package, and the part not containing the target follicle as a negative package, and learning to segment the target follicle through the positive and negative package information; result output: based on the segmented images of all follicles in the data segmentation output result, calculating the maximum diameter of each follicle. The deep learning-based ultrasound follicle image instance segmentation method and system provided in this embodiment are based on a deep learning algorithm and adopt weakly supervised deep learning ultrasound follicle image instance segmentation, aiming to solve the problem of extremely limited pixel-level annotation in the field of assisted reproduction; the AI ​​algorithm is used to assist in the measurement of the maximum diameter of the target follicle, which effectively reduces the errors caused by current clinical manual measurements and improves the accurate assessment of follicle development; the AI ​​algorithm is used to assist in the measurement of the maximum diameter of the target follicle, which can perform follicle measurement in batches and efficiently, improves the work efficiency of medical staff, and also optimizes the patient's medical experience.

[0089] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An instance segmentation method for ultrasound follicle images based on deep learning, It is characterized in that The following steps are involved: Data collection: Collect 3D follicle ultrasound image data of patients; Data processing: slicing the 3D follicle ultrasound image data, slicing each pixel along the z-axis direction, and converting the 3D follicle ultrasound image data into 2D follicle ultrasound image data as the input of the model; Data detection: Determine the possible follicle area in the input 2D follicle ultrasound image data, and use a convolutional neural network to generate a bounding box around the target; predict the bounding box and category probability of the target follicle by operating on the feature map generated by the feature pyramid network; Data segmentation: The multi-instance learning framework is used as the central method of weakly supervised instance segmentation. The part of the bounding box containing the target follicle is regarded as a positive bag, and the part not containing the target follicle is regarded as a negative bag by using the bounding box compactness prior. The target follicle is segmented by learning the positive and negative bag information. Result output: Based on the segmented images of all follicles in the data segmentation output results, the maximum diameter of each follicle is calculated.

2. The instance segmentation method of ultrasound follicle images based on deep learning as claimed in claim 1, It is characterized in that In the data collection step, an overall framework of weakly supervised follicle segmentation is adopted, and the weakly supervised follicle segmentation includes an encoder with a trunk, and a decoder with a detection branch and a segmentation branch.

3. The instance segmentation method of ultrasound follicle images based on deep learning as claimed in claim 1, It is characterized in that In the step of data detection, the feature pyramid network uses a target follicle bounding box loss function to improve detection accuracy. The target follicle bounding box loss function is expressed as: in, represents the overall loss of the target follicle bounding box, is the bounding box regression loss, which uses the full intersection-over-union loss and distribution focus loss to measure the degree of bounding box regression; is the classification loss, using binary cross entropy loss to classify whether the target is a follicle.

4. The instance segmentation method of ultrasound follicle images based on deep learning as claimed in claim 3, It is characterized in that In the data segmentation step, b i is the set of mask probabilities of pixel instances belonging to bag i, and the multi-instance learning loss function is defined as: in, is the multi-instance learning loss function, y i is the judgment parameter. When package i is a positive package, y i =1; when packet i is a negative packet, y i =0, b i is the set of mask probabilities of pixel instances belonging to bag i.

5. The instance segmentation method of ultrasound follicle images based on deep learning as claimed in claim 4, It is characterized in that In the data segmentation step, detection and segmentation losses are combined to perform multi-task learning, so that the feature pyramid network learns features that are beneficial to both detection and segmentation tasks. The multi-task loss is defined as: in, is the multi-task loss, represents the overall loss of the target follicle bounding box, and α is a hyperparameter used to balance the contribution of MIL loss relative to detection loss; Loss function for multi-instance learning.

6. An instance segmentation system for ultrasound follicle images based on deep learning, It is characterized in that include: A data collection module, used for collecting 3D follicle ultrasound image data of patients; A data processing module, used for slicing the 3D follicle ultrasound image data, slicing each pixel along the z-axis direction, and converting the 3D follicle ultrasound image data into 2D follicle ultrasound image data as the input of the model; A data detection module is used to determine the possible follicle area in the input 2D follicle ultrasound image data, and generate a bounding box around the target using a convolutional neural network; by operating on the feature map generated by the feature pyramid network, predict the bounding box and category probability of the target follicle; The data segmentation module is used to adopt the multi-instance learning framework as the central method of weakly supervised instance segmentation, and use the bounding box compactness prior to regard the part of the bounding box containing the target follicle as a positive bag, and the part not containing the target follicle as a negative bag, and learn to segment the target follicle through the positive and negative bag information; The result output module is used to calculate the maximum diameter of each follicle based on the segmented images of all follicles in the data segmentation output results.

7. The instance segmentation system of ultrasound follicle images based on deep learning as claimed in claim 6, It is characterized in that In the data collection module, an overall framework of weakly supervised follicle segmentation is adopted, and the weakly supervised follicle segmentation includes an encoder with a trunk, and a decoder with a detection branch and a segmentation branch.

8. The instance segmentation system of ultrasound follicle images based on deep learning as claimed in claim 6, It is characterized in that In the data detection module, the feature pyramid network uses a target follicle bounding box loss function to improve detection accuracy. The target follicle bounding box loss function is expressed as: in, represents the overall loss of the target follicle bounding box, is the bounding box regression loss, which uses the full intersection-over-union loss and distribution focus loss to measure the degree of bounding box regression; is the classification loss, using binary cross entropy loss to classify whether the target is a follicle.

9. The instance segmentation system of ultrasound follicle images based on deep learning as claimed in claim 8, It is characterized in that In the data segmentation module, b i is the set of mask probabilities of pixel instances belonging to bag i, and the multi-instance learning loss function is defined as: in, is the multi-instance learning loss function, y i is the judgment parameter. When package i is a positive package, y i =1; when packet i is a negative packet, y i =0, b i is the set of mask probabilities of pixel instances belonging to bag i.

10. The instance segmentation system of ultrasound follicle images based on deep learning as claimed in claim 9, It is characterized in that In the data segmentation module, detection and segmentation losses are combined to perform multi-task learning, so that the feature pyramid network learns features that are beneficial to both detection and segmentation tasks. The multi-task loss is defined as: in, is the multi-task loss, represents the overall loss of the target follicle bounding box, and α is a hyperparameter used to balance the contribution of MIL loss relative to detection loss; Loss function for multi-instance learning.

Images