Amniotic fluid depth measurement method and apparatus, computer device, and storage medium
By automatically screening and measuring amniotic fluid sections using computer vision and deep learning technologies, the problem of low efficiency and large error in amniotic fluid depth measurement in traditional prenatal ultrasound examinations has been solved, achieving efficient and accurate amniotic fluid depth measurement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUNAN UNIV
- Filing Date
- 2023-11-23
- Publication Date
- 2026-06-23
AI Technical Summary
Traditional prenatal ultrasound examinations rely on manual operation to measure amniotic fluid depth, which is inefficient and easily affected by subjective errors.
Using computer vision and deep learning technologies, fetal ultrasound video frames are processed through semantic segmentation and section classification models to automatically select suitable amniotic fluid sections for measurement and measure amniotic fluid depth.
It improves the efficiency and accuracy of amniotic fluid depth measurement, and reduces human intervention and subjective error.
Smart Images
Figure CN117653206B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of ultrasound examination technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for measuring amniotic fluid depth. Background Technology
[0002] With the development of ultrasound examination technology, ultrasound has become a common, routine, and indispensable imaging examination method. Due to the special nature of pregnancy, common radiation-based examination methods are not suitable for fetal examination, while ultrasound diagnosis is safe and radiation-free, and can penetrate many opaque objects. Therefore, ultrasound waves can be used to obtain information about the internal structure and acoustic properties of these objects. For example, ultrasound imaging technology transforms this information into images visible to the human eye, making ultrasound examination the preferred imaging examination method for prenatal screening of birth defects.
[0003] Traditional prenatal ultrasound examinations rely primarily on manual observation and interpretation of ultrasound images, requiring extensive experience and expertise from the ultrasound physician, and exhibiting a low degree of standardization. Taking amniotic fluid depth measurement as an example, the acquired amniotic fluid ultrasound data is typically a real-time dynamic image. The ultrasound physician needs to determine the optimal amniotic fluid cross-sectional video frame for measurement. After determining the optimal cross-section, the physician must operate the ultrasound equipment to draw the maximum vertical depth measurement line to measure the vertical depth of the amniotic fluid.
[0004] However, the above-mentioned traditional prenatal ultrasound examination procedures are relatively cumbersome and time-consuming, rely heavily on the experience and skills of the ultrasound physician, and may also carry the risk of subjective judgment and diagnostic errors. In other words, the traditional method of measuring amniotic fluid depth is inefficient. Summary of the Invention
[0005] Therefore, it is necessary to provide an amniotic fluid depth measurement method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the efficiency of amniotic fluid depth measurement in response to the above-mentioned technical problems.
[0006] Firstly, this application provides a method for measuring amniotic fluid depth. The method includes:
[0007] Acquire fetal ultrasound video frames;
[0008] The fetal ultrasound video frames are classified by section, and the initial video frames containing standard amniotic fluid sections with a confidence level higher than a preset confidence threshold are selected.
[0009] Target region detection is performed on the initial video frame to locate the amniotic fluid section measurement area in the initial video frame, thus obtaining the target video frame;
[0010] The amniotic fluid depth measurement value is determined based on the amniotic fluid cross-sectional measurement area in the target video frame.
[0011] In one embodiment, determining the amniotic fluid depth measurement value based on the amniotic fluid cross-sectional measurement region in the target video frame includes:
[0012] Using the target video frame as input, the trained semantic segmentation model is called to perform binarization processing on the target video frame to obtain a binary image of the target video frame. The trained semantic segmentation model is trained based on the historical amniotic fluid section measurement region carrying binary labels.
[0013] Based on the binary image, the amniotic fluid depth measurement value is determined.
[0014] In one embodiment, the step of using the target video frame as input and calling a trained semantic segmentation model to binarize the target video frame to obtain a binary image of the target video frame includes:
[0015] Perform depthwise separable convolution processing on the target video frame to extract the features of the target video frame and obtain the feature map of the target video frame;
[0016] The feature map of the target video frame is deconvolved to obtain the target feature map;
[0017] A binary label is assigned to each pixel of the target feature map to obtain a binary image of the target video frame.
[0018] In one embodiment, determining the amniotic fluid depth measurement based on the binary image includes:
[0019] The pixel values of each column of pixels in the binary image are subtracted one by one, and the boundary pixels of each column of pixels in the binary image are determined based on the pixel value differences.
[0020] For each column of pixels in the binary image, obtain the pixel distance of the boundary pixels;
[0021] The maximum pixel distance in the binary image is determined as the amniotic fluid depth measurement value.
[0022] In one embodiment, determining the maximum pixel distance in the binary image as the amniotic fluid depth measurement further includes:
[0023] If the binary image representation has multiple amniotic fluid measurement regions, obtain the maximum pixel distance of each amniotic fluid measurement region;
[0024] The average of the maximum pixel distances in each amniotic fluid measurement area is determined as the amniotic fluid depth measurement value.
[0025] In one embodiment, the step of performing section classification on the fetal ultrasound video frames to select initial video frames containing standard amniotic fluid sections with a confidence level higher than a preset confidence threshold includes:
[0026] Using the fetal ultrasound video frame as input, the trained section classification model is called to perform section classification on the fetal ultrasound video frame, and video frames carrying amniotic fluid section labels are selected.
[0027] Determine the confidence level of the video frame;
[0028] From the video frames carrying amniotic fluid section labels, select initial video frames containing standard amniotic fluid sections with a confidence level higher than a preset confidence threshold;
[0029] The trained section classification model is trained based on historical fetal ultrasound video frames carrying section classification labels.
[0030] In one embodiment, the trained section classification model includes a feature extractor and a feature mapper;
[0031] The step of using the fetal ultrasound video frames as input, calling a trained section classification model to perform section classification on the fetal ultrasound video frames, and filtering out video frames carrying amniotic fluid section labels includes:
[0032] Using the fetal ultrasound video frame as input, the feature extractor is invoked to extract features from the fetal ultrasound video frame to obtain the category feature data of the fetal ultrasound video frame;
[0033] Using the category feature data as input, the feature mapper is invoked to map the category feature data to preset category labels, and video frames carrying amniotic fluid section labels are filtered out.
[0034] Secondly, this application also provides an amniotic fluid depth measuring device. The device includes:
[0035] The data acquisition module is used to acquire fetal ultrasound video frames;
[0036] The section classification module is used to classify the fetal ultrasound video frames into sections and filter out the initial video frames that contain standard amniotic fluid sections with a confidence level higher than a preset confidence threshold.
[0037] The region detection module is used to perform target region detection on the initial video frame, locate the amniotic fluid section measurement region in the initial video frame, and obtain the target video frame.
[0038] The depth measurement module is used to determine the amniotic fluid depth measurement value based on the amniotic fluid cross-sectional measurement area in the target video frame.
[0039] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.
[0040] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described above.
[0041] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described above.
[0042] The aforementioned method, apparatus, computer equipment, storage medium, and computer program product for measuring amniotic fluid depth acquire fetal ultrasound video frames, classify these frames into sections, and select initial video frames containing standard amniotic fluid sections with a confidence level higher than a preset confidence threshold. This excludes video frames that do not contain amniotic fluid sections or those that, while containing sections, are unsuitable for measuring amniotic fluid depth. Then, target region detection is performed on the initial video frames to locate the amniotic fluid section measurement area, obtaining the target video frame. Based on the amniotic fluid section measurement area in the target video frame, the amniotic fluid depth measurement value is determined. This allows for rapid and accurate selection of the optimal amniotic fluid section video frame from ultrasound video stream data for amniotic fluid measurement, thus improving the efficiency of amniotic fluid depth measurement. Attached Figure Description
[0043] Figure 1 This is a diagram illustrating the application environment of an amniotic fluid depth measurement method in one embodiment.
[0044] Figure 2 This is a flowchart illustrating a method for measuring amniotic fluid depth in one embodiment;
[0045] Figure 3 This is a schematic diagram of the process for determining amniotic fluid depth measurements in one embodiment.
[0046] Figure 4 This is a schematic diagram of the process for determining amniotic fluid depth measurements in one embodiment.
[0047] Figure 5 This is a flowchart illustrating the process of determining amniotic fluid depth measurements in another embodiment;
[0048] Figure 6 This is a schematic diagram of the process of segmenting video frames in one embodiment;
[0049] Figure 7This is a flowchart illustrating the process of segmenting video frames in another embodiment;
[0050] Figure 8 This is a structural block diagram of an amniotic fluid depth measuring device in one embodiment;
[0051] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0053] The amniotic fluid depth measurement method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 acquires fetal ultrasound video frames and sends them to server 104. Server 104 performs cross-sectional classification on the fetal ultrasound video frames, filtering out initial video frames containing standard amniotic fluid cross-sections with a confidence level higher than a preset confidence threshold. Then, server 104 performs target region detection on the initial video frames, locating the amniotic fluid cross-section measurement area in the target video frame, obtaining the target video frame, and determining the amniotic fluid depth measurement value based on the amniotic fluid cross-section measurement area in the target video frame. Terminal 102 can be, but is not limited to, various models of ultrasound examination instruments. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0054] In one embodiment, such as Figure 2 As shown, a method for measuring amniotic fluid depth is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included:
[0055] S200, acquires fetal ultrasound video frames.
[0056] Fetal ultrasound video frames are obtained by preprocessing the raw fetal ultrasound video stream. Ultrasound doctors use ultrasound acquisition devices such as ultrasound machines or probes to scan the fetus and view real-time ultrasound images. In addition, ultrasound devices usually have video recording capabilities, allowing ultrasound doctors to save the raw fetal ultrasound video stream obtained throughout the entire ultrasound acquisition process.
[0057] After obtaining the raw fetal ultrasound video stream, the ultrasound equipment can automatically upload the stream to the server, or the ultrasound physician can upload it manually, followed by subsequent processing. Upon receiving the raw fetal ultrasound video stream, the server first preprocesses it to extract frame images and remove redundant information and noise, resulting in fetal ultrasound video frames. For example, frame images can be extracted from the raw fetal ultrasound video stream, and cropping can be performed as needed to remove redundant information such as menu bars and borders, allowing subsequent processing to focus only on the ultrasound image area and saving resources. Then, denoising techniques, such as smoothing filters or wavelet denoising, are applied to reduce noise in the frame images and improve image quality, resulting in fetal ultrasound video frames. Furthermore, brightness and contrast adjustments can be made to the frame images to make them clearer, and spatial filtering techniques can be used to remove subtle noise and enhance image quality.
[0058] S400 performs section classification on fetal ultrasound video frames and filters out initial video frames containing standard amniotic fluid sections with a confidence level higher than a preset confidence threshold.
[0059] Fetal ultrasound video frames typically include multiple structures and regions, such as the fetal head and limbs. Segmentation classification of these frames can filter out those that do not contain amniotic fluid sections, facilitating further analysis. Furthermore, the classification results can be further filtered for confidence levels. Using a preset confidence threshold, high-confidence initial video frames containing amniotic fluid sections can be selected. High confidence indicates that the amniotic fluid sections contained in the selected initial video frames are more suitable for subsequent amniotic fluid depth measurement.
[0060] For example, traditional image processing methods, such as edge detection and contour extraction, can be used to identify and extract the edge contours of various tissues in fetal ultrasound video frames. Then, for the specific shape of the amniotic fluid section, specific image processing techniques or set rules are used to detect the edge contours contained in each video frame to filter out video frames containing the edge contours of the amniotic fluid section. Alternatively, machine learning, deep learning models, and other methods can be used to classify the fetal ultrasound video frames into sections, and combined with confidence calculation, to filter out initial video frames containing standard amniotic fluid sections with a confidence level higher than a preset confidence threshold.
[0061] S600 performs target region detection on the initial video frame, locates the amniotic fluid section measurement area in the initial video frame, and obtains the target video frame.
[0062] Target region detection refers to the accurate location or labeling of a target region of interest in an image or video. Performing target region detection on an initial video frame that already contains a standard amniotic fluid cross-section can locate the amniotic fluid cross-section measurement area within the initial video frame, providing more accurate and focused image feature information for subsequent amniotic fluid depth measurement. In actual measurements, an initial video frame may require measurement in more than one location, potentially requiring measurement in all four quadrants. Therefore, it is necessary to detect the measurement region to delineate the specific measurement area.
[0063] For example, image processing and computer vision techniques can be combined to perform target region detection on an initial video frame that already contains a standard amniotic fluid cross-section. First, edge detection and contour extraction are performed on the initial video frame to extract regions that may contain the amniotic fluid cross-section, and then image texture or key point features of these regions are extracted. Then, template matching or a classifier based on relevant features is used to detect these regions. After obtaining the region detection results, appropriate post-processing methods, such as non-maximum suppression, are applied to remove redundant regions, ultimately obtaining the target video frame. Alternatively, deep learning techniques can be combined to perform target region detection on the initial video frame to improve the accuracy of the above feature extraction process. For example, the initial video frame can be fed into a region detection model trained using deep learning methods, such as the YOLOv8 (Unified Real-Time Object Detection) model, to perform target region detection. The output is the bounding box coordinates of the amniotic fluid cross-section measurement region in the current initial video frame, thus obtaining the target video frame.
[0064] S800 determines the amniotic fluid depth measurement value based on the amniotic fluid cross-section measurement area in the target video frame.
[0065] After obtaining the amniotic fluid cross-sectional measurement area, the amniotic fluid depth can be estimated accordingly. For example, the target video frame containing the amniotic fluid cross-sectional measurement area can be processed into a grayscale image. When measuring amniotic fluid depth, the pixel values of the grayscale image can be analyzed, for example, using a gradient algorithm to analyze the grayscale value changes of each column of pixels to locate the edge of the amniotic fluid cross-section, thereby estimating the amniotic fluid depth. Alternatively, a more precise edge detection algorithm can be used to locate the edge of the amniotic fluid cross-section, and then the amniotic fluid depth can be estimated along the edge of the amniotic fluid region. In this embodiment, the measured amniotic fluid depth is merely a measurement value, not a diagnostic result, but rather an auxiliary means to assist the doctor in subsequent medical judgment. The ultrasound physician can provide a diagnostic result based on the amniotic fluid depth and the actual clinical situation.
[0066] The aforementioned method for measuring amniotic fluid depth involves acquiring fetal ultrasound video frames and classifying them into sections. Initial video frames containing standard amniotic fluid sections with a confidence level higher than a preset confidence threshold are selected, while frames that do not contain amniotic fluid sections or are unsuitable for measuring amniotic fluid depth are discarded. Then, target region detection is performed on the initial video frames to locate the amniotic fluid section measurement area, resulting in target video frames. Based on the amniotic fluid section measurement area in the target video frames, the amniotic fluid depth measurement value is determined. This method allows for rapid and accurate selection of the optimal amniotic fluid section video frames from the ultrasound video stream data, improving the efficiency of amniotic fluid depth measurement.
[0067] In one embodiment, such as Figure 3 As shown, S800 includes:
[0068] S820 takes the target video frame as input, calls the trained semantic segmentation model to binarize the target video frame, and obtains the binary image of the target video frame. The trained semantic segmentation model is trained based on the historical amniotic fluid cross-section measurement region carrying binary labels.
[0069] The semantic segmentation model combines image classification, object detection, and image segmentation. It segments an image into semantically meaningful regions and identifies the semantic category of each region, achieving a semantic reasoning process from lower to higher levels, ultimately resulting in a segmented image with pixel-by-pixel semantic annotations. In this embodiment, the trained semantic segmentation model is used to binarize the target video frame, assigning a corresponding binary label to each pixel in the target video frame, thereby obtaining a binary image of the target video frame. The trained semantic segmentation model is based on historical amniotic fluid cross-sectional measurement regions carrying binary labels.
[0070] For example, the training data for the semantic segmentation model used in this embodiment can be amniotic fluid section measurement region data acquired by different resolution ultrasound devices from multiple hospitals. Furthermore, the amniotic fluid section measurement region data needs to be manually or automatically labeled with binary labels, that is, to mark whether each pixel belongs to the amniotic fluid section measurement region. For example, the pixel value of the amniotic fluid section measurement region is labeled as 255, and the pixel value of the non-amniotic fluid section measurement region is labeled as 0. The target video frame already contains the preliminarily located amniotic fluid measurement region, for example, it can be marked using bounding boxes, carrying the coordinates of the amniotic fluid bounding boxes.
[0071] Using the target video frame as input, a trained semantic segmentation model is invoked to binarize the target video frame. This model has already learned during training to distinguish the amniotic fluid cross-section measurement region from other regions in the image. The target video frame can undergo preprocessing operations such as denoising and image resizing before being input into the trained semantic segmentation model. For example, removing non-amniotic fluid cross-section measurement regions outside the amniotic fluid bounding box provides more accurate feature information for the semantic segmentation model. The trained semantic segmentation model assigns a binary label to each pixel of the target video frame containing the amniotic fluid cross-section measurement region to indicate whether the pixel belongs to the amniotic fluid cross-section measurement region. Based on these binary labels, the target video frame is binarized to obtain a binary image of the target video frame. The binary image of the target video frame can be used to represent the amniotic fluid measurement region identified by the semantic segmentation model, and this amniotic fluid measurement region can be used for subsequent amniotic fluid depth calculation.
[0072] S840, determine the amniotic fluid depth measurement value based on the binary image.
[0073] For the binary image output by the trained semantic segmentation model, preprocessing operations such as denoising and smoothing can be performed before depth measurement. The measurement method can be related to the binary label carried by each pixel in the target video frame. For example, if the amniotic fluid section measurement area in the target video frame is labeled as 1 and the non-amniotic fluid section measurement area is labeled as 0, the pixel values of each column of pixels in the binary image can be added together to obtain the sum of the column pixel values. The maximum sum of the column pixel values in the binary image can be used as the amniotic fluid depth measurement value. If the amniotic fluid section measurement area in the target video frame is labeled as 255 and the non-amniotic fluid section measurement area is labeled as 0, the amniotic fluid depth measurement value can be estimated by determining edge pixels, for example, finding the pixel whose sum of adjacent pixel values is 255.
[0074] In this embodiment, a trained semantic segmentation model is invoked to binarize the target video frame, resulting in a binary image of the target video frame. Based on the binary image, the amniotic fluid depth measurement value is determined. By using deep learning technology to measure the amniotic fluid cross-sectional depth, the measurement accuracy of the amniotic fluid depth can be effectively guaranteed, while improving the measurement efficiency.
[0075] In one embodiment, such as Figure 4 As shown, S820 includes:
[0076] S822 performs depthwise separable convolution on the target video frame to extract its features and obtain its feature map.
[0077] Depthwise separable convolution is an operation in convolutional neural networks that, compared to ordinary convolution, effectively reduces operational complexity while maintaining the extraction of effective image features. In this embodiment, using depthwise separable convolution reduces computational complexity and improves feature extraction efficiency when extracting features from the target video frame, thereby improving the efficiency of amniotic fluid depth measurement.
[0078] For example, depthwise separable convolution operations include depthwise convolution and pointwise convolution. First, depthwise convolution is performed on the target video frame, meaning convolution is performed independently on each channel, extracting local features in each channel. Then, pointwise convolution is performed on the target video frame, using a 1x1 convolution kernel to map the output of the depthwise convolution to a new feature space, combining the features generated by the depthwise convolution to form a higher-level abstract feature representation. Taking the MobileNetV2 (Lightweight Neural Network) deep learning model as an example, the model needs to be pre-trained. Pre-training data can be ultrasound images and related feature labels provided by multiple hospitals, allowing the model to learn a variety of rich features. MobileNetV2 introduces depthwise separable convolution with a more lightweight parameter count, thus effectively capturing feature information in the target video frame and the feature relationships between different channels without increasing computational cost, and outputting a feature map of the target video frame, improving the efficiency of the feature extraction process.
[0079] S824 performs deconvolution on the feature map of the target video frame to obtain the target feature map. After obtaining the feature map of the target video frame, since the feature map of the target video frame is smaller than the input target video frame, and the subsequent semantic segmentation stage requires restoring the feature map of the target video frame to its original size for further calculation, it is also necessary to scale the feature map of the target video frame to achieve the mapping of the image from the current size to the target size. Deep learning models such as MobileUnet (semantic segmentation network) add multiple deconvolution layers. The deconvolution layers upsample the feature map of the target video frame, achieving the mapping of the image from small resolution to large resolution. At the same time, they map each element in the feature map of the target video frame back to the pixel space of the target video frame, indicating which pixels in the target video frame are involved in activating the feature, and output the target feature map. This improves the expressive power of deep learning models on small samples, thereby improving the efficiency of amniotic fluid depth measurement.
[0080] S826 assigns a binary label to each pixel of the target feature map to obtain a binary image of the target video frame.
[0081] Following the steps above, a semantic segmentation model can be invoked to assign a binary label to each pixel of the target feature map. This binary label indicates whether the pixel belongs to the amniotic fluid section measurement region. The model outputs a binary image of the target video frame. For example, the semantic segmentation model could be MobileUnet, an efficient end-to-end segmentation network that combines a pre-training stage, a feature extraction stage, and a semantic segmentation stage. In the feature extraction stage, the model introduces depthwise separable convolutions, reducing network complexity and model size. Before using this model, pre-training is required. During training, considering the difference between ultrasound images and natural images—color features in natural images can distinguish different objects, but color features in ultrasound images are not the basis for judging standard ultrasound sections—color features are considered noise elements affecting model judgment. Therefore, a feature fusion strategy is employed to suppress this noise. This feature fusion strategy involves converting an ultrasound image to grayscale, then performing smoothing operations to obtain an image with less noise and sharpening operations to obtain an image with strong boundary features. These images are then concatenated by channel and used as training data input into the semantic segmentation model.
[0082] After pre-training, a trained semantic segmentation model is obtained. In the semantic segmentation stage, the target feature map is used as input, and the model is called. Feature thresholds can be preset to map the originally continuous pixel values in the target feature map to binary labels. For example, the pixel value of a pixel greater than the feature threshold is set to 255, and the pixel value of a pixel less than the threshold is set to 0. This segmentation of the target feature map to a certain extent is achieved. The final output of the model is a binary image of the target video frame, which is used for subsequent measurement of amniotic fluid depth.
[0083] In this embodiment, by introducing depthwise separable convolution, feature information in the target video frame and the feature relationships between different channels can be effectively captured without increasing the amount of computation, and the feature map of the target video frame can be output, which improves the efficiency of the feature extraction process and thus improves the efficiency of amniotic fluid depth measurement.
[0084] In one embodiment, such as Figure 5 As shown, S840 includes:
[0085] S842 performs pixel-by-pixel subtraction on each column of pixels in the binary image, and determines the boundary pixels of each column of pixels in the binary image based on the pixel value difference.
[0086] The segmented binary image can be viewed as a two-dimensional matrix, but it only has two elements; for example, its elements are only 0 and 255. To measure the amniotic fluid depth, it is necessary to first determine the boundary pixels of the binary image, that is, to subtract the pixel values of each column of pixels in the binary image, and determine the boundary pixels of each column of pixels based on the pixel value differences.
[0087] For example, the binary image can be viewed as a matrix with elements of 0 and 255. By subtracting the previous pixel value from the next pixel value (e.g., [0,255,0], 255-0 = 255, 0-255 = -255), the pixel at the boundary can be determined. This allows the pixel to be marked and its coordinates in the matrix to be determined.
[0088] S844: For each column of pixels in a binary image, obtain the pixel distance between boundary pixels.
[0089] After determining the boundary pixels, the pixel coordinates of the boundary pixels in each column are calculated to obtain the pixel distance of the boundary pixels. Furthermore, considering the possibility of isolated pixels being misidentified as boundary pixels, a pixel distance threshold can be set to only consider boundary pixels within the threshold range.
[0090] S846 determines the maximum pixel distance in the binary image as the amniotic fluid depth measurement value.
[0091] After obtaining the pixel distance of the edge detection pixels in each column, their sizes can be compared, and the maximum pixel distance can be determined as the amniotic fluid depth measurement value.
[0092] The amniotic fluid depth measurement obtained at this point depends on the physical size of the pixels. If a more intuitive reference to the actual fetal growth parameters is desired, the pixel-level amniotic fluid depth measurement can be mapped to a standard scale to convert it into an actual amniotic fluid depth value. Specifically, different models of ultrasound scanners used by doctors have different standard scale lengths, meaning that one pixel represents a proportion of the actual length. After conversion, the actual amniotic fluid depth can be obtained.
[0093] In this embodiment, the pixel values of each column of pixels in the binary image are subtracted one by one. Since different values between adjacent pixels in a binary image indicate that the pixel is on the edge of the target, the boundary pixels of each column of pixels in the binary image can be determined based on the pixel value differences. For each column of pixels in the binary image, the pixel distance between the boundary pixels is obtained, and the maximum pixel distance in the binary image is determined as the amniotic fluid depth measurement value. The above scheme allows for the measurement of amniotic fluid depth at the pixel level, improving the accuracy of the amniotic fluid depth measurement value.
[0094] In one embodiment, such as Figure 5 As shown, S840 also includes:
[0095] S847, if the binary image representation has multiple amniotic fluid measurement regions, obtain the maximum pixel distance of each amniotic fluid measurement region.
[0096] In the actual process of acquiring fetal ultrasound video frames, the pregnant woman is placed in a supine position, and the maternal abdomen is divided into four quadrants using the midline of the abdomen and the level of the umbilicus as the x and y coordinates. Therefore, the final binary image may contain amniotic fluid measurement areas in multiple quadrants. In this case, it is necessary to determine the boundary point pixels and calculate the maximum pixel distance for the amniotic fluid measurement areas in each quadrant.
[0097] S848 determines the average of the maximum pixel distances of each amniotic fluid measurement area as the amniotic fluid depth measurement value.
[0098] Following the steps above, the average of the maximum pixel distances in the amniotic fluid measurement areas across multiple quadrants is then used to determine the amniotic fluid depth measurement value.
[0099] In this embodiment, if the binary image representation contains multiple amniotic fluid measurement regions, pixel measurement is performed on each region, and the maximum pixel distance of each region is obtained. Then, the average of these maximum pixel distances is calculated, and this average value is determined as the amniotic fluid depth measurement value. In other words, even with multiple amniotic fluid measurement regions, deep learning techniques can still be used to determine the amniotic fluid depth measurement value, thus improving the efficiency of amniotic fluid depth measurement.
[0100] In one embodiment, such as Figure 6 As shown, S400 includes:
[0101] S420 uses fetal ultrasound video frames as input, calls a trained section classification model to perform section classification on the fetal ultrasound video frames, and filters out video frames carrying amniotic fluid section labels. The trained section classification model is trained based on historical fetal ultrasound video frames carrying section classification labels.
[0102] Because fetal ultrasound video frames may contain various tissues, such as amniotic fluid and the fetal torso, a coarse-grained classification is needed to filter out video frames carrying amniotic fluid section labels, i.e., video frames containing amniotic fluid sections. For example, a trained section classification model can be used to classify the fetal ultrasound video frames. This model could be swinTransformerV2 (a hierarchical visual self-attention model with moving windows). Before using this model, it needs to be pre-trained. The training data can be historical fetal ultrasound video frames with section classification labels provided by multiple hospitals, allowing the model to learn a variety of rich features.
[0103] For example, in practical applications, when the doctor's ultrasound probe scans the pregnant woman's abdominal wall and moves parallel to the abdominal wall, the system can automatically determine whether the current section is an amniotic fluid section and automatically filter out video frames that do not contain an amniotic fluid section.
[0104] S440 determines the confidence level of the video frame.
[0105] After obtaining the video frames carrying amniotic fluid section labels, further screening is required because the amniotic fluid sections in these video frames may not be suitable for subsequent amniotic fluid depth measurements. Since a trained section classification model is used to classify fetal ultrasound video frames to screen for those carrying amniotic fluid section labels, a confidence score calculation is introduced to measure the quality of the amniotic fluid sections in the selected video frames.
[0106] Here, confidence level refers to the confidence level of the section classification model in its predicted label. A higher confidence level indicates that the video frame is more likely to carry a standard amniotic fluid section, while a lower confidence level indicates that the video frame may carry a non-standard amniotic fluid section. For example, the confidence level calculation function can be called to calculate the confidence level of each video frame in order to predict the probability value corresponding to the label carried by the video frame.
[0107] S460: From video frames carrying amniotic fluid section labels, select initial video frames that contain standard amniotic fluid sections with a confidence level higher than a preset confidence threshold.
[0108] Following the steps above, the confidence level of each video frame carrying an amniotic fluid section label is calculated. This confidence level is then compared to a preset confidence threshold. Video frames with confidence levels higher than the preset threshold are considered to have more reliable amniotic fluid section labels, and the amniotic fluid sections they contain are more suitable for subsequent amniotic fluid depth measurement. Therefore, initial video frames containing standard amniotic fluid sections with confidence levels higher than the preset confidence threshold are selected from the video frames carrying amniotic fluid section labels for subsequent amniotic fluid depth measurement.
[0109] For example, amniotic fluid profiles can be divided into standard amniotic fluid profiles, basic standard amniotic fluid profiles, and non-standard amniotic fluid profiles based on confidence level and a preset confidence threshold. Video frames containing non-standard amniotic fluid profiles have lower confidence levels, so video frames containing non-standard amniotic fluid profiles need to be filtered out to reduce the error in amniotic fluid depth measurement.
[0110] In this embodiment, fetal ultrasound video frames are used as input. A trained section classification model is invoked to classify the fetal ultrasound video frames into sections and filter out video frames carrying amniotic fluid section labels. Then, the confidence level of the video frames is calculated, and initial video frames containing standard amniotic fluid sections with confidence levels higher than a preset confidence threshold are selected from the video frames carrying amniotic fluid section labels. The trained section classification model is trained based on historical fetal ultrasound video frames carrying section classification labels. Through the scheme in this embodiment, combining the section classification model and confidence level, initial video frames containing standard amniotic fluid sections can be effectively selected from fetal ultrasound video frames, reducing errors in subsequent amniotic fluid depth measurements and improving the accuracy of amniotic fluid depth measurements.
[0111] In one embodiment, such as Figure 7 As shown, the trained section classification model includes a feature extractor and a feature mapper. S420 includes:
[0112] S422: Using fetal ultrasound video frames as input, the feature extractor is called to extract features from the fetal ultrasound video frames, and the category feature data of the fetal ultrasound video frames is obtained.
[0113] Because the image features of ultrasound cross-sections of specific fetal tissues contained in ultrasound video frames are similar, classifying fetal ultrasound video frames based on cross-section categories requires using a feature extractor to first extract category feature data from the fetal ultrasound video frames. This category feature data characterizes which part of the fetus the cross-section contained in the video frame belongs to. For example, the feature extractor can be a convolutional neural network used to extract category feature data, which can exist in the form of feature vectors for subsequent classification.
[0114] S424, taking category feature data as input, calls the feature mapper to map the category feature data to preset category labels, and filters out video frames carrying amniotic fluid section labels.
[0115] Following the steps above, after extracting category feature data from fetal ultrasound video frames, the category feature data is used as input to call the feature mapper to map the category feature data to preset category labels, which can filter out video frames carrying amniotic fluid section labels.
[0116] For example, categorical feature data can indicate which part of the fetus a section contained in a corresponding fetal ultrasound video frame belongs to, such as an amniotic fluid section, a trunk section, etc. A feature mapper can map different categorical feature data to different category labels. For instance, if the input is a fetal ultrasound video frame containing an amniotic fluid section, after passing through a feature extractor, amniotic fluid feature data can be obtained, existing in the form of an amniotic fluid feature vector. This amniotic fluid feature vector is passed to the feature mapper, which maps the amniotic fluid feature vector to an amniotic fluid section label, ultimately outputting a video frame carrying the amniotic fluid section label.
[0117] In this embodiment, the section classification model includes a feature extractor and a feature mapper. Using fetal ultrasound video frames as input, the feature extractor extracts features from the fetal ultrasound video frames to obtain category feature data. Then, using the category feature data as input, the feature mapper maps the category feature data to preset category labels, allowing the filtering out video frames carrying amniotic fluid section labels. This scheme completes coarse-grained classification of fetal ultrasound video frames, filtering out video frames that do not contain amniotic fluid sections, thus facilitating further processing of video frames containing amniotic fluid sections, reducing subsequent workload, and improving the accuracy of amniotic fluid depth measurement.
[0118] To provide a clearer explanation of the amniotic fluid depth measurement method provided in this application, a specific embodiment and accompanying drawings are described below. Figure 7 The specific embodiment includes the following steps:
[0119] S200, acquires fetal ultrasound video frames.
[0120] S422: Using fetal ultrasound video frames as input, the feature extractor is called to extract features from the fetal ultrasound video frames, and the category feature data of the fetal ultrasound video frames is obtained.
[0121] S424, taking category feature data as input, calls the feature mapper to map the category feature data to preset category labels, and filters out video frames carrying amniotic fluid section labels.
[0122] S440 determines the confidence level of the video frame.
[0123] S460: From video frames carrying amniotic fluid section labels, select initial video frames that contain standard amniotic fluid sections with a confidence level higher than a preset confidence threshold.
[0124] S600 performs target region detection on the initial video frame, locates the amniotic fluid section measurement area in the initial video frame, and obtains the target video frame.
[0125] S822 performs depthwise separable convolution on the target video frame to extract its features and obtain its feature map.
[0126] S824 performs deconvolution processing on the feature map of the target video frame to obtain the target feature map.
[0127] S826 assigns a binary label to each pixel of the target feature map to obtain a binary image of the target video frame.
[0128] S842 performs pixel-by-pixel subtraction on each column of pixels in the binary image, and determines the boundary pixels of each column of pixels in the binary image based on the pixel value difference.
[0129] S844: For each column of pixels in a binary image, obtain the pixel distance between boundary pixels.
[0130] S846 determines the maximum pixel distance in the binary image as the amniotic fluid depth measurement value.
[0131] S847, if the binary image representation has multiple amniotic fluid measurement regions, obtain the maximum pixel distance of each amniotic fluid measurement region.
[0132] S848 determines the average of the maximum pixel distances of each amniotic fluid measurement area as the amniotic fluid depth measurement value.
[0133] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0134] Based on the same inventive concept, this application also provides an amniotic fluid depth measuring device for implementing the amniotic fluid depth measurement method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more amniotic fluid depth measuring device embodiments provided below can be found in the limitations of the amniotic fluid depth measurement method described above, and will not be repeated here.
[0135] In one embodiment, such as Figure 8As shown, an amniotic fluid depth measurement device 900 is provided, comprising: a data acquisition module 920, a section classification module 940, a region detection module 960, and a depth measurement module 980, wherein:
[0136] The data acquisition module 920 is used to acquire fetal ultrasound video frames.
[0137] The section classification module 940 is used to classify the sections of fetal ultrasound video frames and filter out the initial video frames that contain standard amniotic fluid sections with a confidence level higher than the preset confidence threshold.
[0138] The region detection module 960 is used to perform target region detection on the initial video frame, locate the amniotic fluid section measurement area in the initial video frame, and obtain the target video frame.
[0139] The depth measurement module 980 is used to determine the amniotic fluid depth measurement value based on the amniotic fluid cross-section measurement area in the target video frame.
[0140] In one embodiment, the depth measurement module 980 is further configured to take the target video frame as input, call the trained semantic segmentation model to perform binarization processing on the target video frame, and obtain a binary image of the target video frame. The trained semantic segmentation model is trained based on the historical amniotic fluid cross-section measurement region carrying binary labels, and the amniotic fluid depth measurement value is determined according to the binary image.
[0141] In one embodiment, the depth measurement module 980 is further configured to perform a depth-separable convolution operation on the target video frame, extract the features of the target video frame to obtain a feature map of the target video frame, perform deconvolution on the feature map of the target video frame to obtain a target feature map, and assign a binary label to each pixel of the target feature map to obtain a binary image of the target video frame.
[0142] In one embodiment, the depth measurement module 980 is further configured to perform pixel-by-pixel subtraction on each column of pixels in the binary image, determine the boundary pixels of each column of pixels in the binary image based on the pixel value difference, obtain the pixel distance of the boundary pixels for each column of pixels in the binary image, and determine the maximum pixel distance in the binary image as the amniotic fluid depth measurement value.
[0143] In one embodiment, the depth measurement module 980 is further configured to, if the binary image representation has multiple amniotic fluid measurement regions, obtain the maximum pixel distance of each amniotic fluid measurement region, and determine the average value of the maximum pixel distance of each amniotic fluid measurement region as the amniotic fluid depth measurement value.
[0144] In one embodiment, the section classification module 940 is further configured to take fetal ultrasound video frames as input, call a trained section classification model to perform section classification on the fetal ultrasound video frames, filter out video frames carrying amniotic fluid section labels, determine the confidence level of the video frames, and filter out initial video frames containing standard amniotic fluid sections with confidence levels higher than a preset confidence threshold from the video frames carrying amniotic fluid section labels. The trained section classification model is trained based on historical fetal ultrasound video frames carrying section classification labels.
[0145] In one embodiment, the section classification module 940 is further configured to take the fetal ultrasound video frame as input, call the feature extractor to extract features from the fetal ultrasound video frame to obtain the category feature data of the fetal ultrasound video frame, and take the category feature data as input to call the feature mapper to map the category feature data to a preset category label, and filter out the video frames carrying the amniotic fluid section label.
[0146] Each module in the aforementioned amniotic fluid depth measurement device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0147] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores amniotic fluid depth measurement data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an amniotic fluid depth measurement method.
[0148] Those skilled in the art will understand that Figure 9The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above embodiments of the amniotic fluid depth measurement methods.
[0150] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above embodiments of the amniotic fluid depth measurement methods.
[0151] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above embodiments of the amniotic fluid depth measurement methods.
[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0153] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0154] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0155] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for measuring amniotic fluid depth, characterized in that, The method includes: Acquire fetal ultrasound video frames; Using the fetal ultrasound video frame as input, the feature extractor of the trained section classification model is called to extract features from the fetal ultrasound video frame to obtain the category feature data of the fetal ultrasound video frame. Using the category feature data as input, the feature mapper of the trained section classification model is invoked to map the category feature data to preset category labels, and video frames carrying amniotic fluid section labels are selected; the confidence level of the video frames is determined; from the video frames carrying amniotic fluid section labels, initial video frames containing standard amniotic fluid sections with confidence levels higher than preset confidence thresholds are selected; the trained section classification model is trained based on historical fetal ultrasound video frames carrying section classification labels. Target region detection is performed on the initial video frame to locate the amniotic fluid section measurement area in the initial video frame, thus obtaining the target video frame; Using the target video frame as input, the trained semantic segmentation model is called to perform binarization processing on the target video frame to obtain a binary image of the target video frame. The trained semantic segmentation model is trained based on the historical amniotic fluid section measurement region carrying binary labels. For each column of pixels in the binary image, the pixel values are subtracted one by one. Based on the pixel value difference, the boundary pixels of each column of pixels in the binary image are determined. For each column of pixels in the binary image, the pixel distance of the boundary pixels is obtained. The maximum pixel distance in the binary image is determined as the amniotic fluid depth measurement value.
2. The method according to claim 1, characterized in that, The step of using the target video frame as input and calling a trained semantic segmentation model to binarize the target video frame to obtain a binary image of the target video frame includes: Perform depthwise separable convolution processing on the target video frame to extract the features of the target video frame and obtain the feature map of the target video frame; The feature map of the target video frame is deconvolved to obtain the target feature map; A binary label is assigned to each pixel of the target feature map to obtain a binary image of the target video frame.
3. The method according to claim 1, characterized in that, The step of determining the maximum pixel distance in the binary image as the amniotic fluid depth measurement value further includes: If the binary image representation has multiple amniotic fluid measurement regions, obtain the maximum pixel distance of each amniotic fluid measurement region; The average of the maximum pixel distances in each amniotic fluid measurement area is determined as the amniotic fluid depth measurement value.
4. The method according to claim 1, characterized in that, The fetal ultrasound video frames are obtained by extracting frame images from the original fetal ultrasound video stream and removing redundant information and noise from the frame images.
5. A device for measuring amniotic fluid depth, characterized in that, The device includes: The data acquisition module is used to acquire fetal ultrasound video frames; The section classification module is used to extract features from the fetal ultrasound video frames as input by calling the feature extractor of a trained section classification model to obtain category feature data of the fetal ultrasound video frames; using the category feature data as input, calling the feature mapper of the trained section classification model to map the category feature data to preset category labels, and filtering out video frames carrying amniotic fluid section labels; determining the confidence level of the video frames; and filtering out initial video frames containing standard amniotic fluid sections with confidence levels higher than preset confidence thresholds from the video frames carrying amniotic fluid section labels; the trained section classification model is trained based on historical fetal ultrasound video frames carrying section classification labels. The region detection module is used to perform target region detection on the initial video frame, locate the amniotic fluid section measurement region in the initial video frame, and obtain the target video frame. The depth measurement module is used to take the target video frame as input, call a trained semantic segmentation model to binarize the target video frame, and obtain a binary image of the target video frame. The trained semantic segmentation model is trained based on historical amniotic fluid cross-sectional measurement regions carrying binary labels. The module performs pixel-by-pixel subtraction on each column of pixels in the binary image, and determines the boundary pixels of each column of pixels based on the pixel value differences. For each column of pixels in the binary image, the pixel distance of the boundary pixels is obtained. The maximum pixel distance in the binary image is determined as the amniotic fluid depth measurement value.
6. The apparatus according to claim 5, characterized in that, The depth measurement module is further configured to: perform depth-separable convolution processing on the target video frame to extract the features of the target video frame and obtain a feature map of the target video frame; perform deconvolution processing on the feature map of the target video frame to obtain a target feature map; and assign a binary label to each pixel of the target feature map to obtain a binary image of the target video frame.
7. The apparatus according to claim 5, characterized in that, The depth measurement module is further configured to: if the binary image representation has multiple amniotic fluid measurement regions, obtain the maximum pixel distance of each amniotic fluid measurement region; and determine the average value of the maximum pixel distance of each amniotic fluid measurement region as the amniotic fluid depth measurement value.
8. The apparatus according to claim 5, characterized in that, The fetal ultrasound video frames are obtained by extracting frame images from the original fetal ultrasound video stream and removing redundant information and noise from the frame images.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.