Placenta umbilical cord insertion opening standard section automatic measurement method based on multi-task model

Abstract

The invention relates to a placenta umbilical cord insertion opening standard section automatic measurement method based on a multi-task model, and the method comprises the steps: inputting a placenta image into a YOLO multi-task model, and carrying out the recognition of a detection frame and corresponding confidence of each target structure in a placenta umbilical cord insertion opening section, and a placenta mask of the placenta umbilical cord insertion opening section; analyzing according to the detection frame of the target structure and the corresponding confidence, and determining a placenta umbilical cord insertion opening standard section and a corresponding placenta mask; according to the placenta mask of the placenta umbilical cord insertion opening standard section, obtaining the central axis and the contour line of the mask and the fan-shaped region boundary of the original image; analysis and measurement are carried out based on the central axis and the contour line of the mask and the sector region boundary of the original image to obtain an optimal placenta measurement line, and efficient and accurate placenta umbilical cord insertion opening standard tangent plane automatic measurement can be realized.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an automatic measurement method for the standard section of the placental umbilical cord insertion site based on a multi-task model. Background Technology

[0002] Placental thickness is an important indicator in fetal ultrasound examinations. Traditional manual measurement requires doctors to constantly monitor the screen, repeatedly reviewing frames to obtain a standard cross-section, and manually selecting measurement points, making the process cumbersome. Therefore, how to achieve automated and accurate measurement of the standard cross-section of the placental umbilical cord insertion site is an urgent problem to be solved. Summary of the Invention

[0003] Therefore, it is necessary to provide an automatic measurement method for the standard cross-section of the placental umbilical cord insertion site based on a multi-task model, which can achieve accurate measurement, to address the above problems.

[0004] An automatic measurement method for the standard section of the placental umbilical cord insertion site based on a multi-task model includes: Acquire the placenta image to be tested, and the YOLO multi-task model for identifying the standard cross-section of the placental umbilical cord insertion site; The placental image is input into the YOLO multi-task model to identify the detection boxes and corresponding confidence scores of each target structure in the cross-section of the placental umbilical cord insertion port, as well as the placental mask of the cross-section of the placental umbilical cord insertion port; the target structures include the placenta and the placental umbilical cord insertion port. Based on the detection frame of the target structure and the corresponding confidence level, the standard section of the placental umbilical cord insertion site and the corresponding placental mask are determined by analysis. Based on the placental mask of the standard section of the placental umbilical cord insertion opening, obtain the central axis, outline, and sector boundary of the original image. The optimal placental measurement line is obtained by analyzing and measuring the central axis, contour line, and sector boundary of the original image based on the mask.

[0005] In one embodiment, the YOLO multi-task model adds a segmentation task head to the YOLO detection model and is trained using a YOLO-formatted detection dataset and a segmentation dataset; the detection dataset is labeled with two structures: the placenta and the placental umbilical cord insertion site, and the segmentation dataset is labeled with a placental mask.

[0006] In one embodiment, analysis is performed based on the detection frame of the target structure and the corresponding confidence level to determine the standard section of the placental umbilical cord insertion site and the corresponding placental mask, including: Based on the detection frame of the target structure, determine whether the placenta and placental umbilical cord insertion site are present in the cross-section. If neither is present, the score is set to 59. If only one structure is present, the score is set to 59 × structure confidence. If both structures are present, the score is calculated based on the corresponding confidence and the weight ratio of the placenta and placental umbilical cord insertion site. If the confidence of a target structure is less than 0.3, the total score is reduced by 30. If the confidence of the umbilical cord insertion site is less than 0.45, the score is reduced by 20. If the confidence of the placental umbilical cord insertion site is less than 0.8, the score is adjusted to 75. Based on the scores of each placental umbilical cord insertion site cross-section, determine the standard cross-section of the placental umbilical cord insertion site and the corresponding placental mask.

[0007] In one embodiment, based on the placental mask of the standard cross-section of the placental umbilical cord insertion opening, the central axis, outline, and sector boundary of the original image are obtained, including: Perform a closing operation on the placental mask to find the region with the largest outer contour as the placental region to be measured, and set the remaining regions to 0 to obtain mask A; The number of pixels in the found placental region is counted x1, and the mask B is obtained by filling the outer contour. The mask A is inverted and ANDed with the mask B to obtain the internal hole mask C. Count the number of pixels in the internal hole mask C by x2. If x2*W<=x1, the internal hole is a hole that is incorrectly segmented by the model. Keep mask B as the final mask to be tested. W is a set coefficient. Otherwise, it means that the placenta to be tested is a ring placenta. Invert mask C and AND it with mask B to obtain the final mask to be tested. The central axis, contour lines, and sector boundaries of the original image are determined based on the final mask analysis.

[0008] In one embodiment, determining the centerline includes: Extract the skeleton line mask using the morphology.skeletonize() function, then obtain the skeleton line point set using the cv2.findContours() function, and extract the branches of the skeleton line point set in sequence. Select the longest branch based on the length of each branch and the distance between the endpoints of each branch, and merge the second longest branch to obtain the main axis of the skeleton line. Use the cv2.approxPolyDP() method to simplify the main axis of the skeleton line. After simplification, add 1-3 points based on the pixel distance between two points to obtain the center axis.

[0009] In one embodiment, determining the boundary of the sector region of the original image includes: Hough transform is used to detect straight lines. For the left and right boundary parameters, a maximum of 4 lines are selected. The threshold is the difference between the farthest points in the vertical direction of the mask. The angle on the left is set to 12° to 72°, and the angle on the right is set to 108° to 164°, resulting in two boundary lines. The symmetry of the two lines is compared. If the angle deviation is >3°, the line with the fewer votes is filtered according to the vote count in the Hough transform to obtain the left and right fan-shaped boundary lines. These lines are marked in the blank array of the original image size and expanded by 2 pixels to obtain the marked area M1. The dilation segmentation mask treats a 7×7 region near the dilated contour points as a potential boundary region. Based on the original image size h×w, if the mask bounding box is in the lower half of the image, the portion above 0.95h is excluded, and vice versa, the portion above 0.85h is excluded, and this is recorded as the potential lower boundary region. If the left and right spacing of the bounding box is greater than 0.4w, the region between 0.2w and 0.8w is excluded, and vice versa, the region between 0.3w and 0.7w is excluded, and this is recorded as the potential left and right boundary regions. The sum of the pixel values ​​of the original image within the 7×7 range of the corresponding point coordinates of the two potential regions is determined. If it is less than 10, it is marked as a boundary region, resulting in the marked region M2 after dilation. The marked regions M1 and M2 are merged to obtain the boundary of the fan-shaped region.

[0010] In one embodiment, the optimal placental measurement line is obtained by analyzing and measuring the central axis, contour lines, and fan-shaped region boundaries of the mask, including: The inflection point of the centerline is checked based on the mask to confirm whether there is a large fold or annular placenta. If a large fold or annular placenta exists, the multi-segment centerline obtained after processing the large fold or annular placenta is processed to obtain the optimal placenta measurement line. If it is a long strip placenta without large corners, the start and end points are corrected and the ordinary placenta is measured to obtain the optimal placenta measurement line.

[0011] In one embodiment, the inflection point is checked based on the central axis of the mask to confirm the presence of a large inverted or annular placenta, including: The cv2.approxPolyDP() function is used to sparse the points on the central axis to obtain the position of the inflection point. If there are fewer than 3 inflection points, it means that the placenta is long and narrow. If there are 3 or more inflection points, the angle between the two sides of each simplified point is calculated: if it is less than 100°, it is recorded as an inflection point, and the angle list of the inflection point and each simplified point is recorded. If no inflection point is found, it means that the placenta is long and narrow. Based on the inflection point and the angle list of each simplified point, the centerline is divided into multiple segments according to the simplified points. First, the index of the centerline point corresponding to the simplified point is found. Then, the centerline is traversed from beginning to end. Points with a pixel distance greater than 5 are included in the centerline of each segment. If the number of points in each segment is greater than 3, it is recorded as a segment. The cv2.pointPolygonTest() function is used to determine the distance between each point in the first and last segment and the hub line. Points with a distance less than 12 are removed. Segments with less than 3 points are filtered out. The point set of each segment divided according to the inflection point, the simplified point set, the angle of the corresponding edge of each simplified point, and the centerline are recorded.

[0012] In one embodiment, the multi-segment midline obtained after processing the large fold and circumferential placenta is processed to obtain the optimal placental measurement line, including: If it is a large folded placenta, delete the points that are less than 25px away from the inflection point, and return the set of segmented axis points with a length greater than 3, the mask outline, and the "folded placenta exists" flag. If it is a ring-shaped placenta, take the previously recorded inner ring mask and coordinates, and divide the ring into four regions: upper left, upper right, lower left, and lower right, based on the midpoint of the inner mask. Count the regions corresponding to the point sets of each segment, find the region with the most points, and extract the mask of the corresponding region. Calculate the distance between the first and last points of the axis in each segment, sort them by distance, and then calculate the distance for each segment separately. First, count the number of axis points. If there are 2 axis points, directly calculate the orthogonal vector based on the coordinates of the two points. Draw a straight line based on the orthogonal vector with the two points as the starting point to obtain the intersection point with the contour. Record the longest measurement line. Calculate the orthogonal vector of each line segment. For a point X, if it is the first and last point, the direction is recorded as the orthogonal vector of the line segment. Otherwise, take the sum of the orthogonal vectors of two adjacent line segments as the direction vector. Draw a straight line based on the direction vector and point X, and record the intersection point, measure the line length and the total length. Sort the obtained measurement lines by length, set the outlier threshold multiplier to 1.4, and the outlier threshold to 1.4 × average length. Traverse the measurement lines from largest to smallest. If two points of the measurement line are not on the boundary mask and the length is not greater than the extreme value threshold, the traversal ends and the measurement line is taken as the placental measurement line of this segment. The longest segment axis is designated as the main measurement line. The comparison threshold for other segments is set to 1.4. If the number of points in the point set is equal to the number of points in the main segment axis, the comparison threshold is set to 1. The outlier threshold is set to 2.5. The measurement lines of the remaining segments are iterated sequentially, checking if they intersect with the main measurement line. If they intersect, the main measurement line is retained, and the iteration continues. If they do not intersect, the comparison threshold multiplied by the main measurement line and the remaining measurement lines are compared. If the comparison threshold multiplied by the main measurement line is greater than the remaining measurement lines, the main measurement line is not replaced. If the length of the remaining measurement lines is greater than the outlier threshold multiplied by the main measurement line, the main measurement line is not replaced; otherwise, the main measurement line is replaced. The iteration continues with the measurement lines of the next segment. The optimal placental measurement line is obtained after this process.

[0013] In one embodiment, measurements are taken on a normal placenta to obtain an optimal placental measurement line, including: Calculate the angle of each axis point, set the angle threshold to 180°, set a maximum of ten loops, and decrease the angle threshold by 4.5 each time until the number of points with angles greater than the threshold is greater than 60% of the total number of points, and record the corresponding points; calculate the orthogonal vector and line segment length of each axis point, and calculate the cumulative length of each point, the total length of the line segment, and the average length; Calculate the average length of the measurement lines and remove those shorter than 0.6 × the average length. After removal, count the remaining measurement lines and remove the first and last measurement lines based on the number of points. If the number of points is greater than 18, remove the first and last 4 points; if the number of points is greater than 15, remove the first and last 3 points; if the number of points is greater than 9, remove the first and last 2 points; if the number of points is greater than 6, remove the first and last 1 point. After removal, recalculate the average length of the measurement lines and set the comparison threshold to 1.4. The critical value is the comparison threshold × the average value. If the length of the longest measurement line exceeds the critical value, it is considered that the longest measurement line is abnormal. Sort the remaining measurement lines in descending order of length and select the measurement line that is not greater than the critical value as the longest measurement line, which is recorded as the optimal placental measurement line.

[0014] The aforementioned automatic measurement method for the standard section of the placental umbilical cord insertion site based on a multi-task model inputs the placental image into the YOLO multi-task model, identifies the detection boxes and corresponding confidence scores of each target structure in the section of the placental umbilical cord insertion site, as well as the placental mask of the section; analyzes the detection boxes and corresponding confidence scores of the target structures to determine the standard section of the placental umbilical cord insertion site and the corresponding placental mask; obtains the central axis, contour line, and fan-shaped region boundary of the original image based on the placental mask of the standard section of the placental umbilical cord insertion site; and performs analysis and measurement based on the central axis, contour line, and fan-shaped region boundary of the original image to obtain the optimal placental measurement line, thus achieving efficient and accurate automatic measurement of the standard section of the placental umbilical cord insertion site. Attached Figure Description

[0015] Figure 1This is a flowchart of an automatic measurement method for the standard section of the placental umbilical cord insertion site based on a multi-task model in one embodiment. Figure 2 A schematic diagram of the centerline is obtained in one embodiment; Figure 3 This is a schematic diagram of different types of placenta in one embodiment; Figure 4 This is a schematic diagram of the large folded placenta measurement process in one embodiment; Figure 5 This is a schematic diagram of the annular placenta measurement process in one embodiment; Figure 6 This is a schematic diagram of a typical placental measurement process in one embodiment; Figure 7 This is a schematic diagram of the measurement results of the large folded placenta in one embodiment; Figure 8 This is a schematic diagram of the measurement results of the annular placenta in one embodiment; Figure 9 This is a schematic diagram of the measurement results of a typical placenta in one embodiment; Figure 10 This is a structural block diagram of an automatic measurement device for the standard cross-section of the placental umbilical cord insertion port based on a multi-task model in one embodiment. Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0017] In one embodiment, such as Figure 1 As shown, an automatic measurement method for the standard section of the placental umbilical cord insertion site based on a multi-task model is provided, including: Step S110: Obtain the image of the placenta to be tested, and the YOLO multi-task model for recognizing the standard cross-section of the placental umbilical cord insertion site.

[0018] The YOLO multi-task model is an improvement on the YOLOv8 model, enabling multi-task performance in both detection and segmentation tasks. It allows for multi-object detection and single-object segmentation on top of the YOLO model, avoiding limitations imposed by the segmentation dataset. To train the YOLO multi-task model, separate detection and segmentation datasets in YOLO format were prepared. The detection dataset labeled the placenta and the placental umbilical cord insertion site, while the segmentation dataset labeled the placenta with a mask.

[0019] Specifically, the improved YOLO multi-task model shares the model's high-dimensional features. It adds a segmentation task head to the YOLO detection model and is trained using both detection and segmentation datasets in YOLO format. During training, the detection model is trained separately first. After obtaining the pre-trained model, the detection task head is frozen, and the high-dimensional features and segmentation task head are trained. Then, the high-dimensional features and segmentation task head are frozen again, and the segmentation task head is trained to obtain the final model. During inference, the high-dimensional features are input into the detection and segmentation task heads respectively, generating detection and segmentation results respectively. This model and training method saves GPU memory, reduces overhead in multi-task, multi-model scenarios, and accelerates task inference speed. The improved YOLO multi-task model can perform detection and segmentation tasks simultaneously. The detection task analyzes the bounding boxes and corresponding confidence scores of target structures in each placental umbilical cord insertion site section, while the segmentation task analyzes the placental mask in each placental umbilical cord insertion site section.

[0020] Step S120: Input the placental image into the YOLO multi-task model to identify the bounding boxes and corresponding confidence scores of each target structure in the cross-section of the placental umbilical cord insertion site, as well as the placental mask of the cross-section. The target structures include the placenta and the placental umbilical cord insertion site. Multi-task analysis is performed simultaneously using the YOLO multi-task model to determine the bounding boxes and corresponding confidence scores of the placenta and the placental umbilical cord insertion site in the cross-section, as well as the placental mask of the cross-section.

[0021] Step S130: Analyze the target structure based on the detection frame and the corresponding confidence level to determine the standard cross-section of the placental umbilical cord insertion site and the corresponding placental mask.

[0022] For the placental cord insertion site section, the placental cord insertion site and the placenta are the main structures. Therefore, sections lacking either the placenta or the placental cord insertion site are considered non-standard sections. If both structures are detected, the optimal section is selected as the standard section for the placental cord insertion site based on the confidence scores of the two structures. In the YOLO multi-task model, the confidence score is an indicator that measures whether the bounding box contains a target and the accuracy of the bounding box's location. Its calculation can be understood as the product of two parts: Confidence Score = Target Presence Probability × IOU of the bounding box. The target presence probability is inferred by the model and represents the probability that a target exists within the bounding box, ranging from 0 to 1. If a target is present in the bounding box, this value should be close to 1; if no target is present, this value should be close to 0. During model training, the confidence score is optimized using a loss function (such as the sum of squared errors), and the final output confidence score comprehensively reflects the model's confidence in the accuracy of the bounding box's location.

[0023] In this embodiment, the standard types of placental umbilical cord insertion site cross-sections are set with the following scores: Standard: 80-100; Basic Standard: 60-79; Non-standard: 0-59. Step S130 includes: The system determines whether the placenta and placental-umbilical cord insertion site are present in the cross-section of the target structure. If neither is present, the score is set to 59. If only one structure is present, the score is set to 59 × the structure confidence level. If both structures are present, the score is calculated based on the corresponding confidence level and the weight ratio of the placenta and placental-umbilical cord insertion site. For example, the score is calculated based on the confidence level and a placental-umbilical cord insertion site:placenta ratio of 8:2. Furthermore, images with low confidence levels (below a set threshold) can be filtered out. If the confidence level of a target structure is less than 0.3, the total score is reduced by 30. If the confidence level of the umbilical cord insertion site is less than 0.45, the score is reduced by 20. If the confidence level of the placental-umbilical cord insertion site is less than 0.8, the score is adjusted to 75.

[0024] After adjusting the scores, the optimal section is selected based on the scores of each placental cord insertion site section. For example, the section with the highest score is used as the standard section for the placental cord insertion site, and the standard section for the placental cord insertion site and the corresponding placental mask are determined.

[0025] Step S140: Based on the placental mask of the standard section of the placental umbilical cord insertion orifice, obtain the central axis, contour line, and fan-shaped region boundary of the original image. After determining the standard section of the placental umbilical cord insertion orifice, analyze the central axis, contour line, and fan-shaped region boundary of the original image based on its placental mask, to be used for subsequent determination of the optimal placental measurement line according to different types of placentas. In one embodiment, step S140 includes: Step 141: Preprocess the mask.

[0026] By preprocessing the mask, the data needed for subsequent processing is obtained. Since the segmentation mask obtained from the model may have multiple blocks, and there may be holes within the blocks, preprocessing is required to obtain the coordinates of the largest block, the coordinates of the inner ring, and the mask corresponding to the largest region.

[0027] First, locate the correct placental region. Perform a closing operation on the placental mask to connect any two masks that might represent the same placenta. Find the region with the largest outer contour as the placental region to be measured, and set the remaining regions to 0, thus obtaining mask A.

[0028] Next, any potential holes are removed, and the presence of a ring-shaped placenta is determined. The number of pixels in the found placental region (x1) is counted, and a mask B is obtained by filling the area based on its outer contour. Mask A is inverted and ANDed with mask B to obtain the internal hole mask C. There may be multiple holes; the hole with the largest contour in mask C is retained. The number of pixels in the internal hole mask C (x2) is counted. If x2*W <= x1, the internal hole is an incorrectly segmented hole in the model, and mask B is retained as the final test mask, where W is a set coefficient (e.g., 16). Otherwise, it indicates that the placenta under test is a ring-shaped placenta; mask C is inverted and ANDed with mask B to obtain the final test mask.

[0029] Based on the final mask to be tested, the central axis, contour lines (specifically, the set of contour points), and the boundaries of the fan-shaped regions of the original image can be analyzed and determined. After obtaining the final mask to be tested, the bounding boxes of the mask and the inner ring are calculated. To speed up the calculation, the pixels within the bounding box of the mask are extracted separately, and the mask within the bounding box is reduced by a factor of two. Finally, the bounding box coordinates, the scaled mask pixels, the inner ring bounding box coordinates, and the contour point set are retained.

[0030] Step 142: Obtain the centerline.

[0031] The central axis is obtained and then mapped back to its original size. First, the skeleton line mask is extracted using the `morphology.skeletonize()` function, and then the skeleton line point set is obtained using the `cv2.findContours()` function. Since the skeleton line may have a branching structure, and considering that the point set obtained by the above method may contain duplicate points, the branches of the skeleton line point set are extracted sequentially based on the duplicate point situation. After extracting the branches, the longest branch is selected based on the length of each branch and the distance between the endpoints of each branch, and the second longest branch is merged to obtain the main axis of the skeleton line. The optimal measurement line is then found on this skeleton line.

[0032] After obtaining the main axis of the skeleton line, due to the high density of points, the `cv2.approxPolyDP()` function is used to simplify the main axis of the skeleton line to improve processing efficiency. After simplification, 1 to 3 points are added based on the pixel distance between two points to obtain the final centerline. This centerline point set obtained based on the scaling mask is then mapped back to the original size. The obtained centerline is shown below. Figure 2 As shown.

[0033] Step 143: Locate the boundary of the fan-shaped region in the ultrasound image.

[0034] When measuring placental thickness, the goal is to measure the area where the thickness is fully displayed in the image. If the placental measurement line is located at the fan-shaped boundary of the ultrasound image, and the image outside the boundary is unclear, it may not represent the correct thickness of the placenta, so it is necessary to avoid measuring at the boundary location. For the boundary area, this application uses two methods to make a comprehensive judgment: First, Hough transform is used to detect straight lines. For the left and right boundaries, the parameters are set to select a maximum of 4 lines. The threshold is the difference between the farthest points in the vertical direction of the mask (set to 30 if less than 30). The angle for the left side is set to 12° to 72°, and for the right side, it is set to 108° to 164°. After obtaining the left and right boundary lines, the symmetry of the two lines is compared. If the angle deviation is >3°, the line with fewer votes is filtered out based on the vote count in the Hough transform. Finally, the left and right fan-shaped boundary lines are obtained, marked in a blank array of the original image size, and expanded by 2 pixels to obtain the marked area M1.

[0035] Secondly, the pixels surrounding the mask are used for judgment. The mask is dilated and segmented, and the dilated contour is taken. The 7×7 area near the contour points is regarded as the potential boundary region. According to the original image size (h, w), if the mask bounding box is in the lower half of the image, the part above 0.95h is excluded, otherwise the part above 0.85h is excluded, and it is recorded as the potential lower boundary region; if the left and right spacing of the bounding box is greater than 0.4w, the area within (0.2w, 0.8w) is excluded, otherwise the area within (0.3w, 0.7w) is excluded, and it is recorded as the potential left and right boundary regions. The sum of the pixel values ​​of the original image within the 7×7 range of the corresponding point coordinates of the two potential regions is judged. If it is less than 10, it is marked as a boundary region, and the marked region M2 after dilation is returned. Finally, the two regions marked M1 and marked region M2 are merged to obtain the boundary of the fan-shaped region.

[0036] After completing the above steps, step S140 outputs the centerline, contour point set, mask detail information, and sector region boundary for processing in the next step.

[0037] Step S150: Analyze and measure the central axis, contour line and sector boundary of the original image based on the mask to obtain the optimal placental measurement line.

[0038] Specifically, firstly, the inflection points are checked based on the central axis of the mask to confirm the presence of large folds or annular placentas. If large folds or annular placentas exist, special processing is performed on the multi-segment central axis obtained after processing the large folds or annular placentas to obtain the optimal placental measurement line. If it is a normal shape, i.e., a long strip placenta without large corners, the start and end points are corrected before measuring a normal placenta to obtain the optimal placental measurement line. In one embodiment, step S150 includes: Step 151: Check the inflection points.

[0039] First, read the previously acquired mask information to obtain the central axis. Then, use the `cv2.approxPolyDP()` function to sparse the points on the central axis and obtain the positions of the inflection points. If there are fewer than 3 inflection points, it indicates a long, narrow placenta, and the reverse folding process in step 152 is skipped. If there are 3 or more inflection points, calculate the included angle between the two sides of each simplified point: if it is less than 100°, it is recorded as an inflection point, and a list of inflection points and angles for each simplified point is recorded; if no inflection point is found, it indicates a narrow placenta, and the reverse folding process in step 152 is skipped. For example... Figure 3 The diagram shows three types of placentas.

[0040] Step 152: Treat the large folded placenta.

[0041] Based on the inflection point and the angle list of each simplified point, the central axis is divided into multiple segments according to the simplified points. First, find the index of the central axis point corresponding to the simplified point, and then traverse the central axis from beginning to end. Points with a pixel distance greater than 5 are included in the central axis of each segment. If the number of points in each segment is greater than 3, it is recorded as a segment.

[0042] Next, the `cv2.pointPolygonTest()` function is used to determine if each point in the first and last segments is too close to the contour. If the distance is less than 12, the point is removed. If this operation results in the removal of all segments, the above operation is repeated, but only the points of the longest segment are removed. Then, segments with fewer than 3 points are filtered out, and the point sets of each segment divided according to the inflection points, the simplified point sets, the angles of the corresponding edges of each simplified point, and the centerline are recorded.

[0043] Step 153: Correct the set of points on the central axis of the large reflexed placenta.

[0044] If it is a large folded placenta, delete points that are less than 25px away from the inflection point, and return the set of segmented axis points with a length greater than 3, the mask outline, and the "folded placenta exists" marker. Figure 4 This is a schematic diagram of the measurement process for the large folded placenta.

[0045] Step 154: Correct the set of points on the central axis of the annular placenta.

[0046] If it is a ring-shaped placenta, take the previously recorded inner ring mask and coordinates, and divide the ring into four regions: upper left, upper right, lower left, and lower right, based on the midpoint of the inner mask. Count the regions corresponding to the point sets of each segment, find the region with the most points, and extract the mask of the corresponding region.

[0047] Taking the top-left region as an example, given the segmented point set and the inner mask bounding box coordinates, the Y-coordinate boundary is: not exceeding the lower bound of the inner bounding box, and the maximum value of the Y-coordinate of each axis point and the leftmost point of the inner mask; the X-coordinate boundary is: not exceeding the right bound of the inner bounding box, and the maximum value of the X-coordinate of each axis point and the topmost point of the inner mask. Based on the X and Y boundaries, extract the mask. The same applies to other regions. After obtaining each region, extract all axis nodes within that region. If the total number of points is greater than 6, delete the first and last points, obtaining the axis, the segmented region mask outline, and the "exists in reflection" marker. Figure 5 This is a schematic diagram of the process of measuring annular placenta.

[0048] Step 155: If the placenta is of a normal shape, that is, a long strip without large corners, then correct the beginning and end points.

[0049] Specifically, we process the first three points and the last three points along the central axis separately. Taking the first three points as an example: take three points A, B, and C, and calculate the angle between vectors BA and CB. When the angle is less than 21°, it indicates that the tail end is straight, and BA is retained as the direction vector. When the angle is greater than 21° but less than 30°, if the length of CB is greater than 18 and the length of BA is greater than half the length of CB, BA is retained as the direction vector; otherwise, CB is retained as the direction vector. When the angle is greater than 30°, CB is retained as the direction vector.

[0050] Next, draw a straight line based on the retained direction vector, and find the intersection point between the corresponding line and the contour. Since there may be multiple intersection points, calculate the projection of the starting point-intersection vector onto the direction vector, and take the point with the largest projection value as the extension point. Depending on whether the retained direction vector is BA or CB, insert the intersection point at the tail end as the final point.

[0051] Finally, record the modified centerline, mask outline, and "no reverse fold" mark. Figure 6 This is a schematic diagram of the standard placental measurement process.

[0052] Step 156: Process the multi-segment centerlines obtained from the large fold and circumferential placenta to obtain the optimal placental measurement line. Calculate the distance between the beginning and end points of each segment's centerline and sort them by distance. Then extract the optimal measurement line for each segment and select the optimal measurement line as the final result of the algorithm.

[0053] Specifically, first, calculate the distance between the first and last points of the axis in each segment and sort them by distance. Then, calculate for each segment separately. First, count the number of axis points. If there are 2 axis points, directly calculate the orthogonal vector based on the coordinates of the two points. Using the two points as the starting point, draw a straight line based on the orthogonal vector to obtain the intersection point with the contour. Record the longest measurement line.

[0054] If the number of points on the axis is greater than 9, then after deleting the first and last points, proceed with the above logic: First, calculate the orthogonal vector of each line segment. For a point X, if it is one of the first or last points, the direction is recorded as the orthogonal vector of the line segment; otherwise, the sum of the orthogonal vectors of two adjacent line segments is taken as the direction vector. Then, draw a straight line based on the direction vector and point X, record the intersection point, measure the line length, and the total length.

[0055] The average length of the measurement lines is obtained based on the sum of their lengths. The measurement lines are then sorted by length. An outlier threshold multiplier of 1.4 is set, with the outlier threshold being 1.4 × the average length. This threshold is used to prevent extreme values ​​due to erroneous measurements. The recorded boundary mask is retrieved, and the measurement lines are traversed from largest to smallest. If two points on a measurement line are not on the boundary mask and the length is not greater than the extreme value threshold, the traversal ends, and this measurement line is taken as the placental measurement line for that segment.

[0056] After obtaining the measurement lines for each segment, the measurement line with the longest segment axis is designated as the main measurement line. The comparison threshold for other segments is set to 1.4. If the number of points in the point set is equal to the number of points on the main segment axis, the comparison threshold is set to 1. The outlier threshold is set to 2.5. The remaining measurement lines are iterated sequentially, checking if they intersect with the main measurement line. If an intersection occurs, the measurement line is incorrect, and the main measurement line is retained, continuing the iteration. If no intersection occurs, the comparison threshold multiplied by the main measurement line and the remaining measurement lines are compared. If the comparison threshold multiplied by the main measurement line is greater than the remaining measurement lines, the main measurement line is not replaced. If the length of the remaining measurement lines is greater than the outlier threshold multiplied by the main measurement line, the main measurement line is not replaced; otherwise, the main measurement line may be too short, and it is replaced. The iteration continues with the measurement lines of the next segment. The optimal placental measurement line is obtained after this process. If no measurement results are obtained due to the limitation of a fan-shaped boundary region, the process is repeated, but the existence of the fan-shaped region is not checked. Finally, the optimal placental measurement line is obtained, and the algorithm ends. Figure 7 This is a schematic diagram showing the measurement results of the large flexure placenta. Figure 8 This is the measurement result for annular placenta.

[0057] Step 157: Measure a normal-shaped placenta.

[0058] First, it is necessary to limit the angles corresponding to the points on the measurement line. Calculate the angles of each axis point, set the angle threshold to 180°, and set a maximum of ten loops, decreasing the angle threshold by 4.5 each time, until the number of points with angles greater than the angle threshold is greater than 60% of the total number of points, and record the corresponding points.

[0059] Next, calculate the orthogonal vectors and line segment lengths at each point on the axis, and calculate the cumulative length, total line segment length, and average length at each point. Set a distance threshold of 0.02 × total length; this threshold is used to avoid repeatedly measuring two points that are too close together. Iterate through each point, calculating the angle between the orthogonal vectors of the two nearest points. If the angle is greater than 35°, mark it as "existing large angle" and do not measure. Then, for each line segment near a point, if the line segment length is greater than 2 × average length, or if the length of one nearby line segment is twice that of another, add the orthogonal vectors recorded by the two nearby points to smooth the direction vector of that point. After smoothing, iterate through each point on the axis to be measured, draw a straight line based on the direction vector, find the intersection with the contour, and record the point if it is not on the boundary of the sector area, along with the point with the longest measured length. Iterate through the point set again; if the distance between two points is greater than 18, take the midpoint and the sum of the direction vectors of the two points to draw a straight line, calculate the measurement line again, and record the maximum value.

[0060] After completion, check the subscript of the longest measurement line. If the longest measurement line is the measurement line at both ends and the length of both sides is large, then take the measurement line at the second point or the second to last point as the longest measurement line.

[0061] Finally, the results were revised to prevent excessively large outliers: First, the average length of the measurement lines was calculated, and measurement lines shorter than 0.6 × the average length were removed. After removal, the number of remaining measurement lines was counted, and the first and last measurement lines were removed based on the number of points. If the number of points was greater than 18, the first and last 4 points were removed; if the number of points was greater than 15, the first and last 3 points were removed; if the number of points was greater than 9, the first and last 2 points were removed; if the number of points was greater than 6, the first and last 1 point was removed. After removal, the average length of the measurement lines was recalculated, and a comparison threshold of 1.4 was set. The critical value was the comparison threshold × the average value. If the length of the longest measurement line exceeded the critical value, the longest measurement line was considered to be abnormal. The remaining measurement lines were sorted from longest to shortest, and the measurement line not exceeding the critical value was selected as the longest measurement line and recorded as the optimal placental measurement line.

[0062] If no measurement results are found in the sector boundary region during the process, the algorithm is re-executed, but without checking whether it is within the sector region. Finally, the optimal placental measurement line is obtained, and the algorithm ends. Figure 9 This is a schematic diagram of the measurement results for a typical placenta.

[0063] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0064] Based on the same inventive concept, this application also provides an automatic measurement device for the standard section of the placental umbilical cord insertion site based on a multi-task model, for implementing the aforementioned automatic measurement method for the standard section of the placental umbilical cord insertion site based on a multi-task model. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the automatic measurement device for the standard section of the placental umbilical cord insertion site based on a multi-task model provided below can be found in the limitations of the automatic measurement method for the standard section of the placental umbilical cord insertion site based on a multi-task model described above, and will not be repeated here.

[0065] In one embodiment, such as Figure 10 As shown, an automatic measurement device for the standard section of the placental umbilical cord insertion site based on a multi-task model is provided, comprising: The first module 110 is used to acquire the placenta image to be tested and the YOLO multi-task model for identifying the standard cross-section of the placenta umbilical cord insertion site.

[0066] The second module 120 is used to input the placental image into the YOLO multi-task model to identify the detection boxes and corresponding confidence scores of each target structure in the cross-section of the placental umbilical cord insertion port, as well as the placental mask of the cross-section of the placental umbilical cord insertion port; the target structures include the placenta and the placental umbilical cord insertion port.

[0067] The third module 130 is used to analyze the detection frame of the target structure and the corresponding confidence level to determine the standard section of the placental umbilical cord insertion site and the corresponding placental mask.

[0068] The fourth module 140 is used to obtain the central axis, outline, and sector boundary of the original image based on the placental mask of the standard section of the placental umbilical cord insertion orifice.

[0069] The fifth module 150 is used to analyze and measure the central axis, contour line and sector boundary of the original image based on the mask to obtain the optimal placental measurement line.

[0070] Each module in the aforementioned automatic measurement device for the standard cross-section of the placental umbilical cord insertion site based on a multi-task model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0071] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an automatic measurement method for the standard cross-section of the placental umbilical cord insertion site based on a multi-task model. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0072] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0073] In one embodiment, a computer device is provided. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0074] In one embodiment, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0075] In one embodiment, a computer program product is also provided. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described above.

[0076] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0077] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An automatic measurement method for the standard section of the placental umbilical cord insertion site based on a multi-task model, characterized in that, include: Acquire the placenta image to be tested, and the YOLO multi-task model for identifying the standard cross-section of the placental umbilical cord insertion site; The placental image is input into the YOLO multi-task model to identify the detection boxes and corresponding confidence scores of each target structure in the cross-section of the placental umbilical cord insertion port, as well as the placental mask of the cross-section of the placental umbilical cord insertion port; the target structures include the placenta and the placental umbilical cord insertion port. Based on the detection frame of the target structure and the corresponding confidence level, the standard section of the placental umbilical cord insertion site and the corresponding placental mask are determined by analysis. Based on the placental mask of the standard section of the placental umbilical cord insertion opening, obtain the central axis, outline, and sector boundary of the original image. The optimal placental measurement line is obtained by analyzing and measuring the central axis, contour line, and sector boundary of the original image based on the mask.

2. The method according to claim 1, characterized in that, The YOLO multi-task model is obtained by adding a segmentation task head to the YOLO detection model and training it using the YOLO format detection dataset and segmentation dataset. The detection dataset is labeled with two structures: the placenta and the placental umbilical cord insertion site. The segmentation dataset is labeled with a placental mask.

3. The method according to claim 1, characterized in that, Based on the detection frame of the target structure and the corresponding confidence level, analysis is performed to determine the standard section of the placental umbilical cord insertion site and the corresponding placental mask, including: Based on the detection frame of the target structure, determine whether the placenta and placental umbilical cord insertion site are present in the cross-section. If neither is present, the score is set to 59. If only one structure is present, the score is set to 59 × structure confidence. If both structures are present, the score is calculated based on the corresponding confidence and the weight ratio of the placenta and placental umbilical cord insertion site. If the confidence of a target structure is less than 0.3, the total score is reduced by 30. If the confidence of the umbilical cord insertion site is less than 0.45, the score is reduced by 20. If the confidence of the placental umbilical cord insertion site is less than 0.8, the score is adjusted to 75. Based on the scores of each placental umbilical cord insertion site cross-section, determine the standard cross-section of the placental umbilical cord insertion site and the corresponding placental mask.

4. The method according to claim 1, characterized in that, Based on the placental mask of the standard section of the placental umbilical cord insertion site, obtain the central axis, outline, and sector boundary of the original image, including: Perform a closing operation on the placental mask to find the area with the largest outer contour as the placental area to be measured, and set the remaining areas to 0 to obtain mask A; count the number of pixels in the found placental area x1 and fill it according to the outer contour to obtain mask B; invert mask A and perform a AND operation with mask B to obtain the internal hole mask C. Count the number of pixels in the internal hole mask C by x2. If x2*W<=x1, the internal hole is a hole that is incorrectly segmented by the model. Keep mask B as the final mask to be tested. W is a set coefficient. Otherwise, it means that the placenta to be tested is a ring placenta. Invert mask C and AND it with mask B to obtain the final mask to be tested. The central axis, contour lines, and sector boundaries of the original image are determined based on the final mask analysis.

5. The method according to claim 4, characterized in that, Determining the central axis includes: Extract the skeleton line mask using the morphology.skeletonize() function, then obtain the skeleton line point set using the cv2.findContours() function, and extract the branches of the skeleton line point set in sequence. Select the longest branch based on the length of each branch and the distance between the endpoints of each branch, and merge the second longest branch to obtain the main axis of the skeleton line. Use the cv2.approxPolyDP() method to simplify the main axis of the skeleton line. After simplification, add 1 to 3 points based on the pixel distance between two points to obtain the central axis.

6. The method according to claim 4, characterized in that, Determining the boundaries of the sector region in the original image includes: Hough transform is used to detect straight lines. The threshold is the difference between the farthest points in the vertical direction of the mask. The angle on the left is set to 12° to 72° and on the right to 108° to 164°, resulting in two boundary lines. The symmetry of the two lines is compared. If the angle deviation is >3°, the line with the fewer votes is filtered according to the number of votes in the Hough transform to obtain the left and right fan-shaped boundary lines. These lines are marked in the blank array of the original image size and expanded by 2 pixels to obtain the marked area M1. The dilation segmentation mask treats a 7×7 region near the dilated contour points as a potential boundary region. Based on the original image size h×w, if the mask bounding box is in the lower half of the image, the portion above 0.95h is excluded, and vice versa, the portion above 0.85h is excluded, and this is recorded as the potential lower boundary region. If the left and right spacing of the bounding box is greater than 0.4w, the region between 0.2w and 0.8w is excluded, and vice versa, the region between 0.3w and 0.7w is excluded, and this is recorded as the potential left and right boundary regions. The sum of the pixel values ​​of the original image within the 7×7 range of the corresponding point coordinates of the two potential regions is determined. If it is less than 10, it is marked as a boundary region, resulting in the marked region M2 after dilation. Merge the marked regions M1 and M2 to obtain the boundary of the sector region.

7. The method according to any one of claims 1 to 6, characterized in that, The optimal placental measurement line is obtained by analyzing and measuring the central axis, contour lines, and fan-shaped region boundaries of the mask, including: The inflection point of the centerline is checked based on the mask to confirm whether there is a large fold or annular placenta. If a large fold or annular placenta exists, the multi-segment centerline obtained after processing the large fold or annular placenta is processed to obtain the optimal placenta measurement line. If it is a long strip placenta without large corners, the start and end points are corrected and the ordinary placenta is measured to obtain the optimal placenta measurement line.

8. The method according to claim 7, characterized in that, The inflection point is checked based on the centerline of the mask to confirm the presence of large inverted or annular placentas, including: The cv2.approxPolyDP() function is used to sparse the points on the central axis to obtain the position of the inflection point. If there are fewer than 3 inflection points, it means that the placenta is long and narrow. If there are 3 or more inflection points, the angle between the two sides of each simplified point is calculated: if it is less than 100°, it is recorded as an inflection point, and the angle list of the inflection point and each simplified point is recorded. If no inflection point is found, it means that the placenta is long and narrow. Based on the inflection point and the angle list of each simplified point, the centerline is divided into multiple segments according to the simplified points. First, the index of the centerline point corresponding to the simplified point is found. Then, the centerline is traversed from beginning to end. Points with a pixel distance greater than 5 are included in the centerline of each segment. If the number of points in each segment is greater than 3, it is recorded as a segment. The cv2.pointPolygonTest() function is used to determine the distance between each point in the first and last segment and the hub line. Points with a distance less than 12 are removed. Segments with less than 3 points are filtered out. The point set of each segment divided according to the inflection point, the simplified point set, the angle of the corresponding edge of each simplified point, and the centerline are recorded.

9. The method according to claim 8, characterized in that, The multi-segment midline obtained after processing the large fold and circumferential placenta is processed to obtain the optimal placental measurement line, including: If it is a large folded placenta, delete the points that are less than 25px away from the inflection point, and return the set of segmented axis points with a length greater than 3, the mask outline, and the "folded placenta exists" flag; If it is a ring-shaped placenta, take the previously recorded inner ring mask and coordinates, and divide the ring into four regions: upper left, upper right, lower left, and lower right, based on the midpoint of the inner mask. Count the regions corresponding to the point sets of each segment, find the region with the most points, and extract the mask of the corresponding region. Calculate the distance between the first and last points of the axis in each segment, sort them by distance, and then calculate the distance for each segment separately. First, count the number of axis points. If there are 2 axis points, directly calculate the orthogonal vector based on the coordinates of the two points. Draw a straight line based on the orthogonal vector with the two points as the starting point to obtain the intersection point with the contour. Record the longest measurement line. Calculate the orthogonal vector of each line segment. For a point X, if it is the first and last point, the direction is recorded as the orthogonal vector of the line segment. Otherwise, take the sum of the orthogonal vectors of two adjacent line segments as the direction vector. Draw a straight line based on the direction vector and point X, and record the intersection point, measure the line length and the total length. Sort the obtained measurement lines by length, set the outlier threshold multiplier to 1.4, and the outlier threshold to 1.4 × average length. Traverse the measurement lines from largest to smallest. If two points of the measurement line are not on the boundary mask and the length is not greater than the extreme value threshold, the traversal ends and the measurement line is taken as the placental measurement line of this segment. The longest segment axis is designated as the main measurement line. The comparison threshold for other segments is set to 1.

4. If the number of points in the point set is equal to the number of points in the main segment axis, the comparison threshold is set to 1. The outlier threshold is set to 2.

5. The measurement lines of the remaining segments are iterated sequentially, checking if they intersect with the main measurement line. If they intersect, the main measurement line is retained, and the iteration continues. If they do not intersect, the comparison threshold multiplied by the main measurement line and the remaining measurement lines are compared. If the comparison threshold multiplied by the main measurement line is greater than the remaining measurement lines, the main measurement line is not replaced. If the length of the remaining measurement lines is greater than the outlier threshold multiplied by the main measurement line, the main measurement line is not replaced; otherwise, the main measurement line is replaced. The iteration continues with the measurement lines of the next segment. The optimal placental measurement line is obtained after this process.

10. The method according to claim 8, characterized in that, Optimal placental measurement lines were obtained by measuring a normal placenta, including: Calculate the angle of each axis point, set the angle threshold to 180°, set a maximum of ten loops, and decrease the angle threshold by 4.5 each time until the number of points with angles greater than the threshold is greater than 60% of the total number of points, and record the corresponding points; calculate the orthogonal vector and line segment length of each axis point, and calculate the cumulative length of each point, the total length of the line segment, and the average length; Calculate the average length of the measurement lines and remove those shorter than 0.6 × the average length. After removal, count the remaining measurement lines and remove the first and last measurement lines based on the number of points. If the number of points is greater than 18, remove the first and last 4 points; if the number of points is greater than 15, remove the first and last 3 points; if the number of points is greater than 9, remove the first and last 2 points; if the number of points is greater than 6, remove the first and last 1 point. After removal, recalculate the average length of the measurement lines and set the comparison threshold to 1.

4. The critical value is the comparison threshold × the average value. If the length of the longest measurement line exceeds the critical value, it is considered that the longest measurement line is abnormal. Sort the remaining measurement lines in descending order of length and select the measurement line that is not greater than the critical value as the longest measurement line, which is recorded as the optimal placental measurement line.

Images