A method and device for spleen ultrasound image completion based on morphology

Abstract

The application discloses a kind of based on morphological spleen ultrasound image completion method and device, the method includes: straight line detection is carried out to spleen mask image, and the curvature difference of the two end points of each straight line segment is calculated as curvature variation value;For each straight line segment, if the length of straight line segment and curvature variation value are judged to exceed corresponding preset judgment threshold, then straight line segment is regarded as occlusion straight line segment;Determine long diameter upper point and long diameter lower point based on spleen mask image;Long diameter upper point, long diameter lower point, the upper end point and lower end point of occlusion straight line segment, and the intermediate point of long diameter lower point and lower end point are regarded as basic point, and optimal fitting ellipse is obtained by solving, and ellipse base edge and multiple expansion points are determined, curve fitting is carried out, and optimal fitting curve is obtained, and then the spleen mask image after completion is determined.The method is more accurate by intelligently identifying rib occlusion, curve fitting is carried out in combination with morphological parameters, and the spleen mask image after completion is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to a morphological-based spleen ultrasound image completion method and device. BACKGROUND

[0002] In the field of ultrasound scanning robots, ultrasound imaging of the spleen is crucial for clinical diagnosis of ultrasound scanning. However, in actual operation, due to the presence of ribs, the complete ultrasound image of the spleen is often blocked, making it difficult to obtain high-quality image data. This problem not only reduces the diagnostic value of the image and affects the accurate assessment of the spleen and its surrounding structures by the ultrasound robot, but also limits its performance in automatic scanning, lesion detection, and navigation positioning.

[0003] To solve this problem, some existing methods attempt to use morphological operations to complete the blocked area. For example, algorithms based on morphological dilation and erosion operations can extend the known spleen boundary to infer the possible internal structure, and some methods use deep learning models to train and predict the missing part. SUMMARY

[0004] To more accurately complete the spleen image, the present application provides a morphological-based spleen ultrasound image completion method and device.

[0005] In a first aspect, the present application provides a morphological-based spleen ultrasound image completion method, which can include:

[0006] Performing straight line detection on the obtained spleen mask image to confirm at least one straight line segment;

[0007] Calculating the difference between the curvatures of the two endpoints of each straight line segment as the curvature change value;

[0008] For each straight line segment, if the length and curvature change value of the straight line segment are both determined to exceed the corresponding preset judgment threshold, the straight line segment is regarded as a blocked straight line segment;

[0009] Determining the upper and lower points on the long diameter based on the spleen mask image;

[0010] Taking the upper and lower points on the long diameter, the upper and lower endpoints of the blocked straight line segment, and the intermediate point between the lower point on the long diameter and the lower endpoint as the base points, an optimal fitting ellipse is obtained;

[0011] Taking the elliptical arc between the upper and lower endpoints on the optimal fitting ellipse as the elliptical base edge;

[0012] Based on the base points and the elliptical base edge, a plurality of expansion points are determined on the spleen mask image;

[0013] According to the base point and the plurality of expansion points, curve fitting is performed to obtain an optimal fitting curve, and the spleen mask image is combined to obtain a completed spleen mask image.

[0014] In one or some optional embodiments of the present application, the plurality of expansion points are determined on the spleen mask image based on the base point and the elliptical base edge, including:

[0015] An intermediate interpolation point between the long-diameter point and the upper end point is taken as a first expansion point;

[0016] An intermediate interpolation point between the first expansion point and the upper end point is taken as a second expansion point;

[0017] A midpoint of the elliptical base edge is taken as a third expansion point;

[0018] A point of the elliptical base edge at a distance of 1 / 4 arc length from the lower end point is taken as a fourth expansion point;

[0019] An intermediate point between the long-diameter lower point and the lower end point and an intermediate interpolation point between the lower end point and the long-diameter lower point are taken as a fifth expansion point and a sixth expansion point, respectively.

[0020] In one or some optional embodiments of the present application, after the intermediate interpolation point between the first expansion point and the upper end point is taken as the second expansion point, the method further includes:

[0021] A first slope between the long-diameter point and the first expansion point and a second slope between the first expansion point and the upper end point are calculated;

[0022] A difference between the first slope and the second slope is calculated;

[0023] If the difference exceeds a preset expansion threshold, a slope average value is calculated according to the long-diameter point, the first expansion point, the second expansion point and the upper end point;

[0024] A distance between the upper end point and the second expansion point is taken as a reference distance;

[0025] An extension line is drawn from the upper end point to one side of the elliptical base edge according to the slope average value, and an expansion expansion point is determined on the extension line according to the reference distance.

[0026] In one or some optional embodiments of the present application, the intermediate interpolation point between the long-diameter point and the upper end point is taken as the first expansion point, including:

[0027] Contour extraction is performed on the spleen mask image to obtain a spleen contour;

[0028] calculating the average of the coordinates of the upper end point and the upper point on the long diameter in the X-axis direction;

[0029] finding a corresponding intermediate interpolation point on the spleen contour according to the average of the coordinates in the X-axis direction as a first extended point.

[0030] In one or some optional embodiments of the application, the intermediate point between the lower point and the lower end point is obtained by the following method:

[0031] contour extraction is performed on the spleen mask image to obtain a spleen contour, and the spleen contour includes a plurality of sequentially arranged points;

[0032] determining the serial numbers of the lower point and the lower end point in the spleen contour;

[0033] calculating the serial number of the intermediate point in the spleen contour according to the serial numbers of the lower point and the lower end point to obtain the intermediate point.

[0034] In one or some optional embodiments of the application, the determination of the upper point and the lower point on the long diameter based on the spleen mask image comprises:

[0035] contour extraction is performed on the spleen mask image to obtain a spleen contour;

[0036] calculating a minimum convex polygon containing the spleen contour to obtain a contour convex hull;

[0037] calculating the two points farthest apart on the contour convex hull based on a rotating caliper algorithm, taking the point above the spleen mask image as the upper point on the long diameter, and taking the point below the spleen mask image as the lower point on the long diameter.

[0038] In one or some optional embodiments of the application, the spleen mask image is obtained by the following method:

[0039] using a preset spleen image segmentation model to perform image segmentation on the obtained spleen ultrasound image to obtain a spleen mask image.

[0040] In a second aspect, the application provides a spleen ultrasound image completion device based on morphology, which can include:

[0041] a straight line detection module for performing straight line detection on the obtained spleen mask image to confirm at least one straight line segment;

[0042] a curvature calculation module for calculating the difference between the curvatures of the two end points of each straight line segment as a curvature change value;

[0043] The occlusion judgment module is configured to, for each straight line segment, if it is judged that the length and the curvature change value of the straight line segment both exceed corresponding preset judgment thresholds, take the straight line segment as an occlusion straight line segment.

[0044] The first determination module is configured to determine an upper point and a lower point on the long diameter based on the spleen mask image.

[0045] The ellipse fitting module is configured to take the upper point and the lower point on the long diameter, upper and lower end points of the occlusion straight line segment, and a middle point between the lower point on the long diameter and the lower end point as base points, and solve an optimal fitting ellipse.

[0046] The second determination module is configured to take an elliptical arc between the upper and lower end points of the optimal fitting ellipse as an elliptical base edge.

[0047] The third determination module is configured to determine a plurality of expansion points on the spleen mask image based on the base points and the elliptical base edge.

[0048] The curve fitting module is configured to perform curve fitting based on the base points and the plurality of expansion points to obtain an optimal fitting curve, and combine the spleen mask image to obtain a completed spleen mask image.

[0049] In a third aspect, a computer readable storage medium is provided, which stores a computer program / instruction, and the computer program / instruction is executed by a processor to implement the morphological spleen ultrasound image completion method as described above.

[0050] In a fourth aspect, a computer program product is provided, which includes a computer program / instruction, and the computer program / instruction is executed by a processor to implement the morphological spleen ultrasound image completion method as described above.

[0051] In a fifth aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the morphological spleen ultrasound image completion method as described above.

[0052] The above technical solution provided by the embodiments of the present application has at least the following beneficial effects:

[0053] The embodiment of the present application provides a morphological-based spleen ultrasound image completion method, which comprises the following steps: performing straight line detection on a spleen mask image, confirming straight line segments in the image, and calculating the curvature change value of the two endpoints of each straight line segment to determine whether there is an occlusion phenomenon, if the occlusion is detected, determining the occlusion straight line segment, then extracting the upper point and the lower point of the long diameter of the spleen mask image, and taking the upper point and the lower point of the long diameter and the endpoints and the intermediate points of the occlusion straight line segment as the base points, solving the optimal fitting ellipse, determining the ellipse base edge, determining a plurality of expansion points in the image, finally, performing curve fitting according to the base points and the expansion points to obtain the optimal fitting curve, so as to restore the occluded spleen region, and finally obtain the completed spleen mask image. The present method is based on the spleen mask image, can intelligently identify whether there is a rib occlusion phenomenon, thereby avoiding unnecessary processing steps, improving the completion efficiency and reliability, determining a plurality of base points for the spleen mask image to be completed, adopting the ellipse base edge fitting technology, ensuring that the completion edge is highly consistent with the actual anatomical structure, and in addition, in order to further improve the completion effect, the present application also introduces a plurality of expansion points based on the ellipse base edge as the optimization morphological parameter, ensures the consistency and reliability between different patients, so that the completed spleen mask image not only can retain the original details, but also can effectively restore the lost key information due to the rib occlusion, and provides a more reliable basis for subsequent medical analysis.

[0054] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and the appended drawings.

[0055] The technical solutions of the present application will be further described in detail below with the help of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0056] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application together with the embodiments thereof, and explain the present application, and do not constitute a limitation on the present application. In the drawings:

[0057] Figure 1 The flowchart of the morphological-based spleen ultrasound image completion method provided by the embodiment of the present application is shown in the figure.

[0058] Figure 2 The straight line detection effect diagram of the spleen mask image provided by the embodiment of the present application is shown in the figure.

[0059] Figure 3 The example diagram of the rotating caliper algorithm provided by the embodiment of the present application is shown in the figure.

[0060] Figure 4An example diagram of an elliptical base edge provided for an embodiment of the present application;

[0061] Figure 5 A process diagram of a completed spleen mask image obtained from a spleen ultrasound image provided for an embodiment of the present application;

[0062] Figure 6 A fitting curve diagram obtained by cubic non-uniform B-spline curve fitting provided for an embodiment of the present application;

[0063] Figure 7 A fitting curve diagram obtained by quartic non-uniform B-spline curve fitting provided for an embodiment of the present application;

[0064] Figure 8 A fitting curve diagram obtained by quintic non-uniform B-spline curve fitting provided for an embodiment of the present application;

[0065] Figure 9 A structure diagram of a spleen ultrasound image completion device based on morphology provided for an embodiment of the present application. DETAILED DESCRIPTION

[0066] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0067] The inventors found that some existing methods attempt to utilize morphological operations to complete the occluded regions. For example, algorithms based on morphological dilation and erosion operations can extend the known spleen boundary and infer the possible internal structure, however these methods are susceptible to image noise and local feature distortion, and may not accurately recover complex morphological occluded regions. Some other methods employ deep learning models for training and predicting the missing parts, although they can achieve good results in some cases, they require a large amount of labeled data and computing resources, and may lack sufficient flexibility and accuracy when dealing with images such as spleen images that have strong anatomical structure features. Therefore, there is still room for improvement in accuracy, robustness and computational efficiency in the prior art. Based on this, the inventors have made further research and development, and made the present application, providing a spleen ultrasound image completion method and device based on morphology.

[0068] Embodiment one

[0069] In an embodiment of the present application, a spleen ultrasound image completion method based on morphology is provided, as shown in Figure 1 the method can include the following steps S101-S108:

[0070] S101: Perform straight line detection on the acquired spleen mask image, and confirm at least one straight line segment.

[0071] S102: Calculate the difference between the curvatures of the two endpoints of each straight line segment as a curvature change value.

[0072] S103: For each straight line segment, if the length of the straight line segment and the curvature change value are both greater than the corresponding preset judgment threshold, the straight line segment is regarded as an occlusion straight line segment.

[0073] S104: Determine the upper and lower points on the long diameter based on the spleen mask image.

[0074] S105: Take the upper and lower points on the long diameter, the upper and lower endpoints of the occlusion straight line segment, and the intermediate point between the lower point on the long diameter and the lower endpoint as the base points, and solve the optimal fitting ellipse.

[0075] S106: Take the elliptical arc between the upper and lower endpoints on the optimal fitting ellipse as the elliptical base edge.

[0076] S107: Based on the base points and the elliptical base edge, determine a plurality of expansion points on the spleen mask image.

[0077] S108: According to the base points and the plurality of expansion points, perform curve fitting to obtain an optimal fitting curve, and combine the spleen mask image to obtain a completed spleen mask image.

[0078] The embodiment of the present application provides a kind of based on morphology's spleen ultrasound image completion method, this method is by carrying out straight line detection to spleen mask image, confirm the straight line section in image, and the curvature variation of each straight line section two end point is calculated, to judge whether there is shielding phenomenon, if detecting shielding, determine the straight line section of shielding, then, by the point of long diameter upper point and long diameter lower point extracted to spleen mask image, and long diameter upper point and long diameter lower point and the end point and intermediate point of straight line section of shielding are as basic point, solve optimal fitting ellipse, determine the ellipse base edge, determine multiple expansion points in image, finally, according to these basic points and expansion points carry out curve fitting, obtain optimal fitting curve, to recover the spleen area that is shielded, finally obtain the spleen mask image after completion.This method is based on spleen mask image, can intelligently identify whether there is rib shielding phenomenon, to avoid unnecessary processing steps, improve completion efficiency and reliability, for the spleen mask image that needs to be completed, determine multiple basic points, adopt ellipse base edge fitting technology, ensure that the completion edge is highly consistent with actual anatomical structure, in addition, in order to further improve the completion effect, the present application also introduces multiple expansion points based on ellipse base edge as optimization morphological parameters, ensure the consistency and reliability between different patients, so that the spleen mask image after completion can not only retain original details, but also effectively recover the key information lost due to rib shielding, provide more reliable basis for subsequent medical analysis.The above step S101, the straight line detection is carried out to the obtained spleen mask image, and at least one straight line section is confirmed.

[0079] Specifically, the cv2.HoughLines function or the cv2.HoughLinesP function in OpenCV (Open Source Computer Vision Library) can be used to perform straight line detection in the spleen mask image to obtain all straight line sections in the spleen mask image.

[0080] In a specific embodiment, the spleen mask image and all corresponding straight line sections are as shown in FIG. Figure 2 The white part in the figure is the spleen region, and the four red straight lines are all straight line sections in the spleen mask image.

[0081] In the embodiment of the present application, the spleen mask image can be realized based on an image segmentation algorithm. Specifically, a pre-set spleen image segmentation model can be used to perform image segmentation on the obtained spleen ultrasound image to obtain the spleen mask image.

[0082] Those skilled in the art can select a suitable neural network according to the detailed description of the prior art to obtain the pre-set spleen image segmentation model. The training process can specifically include:

[0083] First, collect the spleen ultrasound images and label the contours of the spleen in the spleen ultrasound images respectively, and obtain the spleen contour dataset after preprocessing.

[0084] Second, select a suitable neural network model as the initial spleen image segmentation model, such as U-Net model, SegNet model, etc.

[0085] Third, divide the spleen contour dataset into training set and test set.

[0086] Fourth, define the loss function (such as cross-entropy loss function, mean square error loss function, etc.), optimization algorithm (such as Adam, SGD, etc.) and the like.

[0087] Fifth, use the training set of the spleen contour dataset to train the initial spleen image segmentation model, and obtain the trained spleen image segmentation model.

[0088] Repeat the above image segmentation model training process until the preset condition is met, stop training, and obtain the preset spleen image segmentation model. The preset condition can be set to reach a fixed number of iterations, the accuracy reaches a threshold, the accuracy does not change within a preset number of iterations, etc. It can not be specifically limited.

[0089] In the above step S102, the difference between the curvatures of the two end points of each straight line segment is calculated as the curvature change value.

[0090] Specifically, it can be that the spleen mask image is first profiled to obtain the spleen contour.

[0091] Then, the curvature calculation formula shown in the following formula 1 is used to calculate the curvature of the two end points of each straight line segment in combination with the spleen contour.

[0092]

[0093] In the formula, x and y are the coordinates of the end points, x' and y' are the first derivatives of the spleen contour at the end points, and x" and y" are the second derivatives of the spleen contour at the end points.

[0094] The difference between the curvatures of the two end points of each straight line segment is calculated as the curvature change value of the straight line segment.

[0095] In the above step S103, for each straight line segment, if the length and the curvature change value of the straight line segment are both greater than the corresponding preset judgment threshold, the straight line segment is regarded as a shielding straight line segment.

[0096] Specifically, for each straight line segment, when the length and the curvature change value of the straight line segment are both greater than the corresponding preset judgment threshold, it is judged that there is a shielding phenomenon at the straight line segment, and the straight line segment is regarded as a shielding straight line segment.

[0097] In the embodiments of the present application, the steps S102-S103 described above can accurately identify edge distortion in the image caused by occlusion by calculating the curvature of the two endpoints of each straight line segment and evaluating the shape characteristics of the straight line segment according to the curvature change value. By setting the length and curvature change value threshold, it can be accurately determined which straight line segment may be affected by occlusion, thereby accurately extracting the occluded straight line segment. Not only the identification accuracy of the occlusion phenomenon is improved, but also unnecessary processing of irrelevant areas in the image is avoided, ensuring the preservation and recovery of details in the completion process.

[0098] In the step S104 described above, the upper point on the long diameter and the lower point on the long diameter are determined based on the spleen mask image. Specifically, the following steps S1041-S1043 are included:

[0099] S1041: Contour extraction is performed on the spleen mask image to obtain a spleen contour.

[0100] Specifically, the cv.findContours function in OpenCV (Open Source Computer Vision Library) can be used to perform contour extraction on the spleen mask image to obtain the spleen contour.

[0101] S1042: A minimum convex polygon containing the spleen contour is calculated to obtain a contour convex hull.

[0102] Specifically, the cv.convexHull function in OpenCV can be used to process the spleen contour to calculate the minimum convex polygon (Convex Hull) of the contour, thereby obtaining the contour convex hull.

[0103] S1043: The two points farthest apart on the contour convex hull are calculated based on the rotating caliper algorithm, and the point located above the spleen mask image is taken as the upper point on the long diameter, and the point located below the spleen mask image is taken as the lower point on the long diameter.

[0104] Specifically, the two points farthest apart on the convex hull of the contour can be calculated using the Rotating Calipers algorithm. The Rotating Calipers algorithm can find two points on the convex hull such that the distance between the two points is the largest. The basic principle of the Rotating Calipers algorithm is to traverse the edges on the convex hull clockwise, and for each edge, find the point farthest from it. As the edge rotates, the corresponding farthest point also changes. First, select an arbitrary point i on the convex hull as the starting point, and then traverse the other points on the convex hull to calculate the farthest point j corresponding to each edge. Next, enumerate each edge (i, i+1) and check whether the distance from point j+1 to the edge is greater than the distance from point j to the edge. If so, update the farthest point to point j+1; otherwise, point j is the farthest point for this edge. To determine the distance from points j and j+1 to the edge, the vector cross product can be used to calculate the area of the triangle with the same base and different height. An example of the Rotating Calipers algorithm is shown in FIG. 1, where the white part is the spleen region, the red line segment is the convex hull of the contour, and the points i, i+1, j, and j+1 correspond to the points i, i+1, j, and j+1 in the basic principle described above. The distance between edge (i, i+1) and point j is the farthest. Figure 3

[0105] Through this algorithm, the two points farthest apart on the convex hull of the contour can be finally found. Then, according to the upper and lower position relationship of the spleen image, the two points are marked as the upper point and the lower point of the long diameter, respectively.

[0106] In the above step S105, the upper point of the long diameter, the lower point of the long diameter, the upper end point and the lower end point of the shielding straight line segment, and the intermediate point between the lower point of the long diameter and the lower end point are taken as the base points, and the optimal fitting ellipse is obtained by solving.

[0107] Specifically, the upper point of the long diameter, the lower point of the long diameter, the upper end point and the lower end point of the shielding straight line segment, and the intermediate point between the lower point of the long diameter and the lower end point are taken as the base points, and the optimal fitting ellipse is obtained by solving using the LM (Levenberg-Marquardt) algorithm.

[0108] In the embodiments of the present application, the inventors found that in practical applications, the contour of the upper end of the spleen is often relatively straight, and therefore it is not suitable to add extra base points at the upper end of the spleen, which can easily lead to failure of fitting the ellipse. At the same time, the optimal fitting ellipse obtained by fitting the five base points selected in the present method is also more in line with the morphological characteristics of the spleen.

[0109] The intermediate point between the lower point of the long diameter and the lower end point is determined through the following steps S1051-S1053:

[0110] S1051: Contour extraction is performed on the spleen mask image to obtain a spleen contour. The spleen contour includes a plurality of sequentially arranged points.

[0111] ​Specifically, the contour extraction can be performed on the spleen mask image using a cv.findContours function in OpenCV (Open Source Computer Vision Library) to obtain a spleen contour. The spleen contour includes a plurality of sequentially arranged discrete points.

[0112] S1052: Determine the sequence numbers of the lower point of the long diameter and the lower end point in the spleen contour.

[0113] Specifically, the discrete points in the spleen contour can be traversed to find the coordinates of the lower point of the long diameter and the lower end point, and then the sequence numbers of the two points in the spleen contour are determined.

[0114] S1053: According to the sequence numbers of the lower point of the long diameter and the lower end point, the sequence number of the intermediate point in the spleen contour is calculated to obtain the intermediate point.

[0115] Specifically, according to the sequence numbers of the lower point of the long diameter and the lower end point in the spleen contour, the sequence number of the intermediate point between the two points is calculated, and the coordinates of the intermediate point are extracted from the sequence of discrete points of the spleen contour as the intermediate point of the lower point of the long diameter and the lower end point.

[0116] In order to facilitate those skilled in the art to understand the present scheme, the basic principle of the LM algorithm is briefly introduced herein in combination with the base points in step S105:

[0117] First, a general quadratic equation of the ellipse to be solved is established. When B 2 -4AC<0, the equation describes an ellipse. The quadratic equation is shown in the following formula 2:

[0118] Ax 2 +Bxy+Cy 2 +Dx+Ey+F=0 Formula 2

[0119] In the formula, A, B, C, D, E, and F are parameters of the ellipse to be solved.

[0120] According to the five base points, the residual error function is defined as shown in the following formula 3:

[0121]

[0122] In the formula, γ i (β) represents the residual error value of the i-th base point, β represents the fitting parameter of the ellipse to be solved, and i is the sequence number of the base point, i=1,2,...,5.

[0123] Next, the goal is to solve the optimal fitting parameters of the ellipse by minimizing the sum of squares of the residual errors of all base points. Specifically, the minimization objective function is shown in the following formula 4:

[0124]

[0125] In the formula, S(β) represents a minimized objective function, γ i represents a residual value of the i-th base point, β represents an ellipse parameter to be solved, and i is a serial number of the base point, i = 1, 2,..., 5.

[0126] According to the residual function shown in the above formula 3, the partial derivative of the residual with respect to each ellipse parameter is calculated, and the following formula 5 is obtained.

[0127]

[0128] In the formula, S(β) represents a minimized objective function, γ represents the partial derivative of the residual function with respect to the ellipse parameter A, and similarly, represents the partial derivative of the residual function with respect to the ellipse parameters B, C, D, and E, respectively.

[0129] Thus, the partial derivative matrix (Jacobian matrix) J of the residual function with respect to the ellipse parameters is calculated based on the above formula 5, and the partial derivative matrix J is shown in the following formula 6.

[0130]

[0131] In the formula, S(β) represents a minimized objective function, γ represents the partial derivative of the residual value of the i-th base point with respect to the ellipse parameters A, B, C, D, and E, respectively.

[0132] According to the partial derivative matrix J, the augmented matrix linear equation group is constructed as shown in the following formula 7.

[0133]

[0134] In the formula, I is an identity matrix, Δβ is a parameter increment, β represents an update amount of the ellipse parameter in iteration, J is the partial derivative matrix, T represents matrix transposition, is a damping factor.

[0135] Next, by solving the augmented matrix linear equation group shown in the above formula 7, the parameter increment of the ellipse parameter is obtained, and the formula for updating the ellipse parameter is shown in the following formula 8.

[0136] β k+1 = β k + Δβ Formula 8

[0137] In the formula, β k+1 is the updated ellipse parameter obtained after the k-th iteration, β k is the ellipse parameter before the k-th iteration, and Δβ is the parameter increment.

[0138] The iteration process continues until one of the following two conditions is met: one is that the maximum number of iterations is reached, and the other is that the change of the ellipse parameters is small enough, and finally the optimal ellipse parameters A, B, C, D, E, F are obtained to determine the optimal fitting ellipse.

[0139] In the embodiment of the application, the step 5 is used to solve the optimal fitting ellipse by combining the upper point of the long diameter, the lower point of the long diameter, the two end points and the intermediate point of the shielding straight line segment, and using the LM algorithm, which effectively improves the accuracy of the spleen shape fitting. By accurately determining the spleen contour and key points, manual errors are avoided, the accuracy and robustness of the fitting are improved, and the method shows good stability and computational efficiency when processing complex, occluded or irregular shaped spleen images, providing a reliable basis for subsequent analysis.

[0140] In the step S106, the elliptical arc between the upper end point and the lower end point on the optimal fitting ellipse is taken as the elliptical base edge.

[0141] In a specific embodiment, an example diagram of the elliptical base edge is shown in Figure 4 In the diagram, the white part is the spleen region, and five base points are marked, B0 is the upper point of the long diameter, B2 is the upper end point, B3 is the lower end point, B4 is the intermediate point between the lower end point and the lower point of the long diameter, and B5 is the lower point of the long diameter. The red arc on the left side is the elliptical base edge.

[0142] In the step S107, based on the base points and the elliptical base edge, a plurality of expansion points are determined on the spleen mask image. Specifically, the following steps S10701-S10710 are included:

[0143] S10701: The intermediate interpolation point between the upper point of the long diameter and the upper end point is taken as the first expansion point. Specifically, the following steps S107011-S107013 are included:

[0144] S107011: Contour extraction is performed on the spleen mask image to obtain the spleen contour.

[0145] Specifically, the cv.findContours function in OpenCV (Open Source Computer Vision Library) can be used to perform contour extraction on the spleen mask image to obtain the spleen contour. The spleen contour includes a plurality of sequentially arranged discrete points.

[0146] S107012: Calculate the mean value of the coordinates of the upper point of the long diameter and the upper end point in the X-axis direction.

[0147] Specifically, the X-axis coordinates of the upper point of the long diameter and the upper end point can be obtained, and their mean value can be calculated to find a corresponding X-coordinate position in the spleen contour.

[0148] S107013: Find the corresponding middle interpolation point on the spleen contour according to the coordinate mean value in the X-axis direction as the first expansion point.

[0149] Specifically, based on the coordinate mean value in the X-axis direction calculated in the above step S107012, all points in the spleen contour are traversed, and the point in the spleen contour closest to the coordinate mean value is taken as the first expansion point.

[0150] S10702: Take the middle interpolation point between the first expansion point and the upper end point as the second expansion point.

[0151] S10703: Calculate the first slope between the point on the long diameter and the first expansion point, and the second slope between the first expansion point and the upper end point.

[0152] S10704: Calculate the difference between the first slope and the second slope.

[0153] S10705: Determine whether the difference exceeds the preset expansion threshold: if yes, calculate the average slope according to the point on the long diameter, the first expansion point, the second expansion point and the upper end point, and execute steps S10706-S10710; if no, execute steps S10708-S10710.

[0154] Specifically, it can be determined whether the difference exceeds the preset expansion threshold: if yes, it means that the expansion point needs to be expanded outside the spleen region, the slope between each pair of points is calculated according to the coordinates of the point on the long diameter, the first expansion point, the second expansion point and the upper end point, then the average value of these slope values is taken to obtain an average slope, and then steps S10706-S10710 are executed; if no, it means that the spleen region is in the middle of the picture of the spleen ultrasound image, and the expansion point does not need to be expanded outside the spleen region, and steps S10708-S10710 are executed.

[0155] S10706: Take the distance between the upper end point and the second expansion point as the reference distance.

[0156] S10707: Take the upper end point as the base point, draw an extension line to one side of the elliptical base edge according to the average slope, and determine the expansion point outside the expansion according to the reference distance on the extension line.

[0157] S10708: Take the midpoint of the elliptical base edge as the third expansion point.

[0158] S10709: Take the point on the elliptical base edge 1 / 4 arc length away from the lower end point as the fourth expansion point.

[0159] S10710: Take the middle point between the lower end point and the point on the long diameter as the fifth expansion point and the sixth expansion point.

[0160] In the embodiment of the present application, the method of determining the intermediate interpolation point in steps S10702 and S10710 is consistent with step S10701, and thus will not be repeated here.

[0161] In the embodiment of the present application, the selection method of the extension point is determined according to the prior knowledge of the doctor, which can be optimized according to the individual differences of different patients, thereby improving the completion effect. In order to ensure the consistency and reliability of the processing results among different patients, not only the original details in the ultrasound image can be preserved, but also the key information lost due to rib shielding can be effectively recovered. The processed ultrasound image provides a more accurate and reliable basis for subsequent medical analysis, ensuring the accuracy and effectiveness of the diagnosis process.

[0162] In step S108, the curve fitting is performed according to the base points and the plurality of extension points to obtain an optimal fitting curve, and the spleen mask image is combined to obtain a completed spleen mask image.

[0163] Specifically, based on the plurality of base points and the plurality of extension points determined in steps S101-S107, a non-uniform B-spline curve fitting algorithm is used to perform curve fitting to obtain an optimal fitting curve, and the spleen mask image is combined to obtain a completed spleen mask image.

[0164] In a specific embodiment, the process diagram of obtaining the completed spleen mask image from the spleen ultrasound image is as shown in Figure 5 The leftmost side is the spleen ultrasound image, and there is a rib shielding phenomenon on the left side of the spleen ultrasound image, Figure 5 The middle is the spleen mask image obtained from the spleen ultrasound image, Figure 5 The right side is the completed spleen mask image obtained by the spleen mask image through steps S101-S108, wherein the red arc line is the edge of the ellipse base, and the curve shown by the plurality of yellow points is the optimal fitting curve.

[0165] In order to facilitate those skilled in the art to understand the present scheme, the following five base points and a plurality of extension points will be collectively referred to as control points, and the basic principles of the non-uniform B-spline curve fitting algorithm will be briefly introduced based on these control points:

[0166] First, the node interval [u k , u k+1 ] of the parameter u on the curve to be fitted is determined, and according to the definition of the B-spline curve, the mathematical expression of the curve to be fitted is as shown in the following formula 9:

[0167]

[0168] where C(u) represents the curve point of the curve to be fitted at the position corresponding to the parameter u, P i is the i-th control point, N i,d (u) is the basis function value of the i-th control point, i is the control point serial number, d is the degree of the spline, and u is the parameter.

[0169] where the B-spline basis function N i,d (u) in the above formula 9 is calculated by the Cox-de Boor recursive formula shown in the following formula 10:

[0170]

[0171] where N i,d (u) is the basis function value of the i-th control point, i is the control point serial number, d is the degree of the spline, and u is the parameter, u i , u i+d , u i+d+1 , u i+1 is the node in the node vector (knot vector). Wherein, the basis function is only related to d+1 control points in the node interval [u k , u k+1 ], and the index range of these control points is k-d to k.

[0172] The non-uniform B-spline curve fitting algorithm is usually ended when the maximum number of iterations is reached, the fitting error is less than a predetermined threshold, or the fitting result changes very little, and the optimal fitting curve is obtained. Specifically, the algorithm will determine whether to stop and return the final fitting result according to the set convergence conditions, such as the fitting error being lower than a certain threshold or the curve shape no longer changing significantly. This ensures that the fitting process is completed within a reasonable time and can provide accurate curve fitting.

[0173] It should be noted that when the nodes u k and u k+1 are equal, the denominator of the basis function is zero, and the corresponding term is defined as zero at this time, so that this section of the curve degenerates into a point. At this time, the basis function has no effect at this point.

[0174] In a specific embodiment, based on the spleen mask image and the elliptical base edge in the above Figure 4 , after three, four, and five times of non-uniform B-spline curve fitting, respectively, the fitting curve schematic diagram obtained is as shown in Figure 6 , 7, 8, in which five basic points and seven extended points are marked, B0 is the upper point of the long diameter, B2 is the upper end point, B3 is the lower end point, B4 is the middle point between the lower end point and the lower point of the long diameter, B5 is the lower point of the long diameter, C0 is the first extended point, C1 is the second extended point, C2 is the outer extended extended point, C3 is the third extended point, C4 is the fourth extended point, C5 is the fifth extended point, C6 is the sixth extended point, the red arc line is the edge of the elliptical base, and the curve shown by the plurality of yellow points is the corresponding fitting curve. It can be seen that the spline number of the non-uniform B-spline curve fitting algorithm has little effect on the effect of the fitting curve. The larger the spline number, the smoother the curve. A person skilled in the art can determine the maximum iteration number of the non-uniform B-spline curve fitting algorithm according to the smoothness required.

[0175] Example two

[0176] Based on the same inventive concept, the present application also provides a morphological-based spleen ultrasound image completion device, as shown in Figure 9 The device comprises:

[0177] A straight line detection module 101 is configured to perform straight line detection on the obtained spleen mask image and confirm at least one straight line segment.

[0178] A curvature calculation module 102 is configured to calculate the difference between the curvatures of the two end points of each straight line segment as a curvature change value.

[0179] An occlusion judgment module 103 is configured to, for each straight line segment, if it is judged that the length and the curvature change value of the straight line segment both exceed the corresponding preset judgment threshold, the straight line segment is regarded as an occluded straight line segment.

[0180] A first determination module 104 is configured to determine the upper point of the long diameter and the lower point of the long diameter based on the spleen mask image.

[0181] An ellipse fitting module 105 is configured to take the upper point of the long diameter, the lower point of the long diameter, the upper end point and the lower end point of the occluded straight line segment, and the middle point between the lower point of the long diameter and the lower end point as basic points, and solve to obtain an optimal fitting ellipse.

[0182] A second determination module 106 is configured to take the elliptical arc between the upper end point and the lower end point on the optimal fitting ellipse as an elliptical base edge.

[0183] A third determination module 107 is configured to determine a plurality of extended points on the spleen mask image based on the basic points and the elliptical base edge.

[0184] A curve fitting module 108 is configured to perform curve fitting according to the basic points and the plurality of extended points to obtain an optimal fitting curve, and combine the spleen mask image to obtain a completed spleen mask image.

[0185] Embodiment three

[0186] Based on the same inventive concept, the embodiments of the present application further provide a computer readable storage medium, which has stored thereon a computer program / instruction, and the computer program / instruction is executed by a processor to implement the morphological-based spleen ultrasound image completion method as described in the above embodiment one.

[0187] Embodiment four

[0188] Based on the same inventive concept, the embodiments of the present application further provide a computer program product, which comprises a computer program / instruction, and the computer program / instruction is executed by a processor to implement the morphological-based spleen ultrasound image completion method as described in the above embodiment one.

[0189] Embodiment five

[0190] Based on the same inventive concept, the embodiments of the present application further provide a computer device, which comprises a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the morphological-based spleen ultrasound image completion method as described in the above embodiment one.

[0191] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) including computer-usable program code.

[0192] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The means for implementing each flow or multiple flows and / or blocks Figure 1 The means for implementing each flow or multiple flows and / or blocks

[0193] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0194] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0195] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for spleen ultrasound image completion based on morphology, characterized in that, The method comprises the following steps: straight line detection is performed on the obtained spleen mask image to confirm at least one straight line segment; the difference between the curvatures of the two endpoints of each straight line segment is calculated as a curvature change value; for each straight line segment, if the length and the curvature change value of the straight line segment are both greater than the corresponding preset judgment threshold, the straight line segment is regarded as an occlusion straight line segment; a long-diameter upper point and a long-diameter lower point are determined based on the spleen mask image; the long-diameter upper point, the long-diameter lower point, the upper endpoint and the lower endpoint of the occlusion straight line segment, and the intermediate point between the long-diameter lower point and the lower endpoint are taken as basic points to obtain an optimal fitting ellipse; an elliptical arc between the upper endpoint and the lower endpoint on the optimal fitting ellipse is taken as an elliptical base edge; a plurality of expansion points are determined on the spleen mask image based on the basic points and the elliptical base edge; curve fitting is performed according to the basic points and the plurality of expansion points to obtain an optimal fitting curve, and a completed spleen mask image is obtained in combination with the spleen mask image.

2. The method of claim 1, wherein, The determination of the plurality of expansion points on the spleen mask image based on the basic points and the elliptical base edge comprises the following steps: an intermediate interpolation point between the long-diameter upper point and the upper endpoint is taken as a first expansion point; an intermediate interpolation point between the first expansion point and the upper endpoint is taken as a second expansion point; a midpoint of the elliptical base edge is taken as a third expansion point; a point on the elliptical base edge which is 1 / 4 arc length away from the lower endpoint is taken as a fourth expansion point; an intermediate point between the long-diameter lower point and the lower endpoint and an intermediate interpolation point between the lower endpoint and the long-diameter lower point are taken as a fifth expansion point and a sixth expansion point respectively.

3. The method of claim 2, wherein, After the intermediate interpolation point between the first expansion point and the upper endpoint is taken as the second expansion point, the following steps are further included: a first slope between the long-diameter upper point and the first expansion point and a second slope between the first expansion point and the upper endpoint are calculated; the difference between the first slope and the second slope is calculated; if the difference exceeds a preset expansion threshold, an average slope is calculated according to the long-diameter upper point, the first expansion point, the second expansion point and the upper endpoint; the distance between the upper endpoint and the second expansion point is taken as a reference distance; an extension line is drawn from the upper endpoint to the side of the elliptical base edge according to the average slope, and an expansion expansion point is determined on the extension line according to the reference distance.

4. The method of claim 2, wherein, The first expansion point is obtained by the following steps: contour extraction is performed on the spleen mask image to obtain a spleen contour; the average value of the coordinates of the long-diameter upper point and the upper endpoint in the X-axis direction is calculated; the corresponding intermediate interpolation point on the spleen contour according to the average value of the coordinates in the X-axis direction is found as the first expansion point.

5. The method of claim 1, wherein, The intermediate point between the long-diameter lower point and the lower endpoint is obtained by the following steps: contour extraction is performed on the spleen mask image to obtain a spleen contour; the spleen contour includes a plurality of sequentially arranged points; the serial numbers of the long-diameter lower point and the lower endpoint in the spleen contour are determined. According to the sequence numbers of the lower end point and the lower end point, the sequence number of the intermediate point in the spleen contour is calculated to obtain the intermediate point.

6. The method of claim 1, wherein, The determination of the upper end point and the lower end point based on the spleen mask image comprises: contour extraction is performed on the spleen mask image to obtain a spleen contour; a minimum convex polygon containing the spleen contour is calculated to obtain a contour convex hull; the two points farthest apart on the contour convex hull are calculated based on a rotating caliper algorithm, and the point above the spleen mask image is taken as the upper end point, and the point below the spleen mask image is taken as the lower end point.

7. The method of claim 1, wherein, The spleen mask image is obtained in the following manner: a pre-set spleen image segmentation model is used to perform image segmentation on the obtained spleen ultrasound image to obtain a spleen mask image.

8. A morphological-based spleen ultrasound image completion apparatus, characterized by, It comprises: a straight line detection module for performing straight line detection on the obtained spleen mask image to identify at least one straight line segment; a curvature calculation module for calculating the difference between the curvatures of the two end points of each straight line segment as a curvature change value; a shielding judgment module for each straight line segment, if the length and the curvature change value of the straight line segment are determined to exceed the corresponding pre-set judgment threshold, the straight line segment is taken as a shielding straight line segment; a first determination module for determining an upper end point and a lower end point based on the spleen mask image; an ellipse fitting module for taking the upper end point, the lower end point, the upper end point and the lower end point of the shielding straight line segment, and the intermediate point between the lower end point and the lower end point as the base points, and solving to obtain an optimal fitting ellipse; a second determination module for taking the elliptical arc between the upper end point and the lower end point on the optimal fitting ellipse as an elliptical base edge; a third determination module for determining a plurality of expansion points on the spleen mask image based on the base points and the elliptical base edge; a curve fitting module for performing curve fitting according to the base points and the plurality of expansion points to obtain an optimal fitting curve, and combining the spleen mask image to obtain a completed spleen mask image.

9. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the morphological-based spleen ultrasound image completion method of any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program comprises instructions that, when executed by the processor, cause the processor to perform the method of any one of claims 1-9. The processor executes the computer program to implement the morphological-based spleen ultrasound image completion method of any one of claims 1-7.

Images