A method and apparatus for organ surface lesion detection based on deformation field

Abstract

The application provides a method and device for organ surface lesion detection based on a deformation field, and the method comprises the following steps: acquiring a medical image sequence, and performing segmentation on a target organ based on an organ segmentation model; based on the segmentation result of the target organ, performing deformation field initialization, and deforming a twisted target organ contour into a straight line; determining a lesion contour, pre-processing an image, dividing a training set and a test set, training a lesion detection model on the training set, and verifying the lesion detection model on the test set; and detecting organ surface lesions by using the lesion detection model. The performance of the lesion detection model on the organ surface is improved, and the detection performance of the lesion with a thin strip shape attached to the organ surface is improved in particular.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for detecting lesions on the surface of organs based on deformation fields. Background Technology

[0002] With the development of medical imaging technology, techniques such as computed tomography (CT) and magnetic resonance imaging (MRI) are widely used in the analysis and diagnosis of organs and lesions. For the detection of lesions on the organ surface, such as perihepatic metastases of ovarian cancer, these lesions often appear as thin strips attached to the surface in each scan image, posing a significant challenge to detection models. Meanwhile, the location of the organ can provide prior knowledge for lesion detection. How to utilize this prior knowledge to improve the detection efficiency of lesions on the organ surface is an urgent research question.

[0003] Current object detection algorithms can be divided into anchor-based object detection and anchor-free object detection. Both methods show a decrease in detection efficiency for slender objects. In particular, when the slender object is located diagonally in the image, the rectangular detection box-based model introduces a large amount of background noise during training, affecting the model's performance.

[0004] Existing algorithms for detecting slender objects are relatively few, and most studies focus on adaptive feature extraction based on attention mechanisms. However, these algorithms cannot fully utilize the prior anatomical information in medical images and cannot perform targeted analysis of specific lesions. Summary of the Invention

[0005] The present invention aims to provide a method and apparatus for detecting organ surface lesions based on deformation fields to overcome or at least partially solve the above-mentioned problems.

[0006] To achieve the above objectives, the technical solution of the present invention is specifically implemented as follows:

[0007] One aspect of the present invention provides a method for detecting organ surface lesions based on a deformation field, comprising: acquiring a medical image sequence; segmenting a target organ based on an organ segmentation model; initializing a deformation field based on the segmentation result of the target organ to transform the distorted contour of the target organ into a straight line; determining the lesion contour; preprocessing the image; dividing the image into a training set and a test set; training the lesion detection model on the training set; validating the lesion detection model on the test set; and detecting organ surface lesions using the lesion detection model.

[0008] The step of initializing the deformation field based on the segmentation result of the target organ and transforming the distorted contour of the target organ into a straight line includes: obtaining the contour point set S of the organ boundary based on the segmentation result of the target organ; determining the deformation field, the shape of the deformation field being (H, W, 2), where the width W of the deformation field is |S|-2, and the height H of the deformation field is a user-defined hyperparameter; the correspondence described by the deformation field is iteratively calculated for each point in the contour point set S except for the beginning and end, and then sequentially spliced ​​along the W direction to obtain the complete initial deformation field Φ0.

[0009] The correspondence described by the deformation field is iteratively calculated for each point in the contour point set S, excluding the first and last points. This includes: calculating the mapping relationship for each point p2 in the point set, excluding the first and last points, using an equal-interval sampling algorithm passing through point p; wherein, the equal-interval sampling algorithm passing through point p includes: based on the point p2(x2,y2) to be calculated, finding two points before and after it, p1(x1,y1) and p3(x3,y3). Using p1 and p3, calculating the formula for the line L1 passing through points p1 and p3; calculating the formula for the line L2 passing through p2 and perpendicular to L1; and performing equal-interval sampling around p2 on line L2.

[0010] The steps of determining the lesion contour, preprocessing the image, dividing it into a training set and a test set, training the lesion detection model on the training set, and validating the lesion detection model on the test set include: determining the input of the lesion detection model, including the original image X1 and the lesion contour; deforming the original image X1 using the initial deformation field Φ0 to obtain a first deformed image X2; generating a deformation field Φ based on the deformation field network G, and further transforming the first deformed image X2 using the deformation field Φ to obtain a second deformed image X3; deforming the lesion contour using the deformation field Φ to obtain a deformed lesion contour; generating a lesion detection box based on the deformed lesion contour as a regression target; constructing a lesion detection model based on the second deformed image X3 and the regression target; and inputting the original image from the test set into the lesion detection model and outputting the lesion location to validate the lesion detection model.

[0011] The method also includes: initializing the deformation field network G.

[0012] The initialization of the deformation field network G includes: determining that the input is the second deformed image X2, the target is the initial deformation field Φ0, and the loss function is MSE loss; using the Adam optimizer to minimize the loss function, training the model until convergence, and selecting the model with the smallest loss function on the validation set as the trained deformation field network.

[0013] The step of generating a lesion detection box based on the deformed lesion contour includes: comparing the coordinates of each lesion contour point to obtain the minimum value X of the horizontal coordinate. min and maximum value X max The minimum value of the ordinate Y min and maximum value Y max According to the top left corner point (X) min ,Y min ), frame width X max -X min , frame height Y max -Y min Define the lesion detection frame.

[0014] The method further includes converting the lesion detection box into a lesion detection box on the original image.

[0015] Another aspect of the present invention provides an apparatus for detecting organ surface lesions based on deformation fields, comprising: an organ contour segmentation module for acquiring medical image sequences and segmenting a target organ based on an organ segmentation model; a deformation field initialization module for initializing the deformation field based on the segmentation results of the target organ, transforming the distorted contour of the target organ into a straight line; and a lesion detection model module based on deformation fields for determining lesion contours, preprocessing images, dividing the image into a training set and a test set, training the lesion detection model on the training set, validating the lesion detection model on the test set, and detecting organ surface lesions using the lesion detection model.

[0016] The deformation field initialization module initializes the deformation field based on the segmentation results of the target organ in the following way, transforming the distorted contour of the target organ into a straight line: Based on the segmentation results of the target organ, a set of contour points S representing the organ boundary is obtained; the deformation field is determined, with a shape of (H, W, 2), where the width W of the deformation field is |S|-2, and the height H of the deformation field is a user-defined hyperparameter; the correspondence described by the deformation field is iteratively calculated for each point in the contour point set S except for the beginning and end, and then sequentially spliced ​​along the W direction to obtain the complete initial deformation field Φ0.

[0017] The correspondence described by the deformation field is iteratively calculated for each point in the contour point set S, excluding the first and last points. This includes: calculating the mapping relationship for each point p2 in the point set, excluding the first and last points, using an equal-interval sampling algorithm passing through point p; wherein, the equal-interval sampling algorithm passing through point p includes: based on the point p2(x2,y2) to be calculated, finding two points before and after it, p1(x1,y1) and p3(x3,y3). Using p1 and p3, calculating the formula for the line L1 passing through points p1 and p3; calculating the formula for the line L2 passing through p2 and perpendicular to L1; and performing equal-interval sampling around p2 on line L2.

[0018] The lesion detection model module based on deformation field determines the lesion contour, preprocesses the image, divides it into training and test sets, trains the lesion detection model on the training set, and validates the lesion detection model on the test set as follows: The input to the lesion detection model is determined, including the original image X1 and the lesion contour; the original image X1 is deformed using the initial deformation field Φ0 to obtain a first deformed image X2; a deformation field Φ is generated based on the deformation field network G, and the first deformed image X2 is further transformed using the deformation field Φ to obtain a second deformed image X3; the lesion contour is deformed using the deformation field Φ to obtain a deformed lesion contour; a lesion detection box is generated based on the deformed lesion contour as a regression target; a lesion detection model is constructed based on the second deformed image X3 and the regression target; the original image from the test set is input into the lesion detection model, and the lesion location is output to validate the lesion detection model.

[0019] The lesion detection model module based on the deformation field is also used to initialize the deformation field network G.

[0020] The lesion detection model module based on the deformation field initializes the deformation field network G in the following way: the input is determined to be the second deformed image X2, the target is the initial deformation field Φ0, and the loss function is MSE loss; the Adam optimizer is used to minimize the loss function, the model is trained until convergence, and the model with the smallest loss function on the validation set is selected as the trained deformation field network.

[0021] The lesion detection model module based on the deformation field generates a lesion detection box based on the deformed lesion contour in the following way: comparing the coordinates of each lesion contour point to obtain the minimum value X of the horizontal coordinate. min and maximum value X max The minimum value of the ordinate Y min and maximum value Y max According to the top left corner point (X) min ,Y min), frame width X max -X min , frame height Y max -Y min Define the lesion detection frame.

[0022] The lesion detection model module based on deformation field is also used to convert the lesion detection box into a lesion detection box on the original image.

[0023] Therefore, the method and apparatus for detecting organ surface lesions based on deformation field provided by the present invention improve the performance of the lesion detection model on the organ surface, especially the detection performance of lesions with thin strip-shaped attachments on the organ surface. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating the method for detecting organ surface lesions based on deformation fields provided in this embodiment of the invention;

[0026] Figure 2 This is a schematic diagram of target organ segmentation provided in an embodiment of the present invention;

[0027] Figure 3 A schematic diagram showing the correspondence between the original image and the image deformed by the deformation field provided in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of the deformation field initialization sampling method provided in an embodiment of the present invention;

[0029] Figure 5 A flowchart of the detection model based on deformation field provided in an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of the device for detecting organ surface lesions based on deformation field, provided in an embodiment of the present invention. Detailed Implementation

[0031] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0032] Figure 1 A flowchart of the method for detecting organ surface lesions based on deformation fields provided in an embodiment of the present invention is shown. See also... Figure 1 The method for detecting organ surface lesions based on deformation fields provided in this embodiment of the invention includes:

[0033] S1: Acquire medical image sequences and segment the target organ based on the organ segmentation model.

[0034] Specifically, medical image sequences are collected, including sequences from modalities such as CT, MRI, and PET. The target organ is then segmented based on an organ segmentation model. For most organ contour segmentation tasks, there is already ample research and open-source models available. One alternative is to use tools such as TotalSegmentator, which can directly obtain the segmented organ contour. Figure 2 As shown.

[0035] S2, based on the segmentation results of the target organ, initializes the deformation field to transform the distorted outline of the target organ into a straight line.

[0036] Specifically, based on the segmentation results of the target organ, a deformation field is initialized. The goal of the deformation field is to transform the distorted contour of the target organ into a straight line, such as... Figure 3 As shown.

[0037] As an optional implementation of this invention, the deformation field initialization based on the segmentation results of the target organ, transforming the distorted target organ contour into a straight line, includes: obtaining the contour point set S of the organ boundary based on the segmentation results of the target organ; determining the deformation field, the shape of which is (H, W, 2), where the width W of the deformation field is |S|-2, and the height H of the deformation field is a user-defined hyperparameter; the correspondence described by the deformation field is iteratively calculated for each point in the contour point set S except for the beginning and end, and then sequentially spliced ​​along the W direction to obtain the complete initial deformation field Φ0.

[0038] Specifically, the deformation field is a two-channel image, i.e., a three-dimensional matrix Φ of shape (H,W,2) (H and W are the height and width of the deformed image X2. Two-channel means the last dimension of the matrix is ​​2). Let the original image be X1. Then, for a point with coordinates x,y on the deformed image, the corresponding pixel position in the original image is [Φ[x,y,0],Φ[x,y,1]]. In formulaic terms:

[0039] X2(x,y)=X1(Φ(x,y,0),Φ(x,y,1))

[0040] It should be noted that Φ(x,y,0) and Φ(x,y,1) are not necessarily integers; they can be floating-point numbers. In this case, X2(x,y) is obtained through interpolation. Common interpolation methods include nearest neighbor, linear interpolation, bilinear interpolation, and bicubic interpolation.

[0041] As an optional implementation of this invention, the correspondence described by the deformation field is iteratively calculated for each point in the contour point set S, excluding the first and last points. This includes: calculating the mapping relationship for each point p2 in the point set, excluding the first and last points, using a sampling algorithm with equal intervals through point p; wherein, the sampling algorithm with equal intervals through point p includes: finding two points before and after the point p2(x2,y2) to be calculated, p1(x1,y1) and p3(x3,y3). Using p1 and p3, calculate the formula for the line L1 passing through points p1 and p3; calculate the formula for the line L2 passing through p2 and perpendicular to L1; and perform equal-interval sampling around p2 on line L2.

[0042] Specifically, the method for calculating the initial deformation field in the embodiments of the present invention will be described below:

[0043] To obtain the initial deformation field Φ0, firstly, the contour point set S of the organ boundary is obtained based on the organ segmentation results. The width W of the deformation field is |S|-2. The height H of the initial deformation field is a custom hyperparameter that can be adjusted according to project requirements; in this example, it is set to 512. During the initialization process, the correspondence described by the deformation field is iteratively calculated for each point in the contour point set S except for the first and last points, and then sequentially spliced ​​along the W direction to obtain the complete initial deformation field.

[0044] It should be noted that the contour point set S of the organ boundary is a continuous contour without a starting point. To ensure the uniqueness of the deformation field generation result, a starting point rule needs to be specified. This example uses the point with the largest x-coordinate in the contour point set S as the starting point.

[0045] In the specific implementation, the deformation field calculation based on PyTorch requires additional normalization to values ​​between (-1, 1), which represent the proportion of the coordinates in that direction. The original image has width and height W' and H', respectively:

[0046] Φ(x,y,0)=2*Φ(x,y,0) / (W'-1)-1.0

[0047] Φ(x,y,1)=2*Φ(x,y,1) / (H'-1)-1.0

[0048]

[0049]

[0050] The following describes the sampling algorithm with equal intervals through point p provided in the embodiments of the present invention:

[0051] For each point p2(x2,y2) in the point set except for the first and last points, and the two points before and after it, p1(x1,y1) and p3(x3,y3), calculate the mapping relationship, such as... Figure 4 As shown:

[0052] ①The formula for calculating the line L1 passing through points p1 and p3, given p1 and p3.

[0053] ② Formula for calculating the line L2 that passes through p2 and is perpendicular to L1.

[0054] ③ On line L2, perform equally spaced sampling around p2. The sampling x-coordinate interval is the hyperparameter a, and the number of samples is H. If a sampling point exceeds the boundary of the original image, the boundary point is used as the sampling point. It should be noted that the coordinates of the sampling points are not necessarily integers; they can be floating-point numbers.

[0055] From a simple derivation, we can see that the formula for L2 is:

[0056]

[0057] Let the slope of L2 be k, then the sampling points are:

[0058] Points=[(x2-a*n,y2-a*k*n),(x2-a*(n-1),y2-a*(k*(n-1)),...,(x2,y2),...,

[0059] (x²-a*(n-1),y²-a*k*(n-1)),(x²+a*n,y²+a*k*n)], n=floor(H / 2), when H is odd. floor is the floor function.

[0060] or

[0061] Points=[(x2-a*(n-1),y2-a*k*(n-1)),...,(x2,y2),...,(x2-a*(n-1),

[0062] y2-a*k*(n-1)),(x2+a*n,y2+a*k*n)],n=floor(H / 2), when H is an even number.

[0063] S3. Determine the lesion outline, preprocess the image, divide it into training and test sets, train the lesion detection model on the training set, and validate the lesion detection model on the test set.

[0064] S4, using a lesion detection model to detect lesions on the surface of organs.

[0065] Specifically, based on the collected medical image sequences, radiologists can outline the lesions. After outlining, the images undergo preprocessing, including window width and level adjustment, normalization, etc. The model is then divided into training and testing sets proportionally; it is trained on the training set and validated on the testing set.

[0066] As an optional implementation of this invention, determining the lesion contour, preprocessing the image, dividing it into a training set and a test set, training the lesion detection model on the training set, and validating the lesion detection model on the test set includes: determining the input of the lesion detection model, including the original image X1 and the lesion contour; deforming the original image X1 using an initial deformation field Φ0 to obtain a first deformed image X2; generating a deformation field Φ based on the deformation field network G, and further transforming the first deformed image X2 using the deformation field Φ to obtain a second deformed image X3; deforming the lesion contour using the deformation field Φ to obtain a deformed lesion contour; generating a lesion detection box based on the deformed lesion contour as a regression target; constructing a lesion detection model based on the second deformed image X3 and the regression target; inputting the original image from the test set into the lesion detection model, and outputting the lesion location to validate the lesion detection model.

[0067] Specifically, the detection model based on the deformation field is as follows: Figure 5 As shown. During model training, the model input includes the original image X1 and the lesion contours annotated by the doctor. After X1 is deformed by the initial deformation field Φ0, the deformed image X2 is obtained. To fine-tune the deformation result, a deformation field Φ is generated based on the deformation field network G, which further transforms X2 to obtain X3. At the same time, the deformation field Φ also deforms the lesion contours annotated by the doctor. Based on the deformed lesion contours, lesion detection boxes are generated as regression targets. Based on the deformed image X3 and the regression target, a target detection model is constructed. During the model inference phase, the model inputs the original image X1 and outputs the lesion location. The network structure includes the deformation field network G and the detection model.

[0068] As an optional implementation of this invention, the method for detecting organ surface lesions based on deformation fields provided in this invention further includes: initializing a deformation field network G. Initializing the deformation field network G includes: determining that the input is the second deformed image X2, the target is the initial deformation field Φ0, and the loss function is MSE loss; using the Adam optimizer to minimize the loss function, training the model until convergence, and selecting the model with the smallest loss function on the validation set as the trained deformation field network.

[0069] Specifically, first, the deformation field network G is initialized. The input is the deformed image X2, and the objective is to initialize the deformation field Φ0. The loss function is MSE loss. The Adam optimizer is used to minimize the loss function, and the model is trained until convergence. The model with the smallest loss function on the validation set is selected as the trained deformation field network. To improve the model's generalization ability, data augmentation is performed on the input image data, including image flipping, 360-degree rotation, and random scaling.

[0070] Of course, the deformable field network mentioned in this invention can be replaced by an equivalent network structure, such as UNet, VNet, UNet++, etc.

[0071] As an optional implementation of this invention, generating a lesion detection box based on the deformed lesion contour includes:

[0072] By comparing the coordinates of each lesion contour point, the minimum value of the x-coordinate, X, is obtained. min and maximum value X max The minimum value of the ordinate Y min and maximum value Y max According to the top left corner point (X) min ,Y min ), frame width X max -X min , frame height Y max -Y min Define the detection frame for lesions.

[0073] Specifically, the marked lesion contour is deformed using a deformation field to obtain the deformed lesion contour. The bounding box of the deformed lesion contour is then calculated. Specifically, the coordinates of each lesion contour point are compared to obtain the minimum and maximum values ​​of the x-coordinate: X. min and X max Minimum and maximum values ​​of the y-axis: Y min and Y max The bounding box is defined by the top-left corner point: (X min ,Y min ) and frame width and height X max -X min Y max -Y min Sure.

[0074] The following describes the training of the lesion detection model provided in the embodiments of the present invention:

[0075] A lesion detection model is trained based on the deformed image and bounding box. As a feasible example, this invention demonstrates a lesion detection model based on the Faster R-CNN network structure, optimized using L1 loss and BCE loss. This model can also be equivalently replaced by other single-stage or two-stage detection models, and the loss function can be replaced by IOU Loss, FocalLoss, etc.

[0076] As an optional embodiment of the present invention, the method for detecting organ surface lesions based on deformation field provided by the present invention further includes: converting the lesion detection box into a lesion detection box on the original image.

[0077] Specifically, converting the lesion detection bounding box to the lesion detection bounding box on the original image is a process of converting the detection box position on the original image. During the model inference stage, the bounding box predicted by the model is the coordinate on the deformed image. To obtain the bounding box of the predicted result on the original image, it is necessary to combine the deformed network for conversion. The specific method is:

[0078]

[0079]

[0080] Therefore, the method for detecting organ surface lesions based on deformation fields provided in this invention proposes a scheme to detect organ surface lesions by fully utilizing prior knowledge of organ contours through deformation fields. This scheme achieves lesion detection through an organ contour segmentation model, deformation field initialization, and a lesion detection model based on deformation fields. Furthermore, it provides a method for initializing the deformation field based on the target organ segmentation results, offering a scheme for constructing the deformation field from the target organ segmentation results. Simultaneously, this invention proposes a lesion detection model based on deformation fields, which utilizes prior knowledge of deformation fields to deform the original image for detecting organ surface lesions. Finally, this invention provides a scheme for converting the deformed detection results back to the original image coordinates.

[0081] As can be seen, the deformation field construction scheme of this invention fully utilizes the contour information of the target organ, which helps to improve the efficiency of lesion detection on the organ surface. After deformation, the background noise within the detection box of the lesion can be effectively reduced, allowing the model to focus on the features of the lesion itself. In the target detection stage, this invention further performs deformation fine-tuning on the image to optimize the error caused by segmentation and improve the efficiency of the model.

[0082] Figure 6This diagram illustrates the structure of a device for detecting organ surface lesions based on deformation fields, as provided in an embodiment of the present invention. This device applies the aforementioned method. The following is only a brief description of the structure of the device; for other matters not covered herein, please refer to the relevant descriptions in the aforementioned method for detecting organ surface lesions based on deformation fields. Figure 3 The device for detecting organ surface lesions based on deformation field provided in this embodiment of the invention includes:

[0083] The organ contour segmentation module is used to acquire medical image sequences and segment target organs based on the organ segmentation model.

[0084] The deformation field initialization module is used to initialize the deformation field based on the segmentation results of the target organ, transforming the distorted outline of the target organ into a straight line;

[0085] The lesion detection model module based on deformation field is used to determine the lesion outline, preprocess the image, divide it into training set and test set, train the lesion detection model on the training set, and validate the lesion detection model on the test set; and use the lesion detection model to detect lesions on the surface of organs.

[0086] As an optional implementation of this invention, the deformation field initialization module initializes the deformation field based on the segmentation results of the target organ in the following manner, transforming the distorted target organ contour into a straight line: obtaining the contour point set S of the organ boundary based on the segmentation results of the target organ; determining the deformation field, the shape of the deformation field being (H, W, 2), where the width W of the deformation field is |S|-2, and the height H of the deformation field is a user-defined hyperparameter; the correspondence described by the deformation field is iteratively calculated for each point in the contour point set S except for the beginning and end, and sequentially spliced ​​along the W direction to obtain the complete initial deformation field Φ0.

[0087] As an optional implementation of this invention, the correspondence described by the deformation field is iteratively calculated for each point in the contour point set S, excluding the first and last points. This includes: calculating the mapping relationship for each point p2 in the point set, excluding the first and last points, using a sampling algorithm with equal intervals through point p; wherein, the sampling algorithm with equal intervals through point p includes: finding two points before and after the point p2(x2,y2) to be calculated, p1(x1,y1) and p3(x3,y3). Using p1 and p3, calculate the formula for the line L1 passing through points p1 and p3; calculate the formula for the line L2 passing through p2 and perpendicular to L1; and perform equal-interval sampling around p2 on line L2.

[0088] As an optional implementation of this invention, the lesion detection model module based on deformation field determines the lesion contour in the following way: preprocessing the image, dividing it into a training set and a test set, training the lesion detection model on the training set, and validating the lesion detection model on the test set: determining the input of the lesion detection model, including the original image X1 and the lesion contour; deforming the original image X1 using an initial deformation field Φ0 to obtain a first deformed image X2; generating a deformation field Φ based on the deformation field network G, and further transforming the first deformed image X2 using the deformation field Φ to obtain a second deformed image X3; deforming the lesion contour using the deformation field Φ to obtain a deformed lesion contour; generating a lesion detection box based on the deformed lesion contour as a regression target; constructing a lesion detection model based on the second deformed image X3 and the regression target; inputting the original image from the test set into the lesion detection model, and outputting the lesion location to validate the lesion detection model.

[0089] As an optional implementation of this invention, the lesion detection model based on the deformation field is also used to initialize the deformation field network G.

[0090] As an optional implementation of this invention, the lesion detection model based on the deformation field initializes the deformation field network G in the following manner: the input is determined to be the second deformed image X2, the target is to initialize the deformation field Φ0, and the loss function is MSE loss; the Adam optimizer is used to minimize the loss function, the model is trained until convergence, and the model with the smallest loss function on the validation set is selected as the trained deformation field network.

[0091] As an optional implementation of this invention, the lesion detection model based on the deformation field generates a lesion detection box based on the deformed lesion contour in the following way: comparing the coordinates of each lesion contour point to obtain the minimum value X of the horizontal coordinate. min and maximum value X max The minimum value of the ordinate Y min and maximum value Y max According to the top left corner point (X) min ,Y min ), frame width X max -X min , frame height Y max -Y min Define the detection frame for lesions.

[0092] Therefore, the organ surface lesion detection device based on deformation field provided in this invention proposes a scheme for detecting organ surface lesions by fully utilizing prior knowledge of organ contours through deformation field. This scheme achieves lesion detection through an organ contour segmentation model, deformation field initialization, and a lesion detection model based on deformation field. Furthermore, it provides a method for initializing the deformation field based on the target organ segmentation results, offering a scheme for constructing the deformation field from the target organ segmentation results. Simultaneously, this invention proposes a lesion detection model based on deformation field, which utilizes prior knowledge of deformation field to deform the original image for organ surface lesion detection. Finally, this invention provides a scheme for converting the deformed detection results back to the original image coordinates.

[0093] As can be seen, the deformation field construction scheme of this invention fully utilizes the contour information of the target organ, which helps to improve the efficiency of lesion detection on the organ surface. After deformation, the background noise within the detection box of the lesion can be effectively reduced, allowing the model to focus on the features of the lesion itself. In the target detection stage, this invention further performs deformation fine-tuning on the image to optimize the error caused by segmentation and improve the efficiency of the model.

[0094] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting organ surface lesions based on deformation fields, characterized in that, include: Acquire medical image sequences and segment the target organ based on an organ segmentation model; Based on the segmentation results of the target organ, a deformation field is initialized to transform the distorted outline of the target organ into a straight line; The lesion outline is determined, the image is preprocessed, and a training set and a test set are divided. The lesion detection model is trained on the training set and validated on the test set. The lesion detection model was used to detect lesions on the surface of organs; in: The step of initializing the deformation field based on the segmentation results of the target organ, and transforming the distorted contour of the target organ into a straight line, includes: Based on the segmentation results of the target organ, the contour point set S of the organ boundary is obtained; A deformation field is defined, the shape of which is (H,W,2), where the width W of the deformation field is |S|-2, and the height H of the deformation field is a user-defined hyperparameter; wherein, the deformation field is a two-channel image, a three-dimensional matrix Φ of shape (H,W,2), where the two channels are represented by the last dimension of the matrix being 2; The correspondence described by the deformation field is iteratively calculated for each point in the contour point set S except for the beginning and end, and then sequentially spliced ​​along the W direction to obtain the complete initial deformation field Φ0.

2. The method according to claim 1, characterized in that, The correspondence described by the deformation field is iteratively calculated for each point in the contour point set S, excluding the beginning and end, including: For each point p2 in the point set except for the first and last points, the mapping relationship is calculated using the equal-interval sampling algorithm passing through point p. The equal-interval sampling algorithm passing through point p includes: based on the point p2(x2,y2) to be calculated, finding two points before and after it, p1(x1,y1) and p3(x3,y3), and calculating the formula of the line L1 passing through points p1 and p3 using p1 and p3; calculating the formula of the line L2 passing through p2 and perpendicular to L1; and performing equal-interval sampling around p2 on the line L2.

3. The method according to claim 2, characterized in that, The steps of determining the lesion contour, preprocessing the image, dividing it into a training set and a test set, training the lesion detection model on the training set, and validating the lesion detection model on the test set include: The input to the lesion detection model is determined, including the original image X1 and the lesion outline; The original image X1 is deformed using the initial deformation field Φ0 to obtain the first deformed image X2. A deformation field Φ is generated based on the deformation field network G. The first deformed image X2 is further transformed using the deformation field Φ to obtain the second deformed image X3. The lesion contour is deformed using the deformation field Φ to obtain the deformed lesion contour. Based on the deformed lesion contour, a lesion detection box is generated as the target for regression. Based on the second deformed image X3 and the regression target, a lesion detection model is constructed. The original images from the test set are input into the lesion detection model, and the lesion locations are output to verify the lesion detection model.

4. The method according to claim 3, characterized in that, Also includes: Initialize the deformation field network G.

5. The method according to claim 4, characterized in that, The initialized deformation field network G includes: The input is determined to be the second deformed image X2, the target is the initial deformation field Φ0, and the loss function is MSE loss. The Adam optimizer is used to minimize the loss function, and the model is trained until convergence. The model with the smallest loss function on the validation set is selected as the trained deformation field network.

6. The method according to claim 3, characterized in that, The step of generating a lesion detection box based on the deformed lesion contour includes: By comparing the coordinates of each lesion contour point, the minimum value of the x-coordinate, X, is obtained. min and maximum value X max The minimum value of the ordinate Y min and maximum value Y max ; Based on the top left corner point (X) min ,Y max ), frame width X max -X min , frame height Y max -Y min Define the lesion detection frame.

7. The method according to claim 6, characterized in that, Also includes: The lesion detection box is converted into a lesion detection box on the original image.

8. A device for detecting organ surface lesions based on deformation fields, characterized in that, include: The organ contour segmentation module is used to acquire medical image sequences and segment target organs based on the organ segmentation model. The deformation field initialization module is used to initialize the deformation field based on the segmentation results of the target organ, transforming the distorted contour of the target organ into a straight line; The lesion detection model module based on deformation field is used to determine the lesion outline, preprocess the image, divide it into training set and test set, train the lesion detection model on the training set, and verify the lesion detection model on the test set; and use the lesion detection model to detect lesions on the surface of organs. in: The deformation field initialization module initializes the deformation field based on the segmentation results of the target organ in the following manner, transforming the distorted contour of the target organ into a straight line: Based on the segmentation results of the target organ, the contour point set S of the organ boundary is obtained; A deformation field is defined, the shape of which is (H,W,2), where the width W of the deformation field is |S|-2, and the height H of the deformation field is a user-defined hyperparameter; wherein, the deformation field is a two-channel image, a three-dimensional matrix Φ of shape (H,W,2), where the two channels are represented by the last dimension of the matrix being 2; The correspondence described by the deformation field is iteratively calculated for each point in the contour point set S except for the beginning and end, and then sequentially spliced ​​along the W direction to obtain the complete initial deformation field Φ0.

9. The apparatus according to claim 8, characterized in that, The correspondence described by the deformation field is iteratively calculated for each point in the contour point set S, excluding the beginning and end, including: For each point p2 in the point set except for the first and last points, the mapping relationship is calculated using the equal-interval sampling algorithm passing through point p. The equal-interval sampling algorithm passing through point p includes: based on the point p2(x2,y2) to be calculated, finding two points before and after it, p1(x1,y1) and p3(x3,y3), and calculating the formula of the line L1 passing through points p1 and p3 using p1 and p3; calculating the formula of the line L2 passing through p2 and perpendicular to L1; and performing equal-interval sampling around p2 on the line L2.

10. The apparatus according to claim 9, characterized in that, The lesion detection model module based on deformation field determines the lesion contour in the following way: preprocesses the image, divides it into a training set and a test set, trains the lesion detection model on the training set, and validates the lesion detection model on the test set: The input to the lesion detection model is determined, including the original image X1 and the lesion outline; The original image X1 is deformed using the initial deformation field Φ0 to obtain the first deformed image X2. A deformation field Φ is generated based on the deformation field network G. The first deformed image X2 is further transformed using the deformation field Φ to obtain the second deformed image X3. The lesion contour is deformed using the deformation field Φ to obtain the deformed lesion contour. Based on the deformed lesion contour, a lesion detection box is generated as the target for regression. Based on the second deformed image X3 and the regression target, a lesion detection model is constructed. The original images from the test set are input into the lesion detection model, and the lesion locations are output to verify the lesion detection model.

11. The apparatus according to claim 10, characterized in that, The lesion detection model module based on the deformation field is also used to initialize the deformation field network G.

12. The apparatus according to claim 11, characterized in that, The lesion detection model module based on the deformation field initializes the deformation field network G in the following manner: The input is determined to be the second deformed image X2, the target is the initial deformation field Φ0, and the loss function is MSE loss. The Adam optimizer is used to minimize the loss function, and the model is trained until convergence. The model with the smallest loss function on the validation set is selected as the trained deformation field network.

13. The apparatus according to claim 10, characterized in that, The lesion detection model module based on the deformation field generates a lesion detection box based on the deformed lesion contour: By comparing the coordinates of each lesion contour point, the minimum value of the x-coordinate, X, is obtained. min and maximum value X max The minimum value of the ordinate Y min and maximum value Y max ; Based on the top left corner point (X) min ,Y max ), frame width X max -X min , frame height Y max -Y min Define the lesion detection frame.

14. The apparatus according to claim 13, characterized in that, The lesion detection model module based on deformation field is also used to convert the lesion detection box into a lesion detection box on the original image.

Images